Commit Graph
40 Commits
Author SHA1 Message Date
Kyle AltendorfandGitHub 65dfbbb6aa add Service type variables for each combination (#16970) 2023-11-30 13:38:38 -06:00
Kyle AltendorfandGitHub fbdb27bef1 managers for services (#16670) 2023-11-16 19:21:12 -06:00
Kyle AltendorfandGitHub 5484024ef3 move time_out_assert() family back to tests (#16846) 2023-11-16 11:20:18 -06:00
Arvid NorbergandGitHub 038f5e8fb7 Fix deep reorgs (#16594) 2023-11-15 10:51:04 -06:00
Arvid NorbergandGitHub 95e5b97557 Use bls from chia rs (#16715)
* replace blspy imports with chia_rs imports for BLS types

* remove blspy-stubs, since we're dropping the blspy dependency. chia_rs has type stubs already
2023-11-07 09:06:52 -08:00
b1bdbc40ab use anyio for tests (#16738)
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>
2023-11-01 13:04:27 -05:00
Kyle AltendorfandGitHub 90c1441af0 apply pyupgrade --py38-plus --keep-runtime-typing (#14646) 2023-10-24 12:01:23 -05:00
Amine KhaldiandGitHub b06ea0c6e5 Avoid setting test config items to their default values (#16225) 2023-09-19 08:36:26 -05:00
Kyle AltendorfandGitHub cd9d6eff33 add SizedBytes.random()/.secret(), replace token_bytes() usage (#16276) 2023-09-14 14:08:01 -05:00
Arvid NorbergandGitHub 57d1974ef3 fix flake8 warnings (#15897) 2023-07-31 09:46:49 -05:00
Izumi HoshinoandGitHub 85d14f561a Added compression level and harvesting mode to harvester protocol/mes… (#15776)
* Added compression level and harvesting mode to harvester protocol/messages

* Added test

* Fixed lint error
2023-07-18 16:24:56 -05:00
Izumi HoshinoandGitHub 1f9081a225 Added total_effective_plot_size for get_harvesters RPC API (#15770)
* Added `total_effective_plot_size` for `get_harvesters` RPC API

* Fixed lint error

* Change the type of `total_effective_plot_size` in Receiver to `int`

* Fixed lint error

* Fixed lint error 2

* Changed text

* Added test

* Fixed lint error

* Fixed lint error 2

* Fixed a test issue

* Added config.py to tests/cmds
2023-07-18 12:36:12 -05:00
49140b2b3c server: Introduce ApiProtocol (#15466)
* server: Introduce `ApiProtocol`

* genericize (#5)

* `ApiProtocol.api_ready` -> `ApiProtocol.ready()`

* Add `ApiProtocol.log` and give APIs separate loggers

* Fix `CrawlerAPI`

* Drop some unrelated removals

* Fix some of the generic hinting

* Revert some changes in `timelord_api.py`

* Fix `CawlerAPI` readiness

* Fix hinting

* Get some `CrawlerAPI` coverage

---------

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2023-06-13 20:12:25 -05:00
dustinfaceandGitHub ebc5f3c124 util: Improve list_to_batches (#15415)
* Make `list_to_batches` work with collections instead of lists only

* Move `to_batches` into `chia.util.misc`

* Only support `set` and `list`

* Drop `tests.generator.test_to_batches` exclusion

* Improve type restrictions and be more coverage friendly in tests
2023-06-06 12:00:07 -05:00
Amine KhaldiandGitHub d892ed2c97 Fix Farmer imports in plot sync tests (#15346)
Fix Farmer imports in plot sync tests.
2023-05-22 17:37:34 -05:00
dustinfaceandGitHub a7e7b9a290 server: Rework config peer resolving and connection handling (#14997)
merging as the snyk issue is confirmed a false positive
2023-04-14 15:22:24 -06:00
dustinfaceandGitHub f0006b5353 server: peer_host + peer_port -> peer_info in WSChiaConnection (#14151) 2023-04-12 11:14:38 -06:00
Kyle AltendorfandGitHub a42bf49050 catch up with mypy 1.1.1 (#14767) 2023-03-07 11:34:07 -06:00
Kyle AltendorfandGitHub 3a5c1a5bd5 hint chia.harvester.harvester, harvester_api (#14690)
* hint chia.harvester.harvester

* hint chia.harvester.harvester_api
2023-02-28 11:42:58 -06:00
dustinfaceandGitHub a40f04012b server: Replace WSChiaConnection.__getattr__ with call_api (#14052) 2022-12-13 00:35:42 -06:00
2706f5995b server: Enable and fix mypy in ws_connection.py (#13878)
* server: Enable and fix `mypy` in `ws_connection.py`

* Apply suggestions from code review

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Tweak error message

* Tweak formatting

* Make `WSChiaConnection.close_callback` optional

* Tweak assert message

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Don't provide a default for `close_callback`

* Adjust assertion

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-11-12 14:18:11 -06:00
Kyle AltendorfandGitHub 3b084a165b configure isort to add the future annotations import (#13327)
* configure isort to add the future annotations import

* apply the new isort setting

* remove type ignores for new mypy (#13539)

https://pypi.org/project/mypy/0.981/

* another
2022-09-30 03:40:22 -05:00
Kyle AltendorfandGitHub 1dbf575a3e RpcApiProtocol and RpcServiceProtocol (#11522)
* provide a skeleton hint protocol for rpc apis

* more

* more self.custom_get_connections = None

* fix RpcApiProtocol.service hinting

* catch up with RpcApiProtocol._state_changed()

* FullNode.server is now non-optional.  'ish

* drop another todo

* custom_get_connections -> get_connections

* just make get_connections() a mandatory method with a helper provided

* lots

* try again

* ._start_with_fingerprint()

* fill out the Service generic parameter in start scripts

* ... instead of pass

* add docstrings to the protocols

* remove no longer needed hasattr/not None checks

* remove outdated comment

* refactor Harvester._is_shutdown to ._shut_down
2022-09-21 22:49:01 -05:00
dustinfaceandGitHub d9b271dbfa tests: Drop redundant expected_state parameter of run_sync_step (#12398)
The expected state is already present in `sync_step`.
2022-08-15 12:23:08 -05:00
dustinfaceandGitHub 7c9fb654a3 server|tests: Delay the harvester -> farmer connection in some tests (#12862)
* Introduce `Service.add_peer`

* Delay the harvester/farmer connection in some tests to avoid flakiness
2022-08-10 04:57:34 -05:00
Mariano SorgenteandGitHub 4c7a917905 Increase timeouts to reduce flakiness (#12745) 2022-08-03 00:01:39 -05:00
Jack NelsonandGitHub 0bc54ae08b Move simulator test requirements over to new module (#12521)
* create new test.simulator module

to allow other packages to use the simulator without it crashing.
also tiny QOL fixes

* change user mode sim plot filter

Now only plots matching the user set fingerprint are loaded, this decreases the chance of user error.

* fix small oops in configure

* oops

* fix lint

* isort, who dis

* move folder over to chia/simulator

also change mypy settings again
2022-07-25 14:07:38 -05:00
Jack NelsonandGitHub bd5d4b1392 remove extra bt from as many tests as possible ( speed up tests) (#12290)
* remove extra bt from as many tests as possible

max speed

* wallet bt tools removal

* fix error & add bt_tools to yield

* final changes

L bozo

* something funky going on

* fix issues

* a couple more fixes

* mypy happy

* damn lint

* Revert "something funky going on"

This reverts commit 1fcc9a34e1.
2022-07-13 03:57:32 -05:00
dustinfaceandGitHub 9b3bed885e harvester: Make PlotsRefreshParameter streamable + use from_json_dict (#11759)
* Make `PlotsRefreshParameter` streamable

* Use `from_json_dict` instead of `dataclass_from_dict`

* Fix `test_plot_manager.py`
2022-06-03 12:35:50 -05:00
dustinfaceandGitHub bc98e6c5d8 tests: Some renaming in plot manager tests to avoid pytest conflicts (#9692)
Having `Test` or `test_` prefixes obviously lets `pytest` think its a
actual test class/method and it
leads to warnings in test logs if it doesn't comply with its rules:

```
/home/runner/work/chia-blockchain/chia-blockchain/tests/plotting/test_plot_manager.py:45:
PytestCollectionWarning: cannot collect test class 'TestDirectory'
because it has a __init__ constructor (from:
plotting/test_plot_manager.py)
```
or
```
/home/runner/work/chia-blockchain/chia-blockchain/tests/plotting/test_plot_manager.py:135:
PytestCollectionWarning: cannot collect test class 'TestEnvironment'
because it has a __init__ constructor (from:
plotting/test_plot_manager.py)
```
2022-05-31 13:42:11 -05:00
dustinfaceandGitHub 7142ebcae7 tests: Consolidate farmer/harvester fixtures (#11513)
* tests: Consolidate farmer/harvester fixtures

* Fixup after #11578
2022-05-25 15:15:49 -07:00
dustinfaceandGitHub c400d816af harvester: Validate the path in add_plot_directory (#11625)
Make sure it exists, is a directory and hasn't been added already.
2022-05-24 13:21:44 -07:00
Kyle AltendorfandGitHub a9f7cce06b Skip plot_sync_callback if delta is None, hint Harvester.server (#11620)
* hint Harvester.server as Optional[ChiaServer]

* use a Protocol instead of Callable

* oh right, Optional
2022-05-24 13:21:14 -07:00
dustinfaceandGitHub fc2d6d3db0 harvester: Fix deadlock on disconnect after a repeated plot sync start (#11481)
* Test plot sync start and disconnect while a sync is in progress

* Drop `Receiver._lock`, instead wait for `Receiver.sync_active` to clear

* Properly stop the sender and plot manager refreshing on disconnect

* Poll faster, drop the log

* Reset `_sync_id` at the very end of `_finalize_sync`
2022-05-12 15:16:22 -07:00
dustinfaceandGitHub 2d0d8e175e test: Missing duplicates tests in test_receiver.py (#11413) 2022-05-05 08:16:33 -07:00
dustinfaceandGitHub 7ed610bc84 test: Give setup_farmer and setup_harvester a separate chia root (#11408)
* test: Give `setup_farmer` and `setup_harvester` a separate chia root

* test: Wait for `last_sync_time` in `get_harvesters_{summary}`

To make sure the first sync from the harvester to the farmer is done 
before we check plot counts.
2022-05-02 19:32:37 -07:00
dustinfaceandGitHub 387abc3d8e farmer|rpc: Some changes to get_harvesters{_summary} RPC endpoints (#11342)
* farmer|rpc: Always add `last_sync_time` in `Receiver.to_dict`

This changes the resonse of `get_harvesters` and
`get_harvesters_summary` to always include that field with value
`None`/`null` if the receiver is not yet synced.

* plot_sync: Add sync state to dict in `Receiver.to_dict`

* Add `total_plot_size` to `Receiver.to_dict`

* Refactor sync object assertions

* Access `_current_sync` directly

* Generate `syncing` data outside
2022-04-29 18:34:14 -07:00
270c69eb5d plot_sync: Introduce receiver.Sync (#11267)
* plot_sync: Introduce `receiver.Sync`

* Use `dataclasses.replace`

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-04-29 08:16:39 -07:00
dustinfaceandGitHub b693aeb407 farmer|rpc: Introduce get_harvesters_summary RPC endpoint (#11245) 2022-04-28 19:36:21 -07:00
ded9f68583 chia|tests|github: Implement, integrate and test plot sync protocol (#9695)
* protocols|server: Define new harvester plot refreshing protocol messages

* protocols: Bump `protocol_version` to `0.0.34`

* tests: Introduce `setup_farmer_multi_harvester`

Allows to run a test setup with 1 farmer and mutiple harvesters.

* plotting: Add an initial plot loading indication to `PlotManager`

* plotting|tests: Don't add removed duplicates to `total_result.removed`

`PlotRefreshResult.removed` should only contain plots that were loaded
properly before they were removed. It shouldn't contain e.g. removed
duplicates or invalid plots since those are synced in an extra sync step
and not as diff but as whole list every time.

* harvester: Reset `PlotManager` on shutdown

* plot_sync: Implement plot sync protocol

* farmer|harvester: Integrate and enable plot sync

* tests: Implement tests for the plot sync protocol

* farmer|tests: Drop obsolete harvester caching code

* setup: Add `chia.plot_sync` to packages

* plot_sync: Type hints in `DeltaType`

* plot_sync: Drop parameters in `super()` calls

* plot_sync: Introduce `send_response` helper in `Receiver._process`

* plot_sync: Add some parentheses

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* plot_sync: Additional hint for a `Receiver.process_path_list` parameter

* plot_sync: Force named parameters in `Receiver.process_path_list`

* test: Fix fixtures after rebase

* tests: Fix sorting after rebase

* tests: Return type hint for `plot_sync_setup`

* tests: Rename `WSChiaConnection` and move it in the outer scope

* tests|plot_sync: More type hints

* tests: Rework some delta tests

* tests: Drop a `range` and iterate over the list directly

* tests: Use the proper flags to overwrite

* test: More missing duplicates tests

* tests: Drop `ExpectedResult.reset`

* tests: Reduce some asserts

* tests: Add messages to some `assert False` statements

* tests: Introduce `ErrorSimulation` enum in `test_sync_simulated.py`

* tests: Use `secrects` instead of `Crypto.Random`

* Fixes after rebase

* Import from `typing_extensions` to support python 3.7

* Drop task name to support python 3.7

* Introduce `Sender.syncing`, `Sender.connected` and a log about the task

* Add `tests/plot_sync/config.py`

* Align the multi harvester fixture with what we do in other places

* Update the workflows

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-04-07 17:10:44 -07:00