Commit Graph
28 Commits
Author SHA1 Message Date
Izumi HoshinoandGitHub 85d14f561a Added compression level and harvesting mode to harvester protocol/mes… (#15776)
* Added compression level and harvesting mode to harvester protocol/messages

* Added test

* Fixed lint error
2023-07-18 16:24:56 -05:00
Kyle AltendorfandGitHub 3b084a165b configure isort to add the future annotations import (#13327)
* configure isort to add the future annotations import

* apply the new isort setting

* remove type ignores for new mypy (#13539)

https://pypi.org/project/mypy/0.981/

* another
2022-09-30 03:40:22 -05:00
Kyle AltendorfandGitHub bbd032e1c9 embrace click 8 (#13559)
* click==8.1.3, black==22.8.0

* black

* click option creator hinting

* ignores for to-be-resolved issue

* remove unused import
2022-09-29 11:24:36 -05:00
dustinfaceandGitHub 0df4d8cbe4 setup|plotting: chiapos==1.0.11 + drop invalid cache entries (#13502)
* Bump `chiapos` to `1.0.11`

* Drop invalid cache entries
2022-09-23 17:30:30 -05:00
Jack NelsonandGitHub 0bc54ae08b Move simulator test requirements over to new module (#12521)
* create new test.simulator module

to allow other packages to use the simulator without it crashing.
also tiny QOL fixes

* change user mode sim plot filter

Now only plots matching the user set fingerprint are loaded, this decreases the chance of user error.

* fix small oops in configure

* oops

* fix lint

* isort, who dis

* move folder over to chia/simulator

also change mypy settings again
2022-07-25 14:07:38 -05:00
Kyle AltendorfandGitHub 90516263aa remove chia.util.path.mkdir() (#10842)
* remove chia.util.path.mkdir()

* remove submodule change

* less tmpdir, more tmp_path
2022-06-08 10:24:40 -05:00
dustinfaceandGitHub 9b3bed885e harvester: Make PlotsRefreshParameter streamable + use from_json_dict (#11759)
* Make `PlotsRefreshParameter` streamable

* Use `from_json_dict` instead of `dataclass_from_dict`

* Fix `test_plot_manager.py`
2022-06-03 12:35:50 -05:00
dustinfaceandGitHub bc98e6c5d8 tests: Some renaming in plot manager tests to avoid pytest conflicts (#9692)
Having `Test` or `test_` prefixes obviously lets `pytest` think its a
actual test class/method and it
leads to warnings in test logs if it doesn't comply with its rules:

```
/home/runner/work/chia-blockchain/chia-blockchain/tests/plotting/test_plot_manager.py:45:
PytestCollectionWarning: cannot collect test class 'TestDirectory'
because it has a __init__ constructor (from:
plotting/test_plot_manager.py)
```
or
```
/home/runner/work/chia-blockchain/chia-blockchain/tests/plotting/test_plot_manager.py:135:
PytestCollectionWarning: cannot collect test class 'TestEnvironment'
because it has a __init__ constructor (from:
plotting/test_plot_manager.py)
```
2022-05-31 13:42:11 -05:00
dustinfaceandGitHub 541b99e311 harvester: Introduce recursive_plot_scan (#11468)
* harvester: Introduce `recursive_plot_scan`

* Change log and merge boths lookups

* Drop redundant assignments
2022-05-09 16:43:53 -07:00
dustinfaceandGitHub 7d11d54f89 plotting: Enable isort (#11135) 2022-04-21 17:00:00 -07:00
dustinfaceandGitHub 1e7703fe6a plotting|tests|setup: Improve PlotManager cache (#9903)
* plotting|tests: Extend the lifetime of plot data cache entries

* plotting|tests: Add `prover` and `farmer_public_key` to the cache

* plotting: Missing type hints in `Cache` and `CacheEntry`

* setup: Bump `chiapos` to 1.0.10

* plotting: Move cache classes into `chia.plotting.cache`

* plotting: Introduce `CacheEntry.from_disk_prover`

* Make `Cache` a dataclass

* `staticmethod` -> `classmethod`

* Store cache data as `bytes` in `DiskCache`
2022-04-20 11:08:35 -07:00
ded9f68583 chia|tests|github: Implement, integrate and test plot sync protocol (#9695)
* protocols|server: Define new harvester plot refreshing protocol messages

* protocols: Bump `protocol_version` to `0.0.34`

* tests: Introduce `setup_farmer_multi_harvester`

Allows to run a test setup with 1 farmer and mutiple harvesters.

* plotting: Add an initial plot loading indication to `PlotManager`

* plotting|tests: Don't add removed duplicates to `total_result.removed`

`PlotRefreshResult.removed` should only contain plots that were loaded
properly before they were removed. It shouldn't contain e.g. removed
duplicates or invalid plots since those are synced in an extra sync step
and not as diff but as whole list every time.

* harvester: Reset `PlotManager` on shutdown

* plot_sync: Implement plot sync protocol

* farmer|harvester: Integrate and enable plot sync

* tests: Implement tests for the plot sync protocol

* farmer|tests: Drop obsolete harvester caching code

* setup: Add `chia.plot_sync` to packages

* plot_sync: Type hints in `DeltaType`

* plot_sync: Drop parameters in `super()` calls

* plot_sync: Introduce `send_response` helper in `Receiver._process`

* plot_sync: Add some parentheses

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* plot_sync: Additional hint for a `Receiver.process_path_list` parameter

* plot_sync: Force named parameters in `Receiver.process_path_list`

* test: Fix fixtures after rebase

* tests: Fix sorting after rebase

* tests: Return type hint for `plot_sync_setup`

* tests: Rename `WSChiaConnection` and move it in the outer scope

* tests|plot_sync: More type hints

* tests: Rework some delta tests

* tests: Drop a `range` and iterate over the list directly

* tests: Use the proper flags to overwrite

* test: More missing duplicates tests

* tests: Drop `ExpectedResult.reset`

* tests: Reduce some asserts

* tests: Add messages to some `assert False` statements

* tests: Introduce `ErrorSimulation` enum in `test_sync_simulated.py`

* tests: Use `secrects` instead of `Crypto.Random`

* Fixes after rebase

* Import from `typing_extensions` to support python 3.7

* Drop task name to support python 3.7

* Introduce `Sender.syncing`, `Sender.connected` and a log about the task

* Add `tests/plot_sync/config.py`

* Align the multi harvester fixture with what we do in other places

* Update the workflows

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-04-07 17:10:44 -07:00
Adam KellyandGitHub b8e1f3b9a9 Rename confusing fixtures, especially ones with the same name but dif… (#10772)
* Rename confusing fixtures, especially ones with the same name but different implementation

* rename test_environment to test_plot_environment

* Make it so setup_two_nodes is no longer the name of a fixture and a utility function

* revert premature fixture rename: two_wallet_nodes_start_height_1
2022-03-22 11:24:55 -07:00
Adam KellyandGitHub ff324095cc Remove block tools and keychain globals (#10358)
* Use bt fixture

* rebase

* Use local_hostname

* Fix test_json (inheritance from unittest.TestCase)

* Use correct BlockTools fixture for test_simulation

* Pass bt fixture into cost calculation tests

* flake8

* Add missing parameters to test functions

* Fix from rebase issues

* Remove set_shared_instance

* Update comment

* Remove unneeded comments

* Remove unused code

* Remove unused code, run `multiprocessing.set_start_method("spawn")` at correct time.

* Revert unrelated change

* Set daemon_port. Teardown services in correct order.
    BIG thanks to Mariano Sorgente for debugging help.

* Add back type signature - rebase issue

* Apply review fixes from Jeff

* Document why we need a later pytest-asyncio version

* Correct type for _configure_legacy_backend

* See what's going on during CI mypy run

* github workflows

* mypy typing

* Remove legacy Keyring create method

* Start daemon first

* Shutdown daemon coroutine properly

* Remove un-needed daemon_port argument

* Set chia-blockchain-gui to hash in main

* Remove connect_to_daemon_port

* Remove code that set "daemon_port" before calling `setup_daemon`

* Remove self_hostname fixture and extra self_hostname global

* Fix two test files that were not importing self_hostname

* self_hostname fixture

* Remove more unused test code

* Simplify  fixture
2022-03-10 11:06:49 -08:00
dustinfaceandGitHub 25d30a4c72 tests: Fix and improve plot manager caching tests (#9785)
* tests: Fix plot manager caching tests

The new plot manager was compared to itself instead to the one from the 
test environment. This is an old issue introduced in #8875.

* tests: Compare the loaded cache data also, not only the cache size
2022-01-13 17:41:15 -08:00
dustinfaceandGitHub 86e7d694d2 tests: Avoid unintentional refresh cycles in test_plot_manager.py (#9603) 2021-12-20 10:38:11 -08:00
dustinfaceandGitHub 2ab146bf2d plotting|tests|github: Fix and test re-loading/removal of plots where we missed the keys (#9578)
* tests: Allow to use custom keys in `BlockTools.new_plot`

* tests: Create three plots with `G1Element()` as farmer/pool key

But not add them as default plots. Just to have plots in the test cache with different keys from
within and outside the keychain for plot loading tests.

create

* test|github: Bump `test-cache` version to `0.28.0`

* plotting: Drop plots from `no_key_filenames` after the keys appeared

* plotting: Drop plots from `no_key_filenames` if the plot was removed

* tests: Test loading of plots where we miss the keys

* plotting: Use `x.copy()` instead of `list(x)`

* plotting: Add a comment about the missing keys plot removal

* tests: Fix after #9595
2021-12-17 14:36:29 -08:00
234d121c2a chia|tests|pre-commit: Add missing __init__.py files (#8758)
* chia|tests: Add missing `__init__.py` files

* check and create needed __init__.py files via pre-commit

* plotting|tests: Handle `mypy` issues introduced by the `main` rebase

* pre-commit: Run `build-init-files.py` more verbose

* tests: Ignore `__pycache__` folders in `build-init-files.py`

* Update tests/build-init-files.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update tests/build-init-files.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* tests: Add shebang, make executable, add description

To `tests/build-init-files.py`

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2021-12-16 18:52:46 -08:00
dustinfaceandGitHub 08b5d68f2c tests: Move the expected_result_matched assignment out of the loop (#9595)
* tests: Move the `expected_result_matched` assignment out of the loop

This issue was introduced in #9505 and leads to false positives as soon 
as there was one of the attributes in `PlotRefreshResult` checked to be 
valid followed by an invalid one.

* tests: Fix `test_invalid_plots`
2021-12-16 18:52:32 -08:00
dustinfaceandGitHub aabc54bb53 plotting|tests: Fix try/except in PlotManager._refresh_task (#9588)
* plotting: Fix try/except in `PlotManager._refresh_task`

* tests: Move the start refreshing call into `PlotRefreshTester` ctor

It being in `run()` before where it was called every test iteration lead
to not detecting an issue with the try/except introduced in #9409. See
ee1dd1912f4678d7b960b670473bd4b3ffbd890b for the fix.
2021-12-16 15:14:34 -08:00
dustinfaceandGitHub a68ffecea1 plotting|tests: Fix and test removal of invalid plots (#9587)
* plotting: Drop from `failed_to_open_filenames` if the plot was removed

* tests: Test removal from `failed_to_open_filenames`
2021-12-16 15:14:12 -08:00
dustinfaceandGitHub 5496dc1971 tests: Call stop_refreshing in the test_environment fixture (#9567) 2021-12-15 15:27:37 -08:00
dustinfaceandGitHub 47c02aabd2 tests: Rework callback expectation checks with PlotRefreshTester (#9505) 2021-12-13 08:41:24 -08:00
dustinfaceandGitHub 8e6b7afa79 tests: Some cleanups after #9306 and #9409 (#9503)
* tests: Drop `None` check and compare the events directly

* tests: Drop unused `TestRefreshResult`
2021-12-10 09:18:51 -08:00
dustinfaceandGitHub ad7b549cbd plotting|tests: Catch exceptions in the plot manager thread (#9409)
Without this the plot manager thread silently dies on exceptions and
refreshing just stops. This change leads to refreshing reset and a error
log on exceptions.
2021-12-07 09:57:24 -08:00
dustinfaceandGitHub 59b8be5f72 chia|tests: Pass the loaded/removed plots with PlotRefreshResult (#9306)
* chia|tests: Pass the loaded/removed plots with `PlotRefreshResult`

Instead of just the counts.

* plotting: Default to `False` for `expected_result_matched`

* tests: Validate the loaded/removed plots match the expectations
2021-12-06 07:53:26 -08:00
dustinfaceandGitHub 3c21db387c tests: Fix test environment creation in test_plot_manager.py (#9370)
The `bt.refresh_plots()` wasn't even awaited and the test environments 
all shared the same root. Now every environment gets its own temp 
directory where the plots relevant for the tests are copied to.
2021-11-27 18:20:08 -08:00
dustinfaceandGitHub 7178d47d3a tests|github: Improve PlotManager tests + move them to test_plot_manager.py (#8875)
* tests: Improve `PlotManager` tests + move them to `test_plot_manager.py`

* github|tests: Enable `plotting` tests and generate the workflow files
2021-11-17 06:54:25 -08:00