Commit Graph
31 Commits
Author SHA1 Message Date
Matt HauffandGitHub 40db4635a8 [LABS-245] Enable PEP604 Ruff rules (#20269)
* Enable PEP604 Ruff rules

* Fix harcoded signature in test

* Hack CLVMStreamable test with note to fast follow
2025-11-18 12:34:00 -08:00
Almog De PazandGitHub 19a8329cbe Plotmanager v2 (#19832)
* prover protocol and v2Prover

* format name

* format

* refactor filename

* tests/raise unimplemented

* add get_filename_str to mock

* rename methods

* rename

* refactor

* improve coverage

* test from bytes
2025-07-30 11:05:52 -07:00
Kyle AltendorfandGitHub d50b6e0883 use chia_rs directly for sized bytes (#19326) 2025-02-28 09:35:25 -08:00
Kyle AltendorfandGitHub e446d6b726 use chia_rs directly for sized ints (#19303)
* use chia_rs directly for sized ints

* Update data_layer_rpc_api.py
2025-02-24 14:33:09 -08:00
Matt HauffandGitHub 604fc1feb0 [CHIA-1732] Add Ruff specific linting rules (#18845)
* Add Ruff specific linting rules

* Further tighten ignore list

* Address comments by @altendky

* One more comment @altendky
2024-11-14 15:58:12 -07:00
Earle LoweandGitHub 61cf266e28 CHIA-1697: Add new flag to support recursively scanning and following links (#18803)
* Add new flag to support recurvely scanning and following links

* black fix

* hardlink_to only in python 3.10+

* Remove unneeded hardlink checks as hardlinks to directories aren't a thing

* Minor update per code review

* use boolean_datacases

* Add new flag to initial-config
2024-11-06 10:24:02 -07:00
Kyle AltendorfandGitHub 9c8557a1f8 Remove Python 3.8 support and update source to 3.9 standards (#18687)
* manual python 3.8 removals

* poetry

* pyupgrade

* black

* isort

* manually tidy imports

* work around shadowed type

* catch up build network protocol files for pyupgrade changes

* placate pylint

* re-enable 20.04

* placate flake8
2024-10-16 14:21:15 -07:00
Arvid NorbergandGitHub 95e5b97557 Use bls from chia rs (#16715)
* replace blspy imports with chia_rs imports for BLS types

* remove blspy-stubs, since we're dropping the blspy dependency. chia_rs has type stubs already
2023-11-07 09:06:52 -08:00
0deb8e6dc6 Timeout for retrieving a decompressor. (#16047)
* Timeout for retrieving a decompressor.

* Pre-commit fix

* Use new chiapos

* Use chiapos 2.0.1

---------

Co-authored-by: wallentx <william.allentx@gmail.com>
2023-08-16 18:40:29 -05:00
William AllenandGitHub 3a3b7e5ac2 Fix default parallel decompressor count (#15845) 2023-07-24 21:49:45 -05:00
Izumi HoshinoandGitHub ec5ef087fe Added defaults to get_harvester_config API (#15820) 2023-07-20 13:25:16 -05:00
Izumi HoshinoandGitHub 85d14f561a Added compression level and harvesting mode to harvester protocol/mes… (#15776)
* Added compression level and harvesting mode to harvester protocol/messages

* Added test

* Fixed lint error
2023-07-18 16:24:56 -05:00
Kyle AltendorfandGitHub d84917f5df hint chia.plotting.create_plots (#14692)
* hint chia.plotting.create_plots

* correct the import

* fixup

* imports

* get final from typing_extensions
2023-03-08 12:57:50 -06:00
dustinfaceandGitHub 978c3708aa Catch exceptions raised when resolving plot directories (#13786)
* plotting: Catch exceptions raised when resolving plot directories

* `try/except/else` -> `try/except/continue`
2022-11-10 19:49:31 -06:00
Izumi HoshinoandGitHub 42e67f049e Prevent a crash that happens when config["harvester"]["plot_directories"] is not a list (#13551) 2022-09-30 17:49:07 -05:00
Kyle AltendorfandGitHub 3b084a165b configure isort to add the future annotations import (#13327)
* configure isort to add the future annotations import

* apply the new isort setting

* remove type ignores for new mypy (#13539)

https://pypi.org/project/mypy/0.981/

* another
2022-09-30 03:40:22 -05:00
joshpainterandGitHub 29643d1feb Fix bug in remove_plot_directory (#13332)
Revert commit c400d816af
2022-09-06 13:04:49 -05:00
dustinfaceandGitHub 9b3bed885e harvester: Make PlotsRefreshParameter streamable + use from_json_dict (#11759)
* Make `PlotsRefreshParameter` streamable

* Use `from_json_dict` instead of `dataclass_from_dict`

* Fix `test_plot_manager.py`
2022-06-03 12:35:50 -05:00
dustinfaceandGitHub c400d816af harvester: Validate the path in add_plot_directory (#11625)
Make sure it exists, is a directory and hasn't been added already.
2022-05-24 13:21:44 -07:00
dustinfaceandGitHub 541b99e311 harvester: Introduce recursive_plot_scan (#11468)
* harvester: Introduce `recursive_plot_scan`

* Change log and merge boths lookups

* Drop redundant assignments
2022-05-09 16:43:53 -07:00
dustinfaceandGitHub 7d11d54f89 plotting: Enable isort (#11135) 2022-04-21 17:00:00 -07:00
9ff3fc993f Ms.plot load perf2 (#10978)
* 2.7 seconds -> 0.45 seconds

* Merge

* Work on create_plots refactor

* Try to fix tests

* Try to fix tests

* Use new functions

* Fix block_tools by adding dir

* Extra argument

* Try to fix cyclic import

* isort

* Drop warning

* Some cleanups around `exclude_final_dir` and directory adding

* Cleanup `min_mainnet_k_size` checks

* Drop unrelated changes

* Fixes after rebase

* Fix cyclic import

* Update tests/block_tools.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update tests/block_tools.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2022-04-11 09:16:01 -07:00
Kyle AltendorfandGitHub e0a1fc4eaf Switch to integrated lock_and_load_config() context manager (#10698)
* minor lock scope reduction

* use the lock in tests

* Use the passed root_path in configure CLI command

* switch to lock_and_load_config()

* oops

* cleanup

* make _load_config_maybe_locked() private

* black

* Remove future improvement opportunity TODO comment
2022-03-28 12:52:51 -07:00
Kyle AltendorfandGitHub 48cd97cf47 minor followup to config locking (#10696)
* minor lock scope reduction

* use the lock in tests
2022-03-17 09:03:58 -07:00
dbeff36ae0 Add configuration locking (#10680)
* Add configuration locking

Extracted from https://github.com/Chia-Network/chia-blockchain/pull/10631

* note that fasteners will likely be replaced by filelock

* Fix test_multiple_writers on macOS

* create_all_ssl() doesn't need to be inside the config access lock

* add warnings about not using async within get_config_lock() get lock contexts

* no need to pre-touch the lock file

* .yaml.lock instead of just .lock

* test_multiple_writers() is sync

* Revert "add warnings about not using async within get_config_lock() get lock contexts"

This reverts commit 681af3835b.

* reduce lock context size in chia_init()

* use an exit stack in load_config()

* avoid config existence precheck

* only lock around the read in load_config()

* do not raise e, just raise

* tidy new imports

* fix queue empty check in test_config.py

* remove commented out code in test_config.py

* remove unused import

Co-authored-by: Jeff Cruikshank <jeff@chia.net>
2022-03-12 06:52:47 -08:00
dustinfaceandGitHub 59b8be5f72 chia|tests: Pass the loaded/removed plots with PlotRefreshResult (#9306)
* chia|tests: Pass the loaded/removed plots with `PlotRefreshResult`

Instead of just the counts.

* plotting: Default to `False` for `expected_result_matched`

* tests: Validate the loaded/removed plots match the expectations
2021-12-06 07:53:26 -08:00
dustinfaceandGitHub 7c1a72848c plotting|harvester|tests: Improve batch processing (#8385)
* plotting|harvester|tests: Improve batch processing

Let it only process `batch_size` files per `refresh_batch` call.

* plotting: Improve thread shutdown
2021-10-20 18:06:01 -07:00
dustinfaceandGitHub 5fae0aae41 plotting|harvester|tests: Drop some parts of PlotRefreshResult (#8760)
* plotting|harvester|tests: Drop `loaded_size` from `PlotRefreshResult`

* rename: Drop `_plots` and `_files` suffixes in `PlotRefreshResult`

* plotting: Drop `PlotRefreshResult.__add__`

I introduced it but i don't really like it because its not a "real" add
any only works in one direction because of `remaining`.
2021-10-13 14:20:32 -07:00
dustinfaceandGitHub 2db9944687 plotting|util: Adjust plot refresh parameter (#8275) 2021-09-02 08:23:52 -07:00
dustinfaceandGitHub 17dea1b708 plotting|util|tests: Fix and test re-trying of plots which failed to load (#8279)
* Revert "plotting: Fix failed_to_open_filenames re-try interval"

This reverts 8a8abc41a8 which is part of 
#7848. Seems like i confused myself there.

* plotting|util: Introduce `retry_invalid_seconds` in plot refresh params

* plotting: Cleanup `failed_to_open_filenames`

* tests: Test re-try of plots which failed to load
2021-09-01 09:21:29 -07:00
dustinfaceandGitHub 53149f2e57 chia|tests: Introduce PlotManager + some plot loading improvements and fixes (#7848)
* harvester|plotting|tests: Introduce `PlotManager` class

This moves all plots related and plot-directory related stuff from the
harvester into the class `PlotManager`, adjusts all related code
accordingly and adds some extra wrappers there.

* harvester|plotting|tests: Return how many new plots were loaded

* plotting: Fix `failed_to_open_filenames` re-try interval

With `< 1200` it just tries it on the next refresh.

* plotting: Fix and improve duplicates handling

* harvester|plotting: Thread locks for `PlotManager.plots`

* chia|tests: Load plots in batches

* chia|tests: Move plot refreshing into a separate thread

* plotting: Properly handle removed plots

And fix tests accordingly. It seems like this fix https://github.com/Chia-Network/chia-blockchain/pull/3350 wasn't really a fix, rather adjusting to allow for a bug?

* plotting|harvester|tests: Introduce `PlotRefreshResult`

* tests: Expand `test_farmer_harvester_rpc.py`

* chia|tests: Move some stuff from `plot_tools.py` into new file `util.py`

* refactor: Rename `plot_tools.py` to `manager.py`

* chia|tests: Use pure dataclass for `PlotsRefreshParameter`

With `uint16` as type saving to config doesn't work, this is a
preparation for the next commit.

* harvester: Adjust deprecation message, use `info` instead of `warning`

* plotting: Fix typo

* refactor: Rename `filename` to `file_path`

Fits better and does avoid shadowing with filename from outer scope.

* chia|tests: Move some methods from `plotting.manager` to `plotting.util`

* plotting: Make `refresh_callback` mandatory
2021-08-09 10:25:15 -07:00