Commit Graph
38 Commits
Author SHA1 Message Date
Matt HauffandGitHub 40db4635a8 [LABS-245] Enable PEP604 Ruff rules (#20269)
* Enable PEP604 Ruff rules

* Fix harcoded signature in test

* Hack CLVMStreamable test with note to fast follow
2025-11-18 12:34:00 -08:00
Arvid NorbergandGitHub b1c9beed3f improve v2-plot support in plot-sync (#20238) 2025-11-18 10:05:50 -08:00
Arvid NorbergandGitHub 5664516dda Bump chia rs 0.34.0 (#20226)
bump chia_rs to 0.34.0
2025-11-10 17:02:10 -07:00
arvidn 39764bb7c5 use log.exception() instead of log.error() and log.warning() when logging exceptions 2025-10-28 14:18:41 +01:00
Arvid NorbergandGitHub 18fd65d13b pass along plot format version from ProverProtocol into ProofOfSpace (#19930)
pass along plot format version from ProverProtocol into ProofOfSpace objects
2025-08-14 12:31:07 -07:00
Kyle AltendorfandGitHub 7a26610e8f less pre-inting for ints (#19853) 2025-07-23 13:18:04 -07:00
6050235bdb [CHIA-3043] Add (hopefully) non-controversial ruff rules (#19684)
* Add (hopefully) non-controversial ruff rules

* mypy-exclusions.txt

* PIE

* Fix pre-commit?

* PYI

* RSE

* S + ignores

* Whitespace fix

* Re-add build-init-files.py

* Add comment about security ignores

* Update mypy-exclusions.txt

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Fix the executable status of build-init-files.py

* Set executable bit for tools/run_block.py and readd shebang

* Fix new errors

---------

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2025-06-11 08:36:08 -07:00
Richard KissandGitHub a803d18c3b The chia.protocols module no longer depends upon chia.server. (#19636)
<!-- Merging Requirements:
- Please give your PR a title that is release-note friendly
- In order to be merged, you must add the most appropriate category
Label (Added, Changed, Fixed) to your PR
-->
<!-- Explain why this is an improvement (Does this add missing
functionality, improve performance, or reduce complexity?) -->

The modules `chia.protocols` and `chia.server` form a dependency loop.
This change at least removes the direct dependency of `chia.protocols`
on `chia.server`.

### Purpose:

<!-- Does this PR introduce a breaking change? -->

### Current Behavior:

### New Behavior:

<!-- As we aim for complete code coverage, please include details
regarding unit, and regression tests -->

### Testing Notes:

<!-- Attach any visual examples, or supporting evidence (attach any
.gif/video/console output below) -->
2025-05-29 09:24:29 -07:00
Kyle AltendorfandGitHub d50b6e0883 use chia_rs directly for sized bytes (#19326) 2025-02-28 09:35:25 -08:00
Kyle AltendorfandGitHub e446d6b726 use chia_rs directly for sized ints (#19303)
* use chia_rs directly for sized ints

* Update data_layer_rpc_api.py
2025-02-24 14:33:09 -08:00
Kyle AltendorfandGitHub 1ffa73e99e task referencer (#18914)
* put otherwise unheld tasks into the pit

* and the rest too

* cull less often

* task referencer

* private

* report unexpectedly unreferenced tasks

* also warn for automated catching in tests

* reporting task

* undo some

* fixup for 3.9

* ban asyncio.create_task

* oof

* simplify including using `Task.add_done_callback()` for culling
2025-01-02 10:13:51 -07:00
Kyle AltendorfandGitHub dd028574bb enable ruff flake8 async (#18899) 2024-11-20 09:00:14 -07:00
6c90a76b56 Tighten up ruff ignore list (#18837)
* Tighten up ruff ignore list

* Okay that fix was indeed unsafe

* enable type-name-incorrect-variance

* enable literal-membership

* enable non-augmented-assignment

* enable useless-return

* enable global-variable-not-assigned

* -

* fixup

* Clean up roff.toml from annotations

* use ignore instead of explicit re-export

* use more descriptive names

---------

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2024-11-12 12:19:31 -07:00
Kyle AltendorfandGitHub 9c8557a1f8 Remove Python 3.8 support and update source to 3.9 standards (#18687)
* manual python 3.8 removals

* poetry

* pyupgrade

* black

* isort

* manually tidy imports

* work around shadowed type

* catch up build network protocol files for pyupgrade changes

* placate pylint

* re-enable 20.04

* placate flake8
2024-10-16 14:21:15 -07:00
Matt HauffandGitHub 3637bd33c6 Delete chia.util.misc (#18170)
* Move VersionedBlob to chia.util.streamable

* Move format_bytes to chia.cmds.cmds_util

* Move format_minutes to chia.cmds.cmds_util

* Move prompt_yes_no to chia.cmds.cmds_util

* Move get_list_or_len to from chia.plot_sync.receiver

* Move validate_directory_writable to chia.cmds.cmds_util

* Move *termination_signals to test_services

* Move UInt*Range to chia.util.streamable

* Create chia.util.batches

* Create chia.server.signal_handlers

* Create chia._tests.util.split_managers

* Move ValuedEvent to chia.types.transaction_queue_entry

* Move available_logical_cores to chia.util.cpu

* Move caller_file_and_line to chia._tests.util.misc

* Move satisfies_hint to chia._tests.wallet.test_singleton_lifecycle_fast
2024-06-18 08:27:17 -05:00
Earle LoweandGitHub 48e4039e58 Update black to 24.1.1 and update source as needed (#17499)
* Update source to black 24 spec

* update black to 24.1.1

* update pylint to ignore multiple-statements

* update test file

* black updates

* format updates

* Update test_network_protocol_files.py

* Update test_network_protocol_files.py
2024-03-13 11:08:19 -07:00
Felix BruckerandGitHub 23e003aaae Fix negative plot sync durations not crashing the harvester (#17444)
* Fix negative plot sync durations

* Remove unnecessary try/except
2024-02-01 22:56:49 -08:00
Kyle AltendorfandGitHub 90c1441af0 apply pyupgrade --py38-plus --keep-runtime-typing (#14646) 2023-10-24 12:01:23 -05:00
Izumi HoshinoandGitHub 85d14f561a Added compression level and harvesting mode to harvester protocol/mes… (#15776)
* Added compression level and harvesting mode to harvester protocol/messages

* Added test

* Fixed lint error
2023-07-18 16:24:56 -05:00
Izumi HoshinoandGitHub 1f9081a225 Added total_effective_plot_size for get_harvesters RPC API (#15770)
* Added `total_effective_plot_size` for `get_harvesters` RPC API

* Fixed lint error

* Change the type of `total_effective_plot_size` in Receiver to `int`

* Fixed lint error

* Fixed lint error 2

* Changed text

* Added test

* Fixed lint error

* Fixed lint error 2

* Fixed a test issue

* Added config.py to tests/cmds
2023-07-18 12:36:12 -05:00
dustinfaceandGitHub ebc5f3c124 util: Improve list_to_batches (#15415)
* Make `list_to_batches` work with collections instead of lists only

* Move `to_batches` into `chia.util.misc`

* Only support `set` and `list`

* Drop `tests.generator.test_to_batches` exclusion

* Improve type restrictions and be more coverage friendly in tests
2023-06-06 12:00:07 -05:00
dustinfaceandGitHub f0006b5353 server: peer_host + peer_port -> peer_info in WSChiaConnection (#14151) 2023-04-12 11:14:38 -06:00
Kyle AltendorfandGitHub 3a5c1a5bd5 hint chia.harvester.harvester, harvester_api (#14690)
* hint chia.harvester.harvester

* hint chia.harvester.harvester_api
2023-02-28 11:42:58 -06:00
Kyle AltendorfandGitHub c678f00acb cleanup adjacent string literals (#14650) 2023-02-22 17:16:07 -06:00
2706f5995b server: Enable and fix mypy in ws_connection.py (#13878)
* server: Enable and fix `mypy` in `ws_connection.py`

* Apply suggestions from code review

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Tweak error message

* Tweak formatting

* Make `WSChiaConnection.close_callback` optional

* Tweak assert message

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Don't provide a default for `close_callback`

* Adjust assertion

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-11-12 14:18:11 -06:00
Kyle AltendorfandGitHub 3b084a165b configure isort to add the future annotations import (#13327)
* configure isort to add the future annotations import

* apply the new isort setting

* remove type ignores for new mypy (#13539)

https://pypi.org/project/mypy/0.981/

* another
2022-09-30 03:40:22 -05:00
dustinfaceandGitHub 40538fd942 mypy: Fix some asyncio.Task related ignores (#13487) 2022-09-23 17:19:36 -05:00
dustinfaceandGitHub 60908420b6 plot_sync: Change log level to INFO in Receiver.reset (#13463)
It's not necessarily an error, could be just a harvester restart.
2022-09-16 16:21:51 -05:00
dustinfaceandGitHub 3f740d375c plot_sync: Fix type hints in Receiver._process (#11890)
* plot_sync: Fix type hints in `Receiver._process`

* Move `PlotSyncMessage` and `T_PlotSyncMessage` into `plot_sync.util`
2022-07-06 21:02:09 -05:00
dustinfaceandGitHub a221726770 server: Hint data parameter of make_msg (#11876) 2022-06-13 11:41:22 -05:00
dustinfaceandGitHub 3e3dafb0ef farmer: Improve logging in plot_sync.{receiver|delta} (#11474)
* Add the `node_id` to the logs in `Receiver`

* Log all processed messages

* Log reset calls

* Improve `reciever.Sync` and `delta.Delta` logging
2022-05-24 16:21:56 -07:00
Kyle AltendorfandGitHub a9f7cce06b Skip plot_sync_callback if delta is None, hint Harvester.server (#11620)
* hint Harvester.server as Optional[ChiaServer]

* use a Protocol instead of Callable

* oh right, Optional
2022-05-24 13:21:14 -07:00
dustinfaceandGitHub fc2d6d3db0 harvester: Fix deadlock on disconnect after a repeated plot sync start (#11481)
* Test plot sync start and disconnect while a sync is in progress

* Drop `Receiver._lock`, instead wait for `Receiver.sync_active` to clear

* Properly stop the sender and plot manager refreshing on disconnect

* Poll faster, drop the log

* Reset `_sync_id` at the very end of `_finalize_sync`
2022-05-12 15:16:22 -07:00
dustinfaceandGitHub 0d5251c3a7 farmer|gui: Enable paginated plot loading and improved state reporting (#11367)
* farmer: Adjust notifications from the farmer to the UI

- Only send the data for the harvester which actually sent an update
- Notify for each loaded batch during initial loading

* Enable improved farmer/harvester GUI
2022-05-03 15:33:38 -07:00
dustinfaceandGitHub 387abc3d8e farmer|rpc: Some changes to get_harvesters{_summary} RPC endpoints (#11342)
* farmer|rpc: Always add `last_sync_time` in `Receiver.to_dict`

This changes the resonse of `get_harvesters` and
`get_harvesters_summary` to always include that field with value
`None`/`null` if the receiver is not yet synced.

* plot_sync: Add sync state to dict in `Receiver.to_dict`

* Add `total_plot_size` to `Receiver.to_dict`

* Refactor sync object assertions

* Access `_current_sync` directly

* Generate `syncing` data outside
2022-04-29 18:34:14 -07:00
270c69eb5d plot_sync: Introduce receiver.Sync (#11267)
* plot_sync: Introduce `receiver.Sync`

* Use `dataclasses.replace`

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-04-29 08:16:39 -07:00
dustinfaceandGitHub b693aeb407 farmer|rpc: Introduce get_harvesters_summary RPC endpoint (#11245) 2022-04-28 19:36:21 -07:00
ded9f68583 chia|tests|github: Implement, integrate and test plot sync protocol (#9695)
* protocols|server: Define new harvester plot refreshing protocol messages

* protocols: Bump `protocol_version` to `0.0.34`

* tests: Introduce `setup_farmer_multi_harvester`

Allows to run a test setup with 1 farmer and mutiple harvesters.

* plotting: Add an initial plot loading indication to `PlotManager`

* plotting|tests: Don't add removed duplicates to `total_result.removed`

`PlotRefreshResult.removed` should only contain plots that were loaded
properly before they were removed. It shouldn't contain e.g. removed
duplicates or invalid plots since those are synced in an extra sync step
and not as diff but as whole list every time.

* harvester: Reset `PlotManager` on shutdown

* plot_sync: Implement plot sync protocol

* farmer|harvester: Integrate and enable plot sync

* tests: Implement tests for the plot sync protocol

* farmer|tests: Drop obsolete harvester caching code

* setup: Add `chia.plot_sync` to packages

* plot_sync: Type hints in `DeltaType`

* plot_sync: Drop parameters in `super()` calls

* plot_sync: Introduce `send_response` helper in `Receiver._process`

* plot_sync: Add some parentheses

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* plot_sync: Additional hint for a `Receiver.process_path_list` parameter

* plot_sync: Force named parameters in `Receiver.process_path_list`

* test: Fix fixtures after rebase

* tests: Fix sorting after rebase

* tests: Return type hint for `plot_sync_setup`

* tests: Rename `WSChiaConnection` and move it in the outer scope

* tests|plot_sync: More type hints

* tests: Rework some delta tests

* tests: Drop a `range` and iterate over the list directly

* tests: Use the proper flags to overwrite

* test: More missing duplicates tests

* tests: Drop `ExpectedResult.reset`

* tests: Reduce some asserts

* tests: Add messages to some `assert False` statements

* tests: Introduce `ErrorSimulation` enum in `test_sync_simulated.py`

* tests: Use `secrects` instead of `Crypto.Random`

* Fixes after rebase

* Import from `typing_extensions` to support python 3.7

* Drop task name to support python 3.7

* Introduce `Sender.syncing`, `Sender.connected` and a log about the task

* Add `tests/plot_sync/config.py`

* Align the multi harvester fixture with what we do in other places

* Update the workflows

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-04-07 17:10:44 -07:00