45 Commits
Author SHA1 Message Date
e2c3f1dea9 Break daemon ↔ plotting cycle: move harvester config helpers to chia.util (#20544)
* Break daemon ↔ plotting cycle: move harvester config helpers to chia.util

- Add chia.util.harvester_config with add_plot_directory, get_plot_directories,
  remove_plot_directory (extracted from chia.plotting.util)
- Update daemon, harvester, plotters, cmds, simulator, and tests to import
  from chia.util.harvester_config
- Remove daemon's dependency on plotting in tach.toml (cycle broken)

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix mypy: add type parameters for dict in harvester_config

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-03-03 12:12:39 -06:00
Matt HauffandGitHub 40db4635a8 [LABS-245] Enable PEP604 Ruff rules (#20269)
* Enable PEP604 Ruff rules

* Fix harcoded signature in test

* Hack CLVMStreamable test with note to fast follow
2025-11-18 12:34:00 -08:00
Kyle AltendorfandGitHub 1ffa73e99e task referencer (#18914)
* put otherwise unheld tasks into the pit

* and the rest too

* cull less often

* task referencer

* private

* report unexpectedly unreferenced tasks

* also warn for automated catching in tests

* reporting task

* undo some

* fixup for 3.9

* ban asyncio.create_task

* oof

* simplify including using `Task.add_done_callback()` for culling
2025-01-02 10:13:51 -07:00
6c90a76b56 Tighten up ruff ignore list (#18837)
* Tighten up ruff ignore list

* Okay that fix was indeed unsafe

* enable type-name-incorrect-variance

* enable literal-membership

* enable non-augmented-assignment

* enable useless-return

* enable global-variable-not-assigned

* -

* fixup

* Clean up roff.toml from annotations

* use ignore instead of explicit re-export

* use more descriptive names

---------

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2024-11-12 12:19:31 -07:00
Matt HauffandGitHub 73e4a5bbea [CHIA-1680] Finish replacing flake8 with Ruff (#18802)
* Apply ruff unsafe fixes

* Manually fix tuples with `as`

* Remove all references to flake8
2024-11-04 08:59:57 -07:00
Kyle AltendorfandGitHub 9c8557a1f8 Remove Python 3.8 support and update source to 3.9 standards (#18687)
* manual python 3.8 removals

* poetry

* pyupgrade

* black

* isort

* manually tidy imports

* work around shadowed type

* catch up build network protocol files for pyupgrade changes

* placate pylint

* re-enable 20.04

* placate flake8
2024-10-16 14:21:15 -07:00
Kyle AltendorfandGitHub 73a7cb4687 CHIA-420 remove ignore for pkg_resources deprecation warning (#17933)
* remove ignore for `pkg_resources` deprecation warning

* replace pkg_resources usages

* fixup

* fixup

* fixup

* a couple

* imports

* cleanup

* maybe
2024-05-15 09:52:22 -05:00
Felix BruckerandGitHub d09b61b72d Pass full version in Handshake (#15679)
* Pass full version in Handshake

* Move chia version conversion into crawler

* Rename to overloaded `chia_short_version` and always return str

* Ensure we extract major.minor.patch from invalid pep440 if available
2024-03-27 10:40:47 -07:00
Earle LoweandGitHub 48e4039e58 Update black to 24.1.1 and update source as needed (#17499)
* Update source to black 24 spec

* update black to 24.1.1

* update pylint to ignore multiple-statements

* update test file

* black updates

* format updates

* Update test_network_protocol_files.py

* Update test_network_protocol_files.py
2024-03-13 11:08:19 -07:00
Earle LoweandGitHub b5ab1dd243 Update code to find packaged plotters in _internal directory (#17617) 2024-02-29 10:35:13 -08:00
Kyle AltendorfandGitHub 302e7e7f81 use importlib.metadata to get package versions (#14753)
* use importlib_metadata to get package versions

* ignore untyped call

* stop ignoring the deprecation warning

* Revert "stop ignoring the deprecation warning"

This reverts commit 931fe8c794.

* include chiapos version data

* tweak
2023-09-27 15:01:08 -05:00
Izumi HoshinoandGitHub 4278760924 Bladebit Hybrid disk mode (#16208) 2023-09-12 13:54:04 -05:00
Amine Khaldi a91f90b36b Merge commit '27ccd9a86dab07f665820e1ed2e6a6e76c059c72' into checkpoint/main_from_release_2.0.1_27ccd9a86dab07f665820e1ed2e6a6e76c059c72 2023-09-06 17:23:35 +01:00
Earle LoweandGitHub 27ccd9a86d Fix issue with GUI and chia plotters when making compressed plots (#16232)
* don't pass in no-direct-downloads on false

* Remove no-direct-downloads

* Remove no-direct-downloads
2023-09-05 15:46:11 -05:00
Arvid NorbergandGitHub 05500ea840 fix waiting for co-routines in plotters_util.py (#16184) 2023-08-29 13:47:38 -05:00
Earle LoweandGitHub 231bc4638d Use 'Completed writing plot' as the magic final words for cudaplot (#16011) 2023-08-10 21:06:01 -05:00
Izumi HoshinoandGitHub ca10d05399 Fixed an issue where -t option was required in cudaplot (#15963)
Fixed an issue where `-t` option was requierd in cudaplot
2023-08-07 19:56:52 -05:00
Izumi HoshinoandGitHub 51d3a5a7f6 cudaplot fix (#15892)
Fixed an issue where cudaplot failed
2023-08-01 14:01:19 -05:00
Izumi HoshinoandGitHub 2d65fc7114 Update cuda plot log-progress mapping (#15839) 2023-07-24 18:48:49 -05:00
b772ac91db Added cuda and plot compression support for bladebit 3 (#15774)
* Added cuda and plot compression support for bladebit 3

* Update chia/plotters/bladebit.py

Co-authored-by: Arvid Norberg <arvid@libtorrent.org>

* Update chia/plotters/bladebit.py

Co-authored-by: Arvid Norberg <arvid@libtorrent.org>

* Fixed issues

* Fixed lint error

---------

Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
2023-07-18 12:33:42 -05:00
Amine KhaldiandGitHub ef613df2f0 Annotate chia/plotters/chiapos.py (#15363)
Annotate chia/plotters/chiapos.py.
2023-05-23 12:01:37 -05:00
Kyle AltendorfandGitHub d84917f5df hint chia.plotting.create_plots (#14692)
* hint chia.plotting.create_plots

* correct the import

* fixup

* imports

* get final from typing_extensions
2023-03-08 12:57:50 -06:00
Kyle AltendorfandGitHub 20b2d66012 fix reporting of deprecation of plotter install and bladebit2 commands (#14421) 2023-02-07 18:13:01 -06:00
dustinfaceandGitHub 42f6a883ae isort: Remove the remaining exceptions and sort the related files (#14124) 2022-12-13 17:10:55 -06:00
Izumi HoshinoandGitHub b37f2009bd Made it able to RAM plot with bladebit2 (#13833) 2022-11-30 12:09:30 -06:00
wjblankeandGitHub 5c38a39077 upped the number of file handles for madmax (#13856)
* upped the number of file handles. this won't affect the hard limit but only soft limits set previously

* Set soft limit to max (hard limit)
2022-11-09 03:31:51 -06:00
Kyle AltendorfandGitHub fa5bcf3e82 leverage argparse for chia plotters help (#13742) 2022-10-28 23:59:36 -05:00
Izumi HoshinoandGitHub 6dfbc1ef7c Made error message of chia plotters version more traceable (#13704) 2022-10-26 15:48:24 -05:00
Izumi HoshinoandGitHub 0e93464521 Deprecate chia plotters install command (#13584)
* Deprecated `chia plotters install` command

* Fixed lint errors

* Made it able to recognize plotter executables in venv

* Added install-plotter.sh

* Removed strings for debug

* Added install-plotter.ps1

* Fixed an issue where madmax plotter under venv dir was not recognized by 'chia plotters' command

* Added exec permission to install-plotter.sh

* Fixed issues found in install-plotter.sh

* Fixed lint errors

* Added `-p` option to install.sh/install.ps1

* Fixed lint errors

* Added nice deprecated warning

* Fixed a lint error

* Slightly improved branching

* Made log messages look nice

* Fixed an issue where install.sh/ps1 -p failed if virtual env is not acivated
2022-10-17 12:35:58 -05:00
Kyle AltendorfandGitHub 3b084a165b configure isort to add the future annotations import (#13327)
* configure isort to add the future annotations import

* apply the new isort setting

* remove type ignores for new mypy (#13539)

https://pypi.org/project/mypy/0.981/

* another
2022-09-30 03:40:22 -05:00
Izumi HoshinoandGitHub e40772a0bf Added support for --no-t1-direct and --no-t2-direct (#13526) 2022-09-26 09:20:40 -05:00
Izumi HoshinoandGitHub a2b6a9b5f3 Cmj.bladebit (#11187)
* Making source code cleaner

* Cleaned up `install_plotter.py`

* Now `$CHIA_ROOT/plotters/bladebit` is searched first then bundled bladebit

* Fixed a lint error

* WIP: Implement bladebit update/checkout feature

* Enable to specify git branch/tag/commit on bladebit installation

* Modify arguments display order

* Added `chia plotters version` command

* Fixed lint errors

* Fixed lint error 2

* Implemented `chia plotters bladebit2` and handling for its options

* Fixed typo

* Fixed a lint error

* Added comment

* Added missing plotter arguments

* Fixed LGTM warning

* Fixed wrong typing

* Fixed a lint error

* Fixed an issue where `None` was evaluated as an empty string

* Support bladebit2 plotting from web request

* Updated response of `get_plotters` API for upcoming `bladebit2`

* Fixed bladebit2 argument handling

* Modified progress dict

* Added --no-cpu-affinity option to bladebit1

* Fixed a minor error

* Fixed an error where `chia plotters version` failed

* Fixed an issue where building bladebit2 failed

* Fixed an issue where bladebit2 stopped working if `--cache` option was set

* Updated branch for bladebit2

* Fixed an issue where plotting with madmax/bladebit failed for Python < 3.8 on Windows

* Now `chia plotters install madmax|bladebit2` should work on RedHat like OSes

* Fixed a wrong yum package name

* Added yum packages required for Bladebit2

* Changed the way to update bladebit

* Implemented option handling for madmax as well as bladebit

* Fixed a lint error

* Fixed a minor issue

* Fixed LGTM warning

* Applied changes for bladebit 2.0.0-alpha-2

* Fixed a lint error

* Trying to fix windows pre-commit error

* Fixed isort error
2022-09-23 19:42:37 -05:00
dustinfaceandGitHub 47ea161330 plotters: Only produce beta logs if beta: true, not if it just exists (#13445) 2022-09-16 16:20:54 -05:00
dustinfaceandGitHub 70fe981fb3 cmds: Implement chia beta (#12389)
* cmds: Implement `chia beta` for the beta test program

* Unhide and document all `beta` subcommands

* Refactor all subcommands

* Introduce `chia beta configure`

* Introduce `chia beta status`

* Test all `chia beta` commands

* Use a separate file logger for beta logs

* Write the plotting call args to the log file

* Sort potential submissions

* Some refactoring around log file log handler creation

* JSON dump the plotting args
2022-09-12 16:57:31 -05:00
dustinfaceandGitHub 84b0da7587 plotters: Move install_plotter into plotters (#12403) 2022-08-15 12:18:55 -05:00
Kyle AltendorfandGitHub 11b210809b run pre-commit across platforms in ci (#12287)
* run pre-commit across platforms in ci

why?  because we have developers working on multiple platforms.

* pre-clean for pre-commit in ci

* chia setup python action

* fixup

* fix windows specific hinting

* no TypeAlias yet

* debug

* debug

* adjust activated.ps1

* verbose pre-commit

* exit better

* Revert "debug"

This reverts commit 93a100afc7.

* less debug

* less `|`

* Update .pre-commit-config.yaml

* Update activated.ps1
2022-07-15 21:08:07 -05:00
dustinfaceandGitHub c400d816af harvester: Validate the path in add_plot_directory (#11625)
Make sure it exists, is a directory and hasn't been added already.
2022-05-24 13:21:44 -07:00
Zachary BrownandGitHub a1db012b7a Add libgmp-dev to Bladebit installation (#11406) 2022-05-05 08:17:14 -07:00
Zachary BrownandGitHub 3767fed513 update plotters installation to include an apt update before the apt install (#11400) 2022-05-02 15:56:49 -07:00
9ff3fc993f Ms.plot load perf2 (#10978)
* 2.7 seconds -> 0.45 seconds

* Merge

* Work on create_plots refactor

* Try to fix tests

* Try to fix tests

* Use new functions

* Fix block_tools by adding dir

* Extra argument

* Try to fix cyclic import

* isort

* Drop warning

* Some cleanups around `exclude_final_dir` and directory adding

* Cleanup `min_mainnet_k_size` checks

* Drop unrelated changes

* Fixes after rebase

* Fix cyclic import

* Update tests/block_tools.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update tests/block_tools.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2022-04-11 09:16:01 -07:00
Kyle AltendorfandGitHub a691d3c4b2 asyncio.get_event_loop() is deprecated in 3.10, stop using it (mostly) (#10418)
* asyncio.get_event_loop() is deprecated in 3.10, stop using it

https://docs.python.org/3.10/library/asyncio-eventloop.html#asyncio.get_event_loop
> Deprecated since version 3.10: Deprecation warning is emitted if there is no running event loop. In future Python releases, this function will be an alias of get_running_loop().

* black
2022-03-28 13:20:50 -07:00
Kyle AltendorfandGitHub 65c0829f66 madMAx executable discovery error handling cleanup (#9501)
* Add missing version=None for

* Log errors in get_madmax_install_info() instead of printing
2022-02-14 22:28:42 -08:00
JeffandGitHub 1fd7197df0 CLI fixes for plotting with madmax (#9197)
* Fixed waitforcopy when using madmax CLI

* Fixed madmax CLI usage with the tmptoggle -G option

* Don't specify a default for tmpdir2

* Removed defaults for tmp/tmp2/final dirs. Values must be provided.
2021-11-09 10:52:09 -08:00
JeffandGitHub c80208cda9 Terminate madmax/bladebit on Windows on Ctrl-C (#9113) 2021-11-04 08:59:13 -07:00
f550555c7d Add plotters. (#8497)
* Initial commit add plotters.

* Lint.

* Add progress for bladebit.

* Address some review comments.

* Oops...

* Add install option.

* Change chiapos to fit the old standard.

* Update chia/plotters/bladebit.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/install_plotter.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Lint.

* Remove farmerkey as required.

* Chia plotters chiapos works with no arguments.

* Added get_plotters RPC to support the GUI (#8530)

* Added 'get_plotters' daemon RPC. Probes installed/available plotters on behalf of the GUI.

* Linter fix

* Minor type tweak

* Tweaks.

* Run with default arguments all plotters.

* Fix bug.

* Update chia/plotters/bladebit.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Change bladebit repo.

* Update chia/plotters/bladebit.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Re-added the connect-to-daemon hidden option

* Use connect_to_daemon value for madmax and bladebit plot key resolution.

* Updated --tmp_dir, --tmp_dir2, --final_dir options to match old options from chia plots create

* Add CONNECT_TO_DAEMON as a valid option for madmax/bladebit

* Thread multiplier should be an int

* Passing params for madmax to start_plotting. Still needs cleanup/refactoring.

* Update chia/plotters/bladebit.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/plotters/madmax.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Filename option -z.

* Factor out calling the plotter.

* First attempt refactor install scripts.

* Switch to exec.

* Attempt to fix mypy warning.

* Remove filename.

* Increase RLIMIT_NOFILE for madmax on non-Windows platforms

* Add trailing path separator to madmax tmpdir/tmp2dir/finaldir arguments (required by madmax)

* Fixes to support madmax plotting from the GUI.
Writing output from the plotters now includes a flush to ensure the plotter log (used by the GUI) is updated frequently.

* Handle madmax's tmptoggle option internally when plotting with the GUI

* Formatting and linter fix

* Fixed the -i option for bladebit

* Construct BladeBit plotting options

* Cleanup code for building plotter command line options

* Added a post-processing step after each plotting job completes. Adds the final_dir plot directory as necessary.

* Fix plotter root path

* Reverting prior checkin. Need to figure out how to handle CHIA_ROOT being overridden

* BladeBit support for Windows

* BladeBit's --memory-json option is used to check memory requirements

* Madmax Windows support

* Plotters directory is now under CHIA_ROOT

* Madmax version detection

* BladeBit will default to 0 threads. BladeBit will max-out available threads with this configuration.

* LGTM fixes

* Module definition for chia.plotters to resolve mypy issues with chiapos (package vs chiapos.py)

* Updated BladeBit build script to account for 1.2.0 changes.
Replaced remaining subprocess.run calls with run_command.
Use BladeBit's reported memory requirement instead of hardcoded value.

* Show a disclaimer when using thirdparty plotter

* Test adding mac madmax plotter to the installers

* Get latest madmax from the latest GH release

* Fix bad var name

* m1 madmax

* Add linux/linux arm

* pip install -e for arm installer, so that its consistent with the other platforms when looking for additional files

* madmax + windows

* Get madmax with Invoke-WebRequest

* Use the correct windows slashes

* add madmax to the list of windows binaries

* Check if madmax exists on windows install and move it to site packages if it does

* Make sure windows has .exe extension for madmax

* Update azure to get latest version of madmax from GH releases

* Bladebit for linux/linux arm

* Fix error with binaries.extend

* Bundle bladebit + windows

* Fix download url for bladebit

* Check for bladebit in windows script. move to correct directory if it exists

* Detect and use packaged plotters

* Removed unnecessary import

* Updating the branch to use chiaplotters_gui for installer verification

* Removed a change that was intended for debugging only

* Remove disclaimer

* Updated for new madmax plotter with k33, k34 support.
Updated chia-blockchain-gui submodule

* Fixed typo

* Package the chia_plot_k34 executable

* Boink

* Revert "Boink"

This reverts commit 8d13c07110.

* Additional chia_plot_k34 spots that I missed

* pyinstaller.spec fix for chia_plot_k34

* Windows installer fix for chia_plot_k34.exe

* Restoring chia-blockchain-gui submodule to 047ce16 (as in main)

* Update to chiapos 1.0.6

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>
Co-authored-by: Jeff Cruikshank <jeff@chia.net>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
Co-authored-by: Earle Lowe <30607889+emlowe@users.noreply.github.com>
2021-10-28 15:37:46 -07:00