* create new test.simulator module
to allow other packages to use the simulator without it crashing.
also tiny QOL fixes
* change user mode sim plot filter
Now only plots matching the user set fingerprint are loaded, this decreases the chance of user error.
* fix small oops in configure
* oops
* fix lint
* isort, who dis
* move folder over to chia/simulator
also change mypy settings again
* run pre-commit across platforms in ci
why? because we have developers working on multiple platforms.
* pre-clean for pre-commit in ci
* chia setup python action
* fixup
* fix windows specific hinting
* no TypeAlias yet
* debug
* debug
* adjust activated.ps1
* verbose pre-commit
* exit better
* Revert "debug"
This reverts commit 93a100afc7.
* less debug
* less `|`
* Update .pre-commit-config.yaml
* Update activated.ps1
* Simplify service start configuration loading
* oops
* more oops
* rename `config` to `service_config` as appropriate
* rename `full_config` to `config`
* new blob block api method integrated into wallet
* direct msg streaming of headers, rename, tests
* perform_handshake call fix
* updated trusted sync with new block header calls
* add max blocks limit to fetch
* added tests for rejected block header msgs
* avoid parsing transactions info if not required
* avoid looking up capabilities setting
* move block tests out of a class
* test fix
* Merge changes
* added docs and increased rate limits
* increased block header request interval from 32 to 128
* remove fetching hashes and use height range
* fetching by height in db v2
* update capabilities, other fixes
* fixed range block header call
* Add type hints
* Start work on optimizing fetch_last_tx_from_peer
* Huge speedup in trusted wallet sync
* Revert unintentional changes
* Fix trade issue
* Improve the code
* Str format
* Optimize handling of farming rewards
* Fix bug
* Performance fixes
* Optimizations to wallet syncing
* Don't return all coins in respond_additions
* Revert concurrency numbers
* More optimization of the caches
* Small optimization in coin_added
* Optimize request_additions significantly by using a cache
* fixes from feedback
* capabilities check fixes
* Increase rate limits to allow 250tps in verification requests
* Start work on rate limits
* New rate limit versioning support
* Revert unrelated changes
* revert return False
* Lint
* Revert cbi
* try tests with trusted peer
* Revert unrelated wallet changes
* Revert more debug changes
* Add test and throw on an error if not found
* Reject invalid requests
* Revert bad change with uint32, and change warning to info
* Parametrize wallet sync test
* Merge and LGTM
* More clean way to choose peers
* Fix lint
* add the new RejectBlockHeaders, RequestBlockHeaders and RespondBlockHeaders to the network protocol regression test and regenerate test files
* Rate limit diffs only
* Improve performance
* Simpler
* Lint
Co-authored-by: Sebastjan <trepca@gmail.com>
Co-authored-by: arvidn <arvid@libtorrent.org>
* Hint and lightly refactor chia.server.upnp.UPnP
* Remove chia.server.upnp from mypy.ini lax rules
* black
* Revert the pointless int for shutdown
* isort
* from __future__ import annotations
* Resend transactions (#11167)
* Resend transactions
* Don't recheck transactions more frequently than timeout
* Add wallet resend parameter to config, move timeout code out of tx store, but close to call site
* Add a test for wallet transaction resend
* Add test for wallet retry
* isort new files for precommit and update workflows
* Use correct fixture name
* LGTM - remove unused import
Co-authored-by: Earle Lowe <e.lowe@chia.net>
* less except -> false (#10219)
* less except -> false
* establish_connection() and .perform_handshake() can just return None
* remove more pointless, and no wrong, assertions
* Write python version error to stderr (#11239)
Co-authored-by: wjblanke <wjb98672@gmail.com>
* simplify SizedBytes and StructStream (#11429)
* simplify SizedBytes and StructStream
* lint
* super() !!!
* do not pass parameter up to super().__init__()
* Update chia/util/struct_stream.py
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
* parse fixed-width int data from class name
* add int512 and uint128 .from_bytes(), test .parse() failures
* test serialization against struct.pack()
* use typing_extensions for final
* override ignore
* stop using struct for StructStream
oh the irony
* fixup .to_bytes() to accept parameters again for where we use that
* bring back signed parameter
* format
* adjust tests for new exception
* eliminate custom coding for uint128 and int512
* tidy
* remove unused StructStream.PACK attribute
* add direct tests for parse_metadata_from_name()
* stricter hinting
* remove no-longer-needed typeshed work-around
* apply strict type checking to all touched files
* remove StructStream override of .to_bytes()
* tidy
* types touchup
* add unused parameter comments
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: wjblanke <wjb98672@gmail.com>
* Allow services to set a non-default max request body size limit (#11516)
* updated bls to 13 (#11529)
* reduce the redundant computations of coin_ids in block_body_validation (#11530)
* add test for streamable -> json conversion (#11527)
* add test for streamable -> json conversion
* fixup test
* Uses the new `from_bytes_unchecked` method in blspy, to improve perfo… (#11463)
* Uses the new `from_bytes_unchecked` method in blspy, to improve performance
* Update test
* Fix merge conflict
* Use from_bytes_unchecked in post_init and from_json
* More uniform code
* rename test files that are missing test_ prefix (#10712)
* rename test files that are missing test_ prefix
* update mypy and isort excludes
* skip test_get_host_addr6() in GitHub Actions under macOS
* rebuild workflows
* Coin Selection Refactor With CAT Coin Selection Refactor (#9975)
* add exact match and best exact match algorithms
* optimize algorithm further
this might be good.
* lint
* fix bad logic
* add final algorithms
* delete lint
* oops
* Update coin_selection.py
* simplify and fix knapsack algoritm
* simplify code and correct logic
* make it way better.
* clarify comments and check for edge cases.
* add comments and stuff
* improve coin selection
addressed comments
Thanks!
* add coin_selection rpc tests.
* clean up and add new unit tests
* undo test changes
* add extra test cases
* move coin_selection to its own function and switch to it for cat and main wallet.
* add cat tests
* lint
* make function align with standards
also removed test
* make test better
* add proper types
* Improve code clarity
* wallet: fix coin selection bugs
* wallet: add an assert just in case
* tests: add some sleeps to reduce flakiness
* Isort
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* fix bad merge
* lint
* fix tests
* address aforementioned changes.
* remove wallet test
* isort
* more tests and fixes
* lint
* rename to amount for coin selection rpc
* fix incase we have no smaller coins
* fix tests + lint
* re add asserts
* oops
missed me.
* lint
* fix test
* Squashed commit of the following:
commit 34a2235de5
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Apr 13 10:09:42 2022 -0400
clarify comment
commit adbf7f4f94
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Tue Apr 12 20:27:05 2022 -0400
linty lint
commit 5ebc1ac9fd
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Tue Apr 12 20:17:19 2022 -0400
add failure test and final changes
commit 7e5a21b4c2
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Tue Apr 12 19:35:18 2022 -0400
add descriptions and slim down code
commit 31c95b916d
Merge: d7b91295bd9b0ef5f3
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Mon Apr 11 10:12:05 2022 -0400
Merge branch 'jack-cat-coinselection' into jn_coinselection_dust
commit d7b91295b5
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Sun Apr 10 20:31:09 2022 -0400
lint
commit 30dc7c0ab4
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Sun Apr 10 20:25:52 2022 -0400
fix tests
commit 6c8c2e4874
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Thu Mar 31 15:06:00 2022 -0400
remove duplicate code.
commit 9f79b6f304
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Thu Mar 31 15:01:10 2022 -0400
address more concerns
commit 67c1b3929f
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Thu Mar 31 12:59:05 2022 -0400
fix logic error
commit 2d19a53245
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Thu Mar 31 11:47:52 2022 -0400
simplify and de duplicate code
commit 6ab1cc79bb
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 21:34:50 2022 -0400
add function and select individual coin
commit 582c17aa8d
Merge: ce2165942618fbaeba
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 21:14:37 2022 -0400
Merge branch 'jack-cat-coinselection' into jn_coinselection_dust
commit ce21659429
Merge: 16aabb3fd6daba28db
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 20:53:21 2022 -0400
Merge branch 'jack-cat-coinselection' into jn_coinselection_dust
commit 16aabb3fd5
Merge: 0b9fc28452286fe426
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 20:49:02 2022 -0400
Merge branch 'jack-cat-coinselection' into jn_coinselection_dust
commit 0b9fc28455
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 20:38:12 2022 -0400
lint
commit 62e74c72f4
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 20:34:22 2022 -0400
fix logic and tests
commit e738f44320
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 18:52:05 2022 -0400
deal with dust and add tests
* make sure that we do not use any dust
* minor change
* address concerns
* adjust comments
* adjust comment
Co-authored-by: Mariano Sorgente <sorgente711@gmail.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* rebuild workflows (#11543)
* remove the cache from CoinStore. It appears the cost of maintaining the cache outweighs the gains. (#11540)
* Keep daemon websocket alive during keyring unlock (#11371)
* When prompting to unlock the keyring during daemon launch, it's possible
that the daemon websocket connection will timeout before the user has
entered their passphrase. We can't quickly detect that the connection
has timed-out, so instead we now reconnect after collecting the
passphrase, unlock the keyring, and then close the old connection.
* mypy fix
* Switch to using asyncio.loop.run_in_executor to collect the passphrase on its own thread.
* Use a new ThreadPoolExecutor configured with 1 worker thread.
* Support searching derived addresses on testnet. (#11449)
* Support searching derived addresses on testnet.
* Added tests
* Optimize code to not perform useless subgroup checks (#11546)
* Optimize code to not perform useless subgroup checks
* Revert less important optimizations
* hints and strict type checking for test_wallet (#11541)
* hints and strict type checking for test_wallet
* fixup
* flip flop
* disable pytest-monitor by default (#11507)
* disable pytest-monitor by default
* take 2
* make recurse_jsonify() work directly on types (#11537)
* make recurse_jsonify() work directly on types, circumventing the dataclasses.asdict() step. This enables simpler integration of non dataclasses into the Streamable and JSON protocols
* add benchmark for Streamable.to_json_dict()
* tests: Split up and improve `test_wallet_rpc.py` (#11552)
This breaks up the one big test into
```
test_send_transaction
test_create_signed_transaction
test_create_signed_transaction_with_coin_announcement
test_create_signed_transaction_with_puzzle_announcement
test_send_transaction_multi
test_get_transactions
test_get_transaction_count
test_cat_endpoints
test_offer_endpoints
test_key_and_address_endpoints
```
Note that there is still much more room for improvements. Also the
following tests can still be split up more:
```
test_get_transactions
test_cat_endpoints
test_offer_endpoints
test_key_and_address_endpoints
```
* Remove unneeded uint64() intermediate in CAT wallet (#11575)
* Get get_args() and get_origin() from typing_extensions (#11571)
* Get get_args() and get_origin() from typing_extensions
* Update streamable.py
* cleanup
* Fix several flaky tests (#11576)
* Increase some of the timeouts to reduce flakiness
* Experiment with CI for flakiness
* Force more runs
* Add argument name
* Fix flaky mempool test
* Attempt to fix another flaky test
* Fix wallet retry test
* Missing arg
* Lint
* Increase benchmark test time
* Debug
* Debug
* Simplify retry test
* Lint
* More lint
* Return false instead of assert
* No need to check disconnect twice (since we don't break, this was flaky)
* Remove useless changes
* Accidental changes revert
* Accidental changes revert 2
* Accidental changes revert 3
* Revert fixture
* Fix a few additional flaky tests (#11597)
* Fix a few additional flaky tests
* Lint
* Update tests/core/full_node/stores/test_full_node_store.py
Co-authored-by: Jeff <jeff@chia.net>
* Fix merge conflict
* Remove unnecessary function
Co-authored-by: Jeff <jeff@chia.net>
* Bind port 0 to fix race condition when grabbing available ports (#11578)
* port 0 to fix flakiness
* Try fixing setup_full_system
* Try fixing setup_full_system, and lint
* More attempts to fix
* No more calls to get random ports in setup_nodes
* Revert accidental changes
* Timelord extra arg
* Try with port 0
* Fix daemon test, and lint
* Try without 0.0.0.0
* Back to 0.0.0.0
* Try a few timelord changes to get test running
* Increase timeout again
* Use the correct interface to get the port
* INFO logging to debug issue
* Revert "INFO logging to debug issue"
This reverts commit 7c379e5cca.
* Fix advertised port log
* Add extra log
* Logging back
* Rollback the timelord changes
* Try port 0 timelord
* Revert "Try port 0 timelord"
This reverts commit 4997faf3b2.
* Try full green, change ordering
* Remove unused var
* speed up simulation and cleanup
* Now try without the port config
* Fix a flaky call to get_event_loop
* Try getting the port dynamically
* No dynamic port
* Try changing the ordering
* Try adding a sleep
* Back to what works
* Timelord before vdf clients
* Dynamic port for 1st timelord
* Revert "Dynamic port for 1st timelord"
This reverts commit 0f322a15b7.
* Revert "Timelord before vdf clients"
This reverts commit 3286c34696.
* Revert "Back to what works"
This reverts commit 30380dffb7.
* Revert "Try adding a sleep"
This reverts commit 9212b665a6.
* Revert "Try changing the ordering"
This reverts commit a62597d70d.
* Revert "No dynamic port"
This reverts commit 5d2e15749b.
* Revert "Try getting the port dynamically"
This reverts commit ef9cd75679.
* Revert "Fix a flaky call to get_event_loop"
This reverts commit 01a000fdfb.
* Try one to 0
* Just not 0
* Don't get port dynamically
* Cleanup a bit
* Fix
* Some cleanup work
* Some cleanup work
* Fix daemon test
* Cleanup
* Remove arguments
* restore missing hints being stored as None (instead of 0-length bytes) (#11568)
* Coin simplification (#11567)
* factor out as_list() from Coin into free function. Remove unused name_str() from Coin. Minor optimization of hash_coin_ids() for common case.
* extend Coin unit test
* test installer on fedora:36, drop 33
* add tests.util.misc.assert_maximum_duration for benchmark assertions (#11589)
* add tests.util.misc.assert_maximum_duration for benchmark assertions
* fix some imports
* more
* lint
* correct duration
* hint fixup
* rename to caller_file_and_line()
* set default timer to thread_time()
* extract gc manager and default to disabling
* self calibrate overhead
* wrap none-ness behind .results() method
* add messages
* from __future__ import annotations
* yield out self
* final from typing_extensions
* split it
* just use Future
* tweak
* tweak
* correction in loosely hint checked file
* all future indexing in hints
* union for the hints
* rename
* assert_runtime()
* another rename...
* message -> label
* label all results with test names
* oops
* early return from _set_spent function (#11594)
* check for removals before calling function
* move condition to function
* remove redundant condition (#11582)
* fix built-in profiler to work for mac (#11590)
* Bump actions/setup-python from 2 to 3 (#10949)
* Bump actions/setup-python from 2 to 3
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 3.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v3)
---
updated-dependencies:
- dependency-name: actions/setup-python
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* Fixup templates for python@v3
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
* avoid cancelling release/** and long_lived/** (#11519)
* tests: Some adjustments in `test_offer_endpoints` (#11558)
This is a follow up after #11552 which basically used the initial parts
of `test_cat_endpoints` to make `test_offer_endpoints` working. Now this
PR drops a lot of the stuff we can assume to be not needed in this test
because its tested in `test_cat_endpoints` and it adds some general
improvments to it.
* async sleep and require connection reset error for DoS test (#11612)
* fix jsonify bool
* Remove is not None and length assertion in select_coins() (#11569)
* Remove is not None and length assertion in select_coins()
The `coins is not None` check seems unneeded since `select_coins()` is hinted to return `Set[Coin]`. The length requirement needlessly restricts from being able to request a zero amount. Either situation would either trigger an exception in the next assert, or trigger the assert itself for non-zero amounts requested.
* also remove from cat_wallet.py
* new port 0 stuff
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: Earle Lowe <e.lowe@chia.net>
Co-authored-by: Evan Graham <me@luakt.net>
Co-authored-by: wjblanke <wjb98672@gmail.com>
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: Jack Nelson <jack@jacknelson.xyz>
Co-authored-by: Mariano Sorgente <sorgente711@gmail.com>
Co-authored-by: Jeff <jeff@chia.net>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: neurosis69 <83925572+neurosis69@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
Co-authored-by: William Allen <wallentx@users.noreply.github.com>
* port 0 to fix flakiness
* Try fixing setup_full_system
* Try fixing setup_full_system, and lint
* More attempts to fix
* No more calls to get random ports in setup_nodes
* Revert accidental changes
* Timelord extra arg
* Try with port 0
* Fix daemon test, and lint
* Try without 0.0.0.0
* Back to 0.0.0.0
* Try a few timelord changes to get test running
* Increase timeout again
* Use the correct interface to get the port
* INFO logging to debug issue
* Revert "INFO logging to debug issue"
This reverts commit 7c379e5cca.
* Fix advertised port log
* Add extra log
* Logging back
* Rollback the timelord changes
* Try port 0 timelord
* Revert "Try port 0 timelord"
This reverts commit 4997faf3b2.
* Try full green, change ordering
* Remove unused var
* speed up simulation and cleanup
* Now try without the port config
* Fix a flaky call to get_event_loop
* Try getting the port dynamically
* No dynamic port
* Try changing the ordering
* Try adding a sleep
* Back to what works
* Timelord before vdf clients
* Dynamic port for 1st timelord
* Revert "Dynamic port for 1st timelord"
This reverts commit 0f322a15b7.
* Revert "Timelord before vdf clients"
This reverts commit 3286c34696.
* Revert "Back to what works"
This reverts commit 30380dffb7.
* Revert "Try adding a sleep"
This reverts commit 9212b665a6.
* Revert "Try changing the ordering"
This reverts commit a62597d70d.
* Revert "No dynamic port"
This reverts commit 5d2e15749b.
* Revert "Try getting the port dynamically"
This reverts commit ef9cd75679.
* Revert "Fix a flaky call to get_event_loop"
This reverts commit 01a000fdfb.
* Try one to 0
* Just not 0
* Don't get port dynamically
* Cleanup a bit
* Fix
* Some cleanup work
* Some cleanup work
* Fix daemon test
* Cleanup
* Remove arguments
* streamable: Merge `strictdataclass` into `Streamable` class
* tests: Test not supported streamable types
* streamable: Reorder decorators
* streamable: Simplify streamable decorator and force correct usage/syntax
* streamable: Just move some stuff around in the file
* streamable: Improve syntax error messages
* mypy: Drop `type_checking.py` and `test_type_checking.py` from exclusion
* streamable: Use cached fields instead of `__annotations__`
This is now possible after merging `__post_init__` into `Streamable`
* Introduce `DefinitionError` as `StreamableError`
* `/t` -> ` `
* protocols|server: Define new harvester plot refreshing protocol messages
* protocols: Bump `protocol_version` to `0.0.34`
* tests: Introduce `setup_farmer_multi_harvester`
Allows to run a test setup with 1 farmer and mutiple harvesters.
* plotting: Add an initial plot loading indication to `PlotManager`
* plotting|tests: Don't add removed duplicates to `total_result.removed`
`PlotRefreshResult.removed` should only contain plots that were loaded
properly before they were removed. It shouldn't contain e.g. removed
duplicates or invalid plots since those are synced in an extra sync step
and not as diff but as whole list every time.
* harvester: Reset `PlotManager` on shutdown
* plot_sync: Implement plot sync protocol
* farmer|harvester: Integrate and enable plot sync
* tests: Implement tests for the plot sync protocol
* farmer|tests: Drop obsolete harvester caching code
* setup: Add `chia.plot_sync` to packages
* plot_sync: Type hints in `DeltaType`
* plot_sync: Drop parameters in `super()` calls
* plot_sync: Introduce `send_response` helper in `Receiver._process`
* plot_sync: Add some parentheses
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* plot_sync: Additional hint for a `Receiver.process_path_list` parameter
* plot_sync: Force named parameters in `Receiver.process_path_list`
* test: Fix fixtures after rebase
* tests: Fix sorting after rebase
* tests: Return type hint for `plot_sync_setup`
* tests: Rename `WSChiaConnection` and move it in the outer scope
* tests|plot_sync: More type hints
* tests: Rework some delta tests
* tests: Drop a `range` and iterate over the list directly
* tests: Use the proper flags to overwrite
* test: More missing duplicates tests
* tests: Drop `ExpectedResult.reset`
* tests: Reduce some asserts
* tests: Add messages to some `assert False` statements
* tests: Introduce `ErrorSimulation` enum in `test_sync_simulated.py`
* tests: Use `secrects` instead of `Crypto.Random`
* Fixes after rebase
* Import from `typing_extensions` to support python 3.7
* Drop task name to support python 3.7
* Introduce `Sender.syncing`, `Sender.connected` and a log about the task
* Add `tests/plot_sync/config.py`
* Align the multi harvester fixture with what we do in other places
* Update the workflows
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Kyle Altendorf
Amine Khaldi
Jack Nelson
dustinface
Earle Lowe
