Commit Graph
14659 Commits
Author SHA1 Message Date
8ec2631d70 use context managers for database cursor objects & stream results (#9582)
* use context managers for database cursor objects, to ensure they are closed. Also stream results back from cursors, rather than reading the full list of results up-front

* Update chia/full_node/block_store.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2021-12-20 10:39:53 -08:00
dustinfaceandGitHub 7949033929 tests: Use temp_dir as tmp2_dir, drop temp_dir if canceled (#9601)
If you currently cancel the test during the plot setup phase it just
removes the whole directoy and with it all the test plots for no good
reason. At least in my opinion its just annoying. Sure, you can clone
the `test-cache` repo and copy them over if this happens but i still
think its better to just merge this PR :)
2021-12-20 10:38:43 -08:00
dustinfaceandGitHub 86e7d694d2 tests: Avoid unintentional refresh cycles in test_plot_manager.py (#9603) 2021-12-20 10:38:11 -08:00
risnerandGitHub 598a0e8555 colorlog apparently changed, tests fail on colorlog-6.6.0 (#9620)
* Update test_simple_sync_protocol.py

* Update test_wallet_sync.py
2021-12-20 10:37:29 -08:00
Kyle Altendorf c33a8cf51c more # noqa: E501 2021-12-17 21:52:52 -05:00
Kyle Altendorf a51b21516a some hint ignores 2021-12-17 21:48:31 -05:00
Kyle Altendorf 55e67d8846 some hint fixups 2021-12-17 21:40:17 -05:00
Arvid NorbergandGitHub f1dcd36c2a In main chain (#9550)
* add in_main_chain column to full_blocks table

* add test for block_store in_main_chain

* test block_store invariant
2021-12-17 14:37:45 -08:00
Kyle AltendorfandGitHub 499d01a6c5 Extend the use of AmountWithPuzzlehash(TypedDict) and more uint64() (#9563) 2021-12-17 14:36:48 -08:00
dustinfaceandGitHub 2ab146bf2d plotting|tests|github: Fix and test re-loading/removal of plots where we missed the keys (#9578)
* tests: Allow to use custom keys in `BlockTools.new_plot`

* tests: Create three plots with `G1Element()` as farmer/pool key

But not add them as default plots. Just to have plots in the test cache with different keys from
within and outside the keychain for plot loading tests.

create

* test|github: Bump `test-cache` version to `0.28.0`

* plotting: Drop plots from `no_key_filenames` after the keys appeared

* plotting: Drop plots from `no_key_filenames` if the plot was removed

* tests: Test loading of plots where we miss the keys

* plotting: Use `x.copy()` instead of `list(x)`

* plotting: Add a comment about the missing keys plot removal

* tests: Fix after #9595
2021-12-17 14:36:29 -08:00
Arvid NorbergandGitHub 9de8235691 extend chia init to support initializing an (experimental) v2 blockchain database (#9599) 2021-12-17 14:35:43 -08:00
Mariano SorgenteandGitHub 2f86d0eee1 Ms.fix puzzle store (#9606)
* Fix puzzle store to 100 addresses

* Remove unnecessary function call

* Lint
2021-12-17 14:34:27 -08:00
Kyle AltendorfandGitHub 391667ad32 Merge pull request #9561 from Chia-Network/atari-merge_main_a3c79c939da2f65ec1fc5a6c9423ffae1e63cccd 2021-12-17 12:06:20 -05:00
234d121c2a chia|tests|pre-commit: Add missing __init__.py files (#8758)
* chia|tests: Add missing `__init__.py` files

* check and create needed __init__.py files via pre-commit

* plotting|tests: Handle `mypy` issues introduced by the `main` rebase

* pre-commit: Run `build-init-files.py` more verbose

* tests: Ignore `__pycache__` folders in `build-init-files.py`

* Update tests/build-init-files.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update tests/build-init-files.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* tests: Add shebang, make executable, add description

To `tests/build-init-files.py`

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2021-12-16 18:52:46 -08:00
dustinfaceandGitHub 08b5d68f2c tests: Move the expected_result_matched assignment out of the loop (#9595)
* tests: Move the `expected_result_matched` assignment out of the loop

This issue was introduced in #9505 and leads to false positives as soon 
as there was one of the attributes in `PlotRefreshResult` checked to be 
valid followed by an invalid one.

* tests: Fix `test_invalid_plots`
2021-12-16 18:52:32 -08:00
dustinfaceandGitHub aabc54bb53 plotting|tests: Fix try/except in PlotManager._refresh_task (#9588)
* plotting: Fix try/except in `PlotManager._refresh_task`

* tests: Move the start refreshing call into `PlotRefreshTester` ctor

It being in `run()` before where it was called every test iteration lead
to not detecting an issue with the try/except introduced in #9409. See
ee1dd1912f4678d7b960b670473bd4b3ffbd890b for the fix.
2021-12-16 15:14:34 -08:00
dustinfaceandGitHub a68ffecea1 plotting|tests: Fix and test removal of invalid plots (#9587)
* plotting: Drop from `failed_to_open_filenames` if the plot was removed

* tests: Test removal from `failed_to_open_filenames`
2021-12-16 15:14:12 -08:00
Kyle AltendorfandGitHub 7af4a58b63 DaemonProxy request IDs should be hinted as strings (#9584) 2021-12-16 15:13:41 -08:00
Arvid NorbergandGitHub fa3b90a089 simplify test_block_store (#9594) 2021-12-16 14:53:27 -08:00
Arvid NorbergandGitHub b554b0bfa3 simplify test_block_compression (#9590) 2021-12-16 09:44:10 -08:00
Kyle AltendorfandGitHub 1ec2fc1526 Fix new complaints from mypy 0.920 (#9586) 2021-12-15 17:52:25 -08:00
dustinfaceandGitHub 5496dc1971 tests: Call stop_refreshing in the test_environment fixture (#9567) 2021-12-15 15:27:37 -08:00
Arvid NorbergandGitHub 31bb2c987a remove block_records table, use the full_blocks table instead (with a new column) (#9510) 2021-12-15 15:25:00 -08:00
Kyle AltendorfandGitHub 64c433b615 Make sure to fail CI when unable to get madMAx version (#9573)
* Make sure to fail CI when unable to get madMAx version

* fail more
2021-12-15 10:00:36 -06:00
Chris MarslenderandGitHub a6ddf1bd36 Add token to Azure to get around github rate limits for anonymous users (#9576) 2021-12-15 09:46:09 -06:00
dustinfaceandGitHub b76b75f317 farmer|server|tests: Fix for farmer service crash if the keychain is empty (#9515)
* farmer: Run a start background task to wait if there are no keys yet

* farmer: Run the handshake in a background task if there are no keys yet

* tests: Yield the service in farmer/harvester setups

* tests: Introduce `start_service` in farmer/harvester setups

* server: Make `ChiaServer` and `Service` restartable

* tests: Test the delayed start/handshake in `test_farmer_harvester.py`

* tests: Move `test_farmer_harvester.py` to `tests/core/farmer_harvester`

Attempt to avoid issues due to the global block tools / shared keychain 
between tests

* farmer: Return `bool` in `Farmer.setup_keys`

* tests: Add `__init__.py` to `tests/farmer_harvester`
2021-12-14 18:03:59 -08:00
Chris MarslenderandGitHub fc0b1dd7a4 Crawler retry db on lock (#9562)
* Remove initial writes to DB, since there is nothing new to write at this point

Both of these methods are called later after each crawler iteration. The lines being deleted write all the same data back to the DB right after the initial data was loaded FROM the DB in the "create" method call, which is a waste of time, especially for mainnet where there are hundreds of thousands of records to write back

* Add additional logs around DB writes

* Add a retry around writing to the DB in case something else has a lock on the DB

Metrics processes read from this DB periodically and sometimes things time out just right and we get a lock exception. Without this retry, we'd hit the other exception handler which doesn't allow the loop to continue
2021-12-14 18:01:45 -08:00
Kyle Altendorf f2eb207fe2 undo unneeded formatting change 2021-12-14 15:36:41 -05:00
Kyle Altendorf 1946cfd65f lint 2021-12-14 15:11:26 -05:00
Kyle Altendorf 1b0b7f6082 mypy 2021-12-14 13:52:31 -05:00
Kyle Altendorf aefd6588a8 Extend the use of AmountWithPuzzlehash(TypedDict) and more uint64() 2021-12-14 13:49:57 -05:00
Kyle Altendorf b95dd3a18a remove no longer needed type ignores 2021-12-14 11:31:50 -05:00
Kyle Altendorf bd0ecc1646 Merge branch 'main' into atari-merge_main_a3c79c939da2f65ec1fc5a6c9423ffae1e63cccd 2021-12-14 11:14:10 -05:00
Arvid NorbergandGitHub a3c79c939d use zstd compression for full blocks (#9492) 2021-12-13 22:44:29 -08:00
xchdata.ioandGitHub 41ff79a86c Properly assign keyring observer to instance field (#9556) 2021-12-13 22:41:51 -08:00
JeffandGitHub 30a0ff02bc Restore peer_db_path and wallet_peers_path config keys/values for downgrading from a newer install. (#9557) 2021-12-13 22:01:14 -08:00
Kyle Altendorf 4c8e800cf2 cleanup 2021-12-13 23:12:07 -05:00
Kyle Altendorf c586372bd5 remove some not-yet used fixtures 2021-12-13 23:10:39 -05:00
Kyle Altendorf 0b14453ca0 recover some deleted tests 2021-12-13 23:05:08 -05:00
Kyle Altendorf 2fd5ce6eb6 Move data layer fixtures to conftest 2021-12-13 22:56:54 -05:00
Kyle Altendorf c34a91b0d3 Test DataStore.insert() with existing key/value pair 2021-12-13 14:57:04 -05:00
dustinfaceandGitHub 47c02aabd2 tests: Rework callback expectation checks with PlotRefreshTester (#9505) 2021-12-13 08:41:24 -08:00
Earle LoweandGitHub 7d801144ea Add TLS version to collected crawler stats (#9534) 2021-12-13 08:40:22 -08:00
Kyle AltendorfandGitHub 8bbbc7f9d8 Fix odd string format (#9508)
I would like to just have a list of strings instead of splitting, but let's at least have it be a clean string for now.
2021-12-13 08:39:36 -08:00
Arvid NorbergandGitHub 2801d96477 unify the name we use for strict/safe mode (#9545)
* unify the name we use for the special mode we run CLVM in, when accepting transactions into the mempool. We currently call this, strict mode, safe mode, safe and unsafe. Neither of these names are very descriptive. This patch renames this mode to mempool-mode

* import as
2021-12-13 08:38:35 -08:00
Arvid NorbergandGitHub 5076348f2d when validating blocks in the blockchain, we don't use strict-mode (safe mode) (#9544) 2021-12-11 15:20:09 -08:00
Matt HauffandGitHub c6637455b3 Added sorting functionality to /get_transactions (#9278)
* Added sorting functionality to /get_transactions

* Added comment about reversals

* Add determinism and clean up reversal

* Change default back to 50
2021-12-10 15:19:58 -08:00
Kyle AltendorfandGitHub bc2ae4bcbd Add pre-commit to dev dependencies (#9512) 2021-12-10 15:01:31 -08:00
Kyle AltendorfandGitHub 4ab1a8a798 Correct coin hints type hint to bytes (#9493)
* Correct coin hints type hint to bytes

* Update full_node.py
2021-12-10 15:01:09 -08:00
Kyle AltendorfandGitHub 44b4d69ad7 Fix some hints (#9484)
* Fix some hints

* remove unused import

* correct AmountWithPuzzlehash.amount to be uint64
2021-12-10 15:00:43 -08:00