Simplify add_block_batch.
1. No need to accept AugmentedBlockchain and reconstruct it again.
2. No need to return both success and an unused optional error.
* make pre_validate_multi_processing() return futures, rather than the awaited and joined results from the thread jobs. This is crucial to allow the validaion jobs to run concurrently with the database interaction in the main thread
* Split up the add_block_batch() function into 3 parts, preparing them to be pipelined in long-sync
* pass AugmentedBlockchain into pre_validate_multiprocessing() and prevalidate_blocks(), to support the augmented blockchain to outlive a single batch
* pipeline pre-validation in sync_from_forkpoint()
* optimize the common case of _reconsider_peak(), to avoid one database lookup
* address review comments
* rename filter_blocks() -> skip_blocks(). It's is only relevant at the start of the sync, add comment and assert
* refactor
* minor fixes
* fix_tests
* add finish sync to add_blocks_in_batches
* add_blocks_in_batches updates wallets
* test fixes
* fix condition in add_block
* add known block to cache
* restore comment
* remove init file from log dir
* fix failed reorg test
* revert known block info change
* assert height in cat test
* wait for wallet height in nft test
* assert sync all height for all wallets
* test_cat_creation assert height
* rename vars
* assert height in process states
* combine current sub slot iterations, current difficulty and previous ses block into ChainState, to make it easier to pass around and update
* address review comments
* address review comments. rename ChainState -> ValidationState. add more comments
* refactor pre_validate_blocks_multiprocessing
* fix condtion
* get diff and ssi from blocks
* works_with_skip_ses
* add default
* propegate_to_header_validation
* pass data between batches
* remove redundant code
* merge fix
* remove optionals and refactor tests
* only use cache
* merge fix
* fix test
* dont traverse if we have prev sub epoch block
* return bad_prev error if block is missing
* mandetory param names
* clean
* fix_benchmark_test
* bump chia_rs to 0.7.0 and transition using sized ints and sized bytes from chia_rs
* transition StructStream to use the version from chia_rs
* Initial port of ConsensusConstants to Rust
* Revert max cost calculation to use int for now
* Remove SizedBytes, since it was moved to chia_rs
* Remove test_struct_stream, since it was moved to chia_rs
* use [0] * 100 instead of bytes(100)
* restore wallet_rpc_api, and make type hints match JsonDict (Any)
* fixup ConsensusConstants; restore return value of calculate_prefix_bits()
* fixup ConsensusConstants; construct int for intermedate cost values
* fixup ConsensusConstants; construct int for intermedate cost values
* fixup: transition StructStream to use the version from chia_rs
* revert changing type annotations for json dict
---------
Co-authored-by: Rigidity <me@rigidnetwork.com>
* more mypy platforms in ci
* check not checks
* oh yeah, activate...
* add 3.11
* adjust job names
* actually use the right python version for install
* adjust for 3.7 hinting support
* exclude macos 3.7 and macos arm 3.8
* back to 3.8 for now
* major_dot_minor
* oops
* move ignore comments for 3.7
* and more
* full_node: Rename `FullNode.resond_block` to `FullNode.add_bock`
The naming `respond_block` is confusing here imo, we have a full node
API method called `respond_block` but here in the `FullNode` class this
method is there to try adding a block to the full nodes's chain so i
think `add_block` is the better choice?
Also change the `respond_block: full_node_protocol.RespondBlock`
parameter to `block: FullBlock` which lets us get rid of many
`full_node_protocol.RespondBlock` wrappings.
* `FullNode.receive_block_batch` -> `FullNode.add_block_batch`
* `receive_unfinished_block` -> `add_unfinished_block` + change parameter
* `FullNode.respond_transaction` -> `FullNode.add_transaction`
* `FullNode.respond_end_of_sub_slot` -> `FullNode.add_end_of_sub_slot`
* `FullNode.respond_compact_vdf` -> `FullNode.add_compact_vdf`
* `respond_compact_proof_of_time` -> `add_compact_proof_of_time`
* `respond_transaction_semaphore` -> `add_transaction_semaphore`
* configure isort to add the future annotations import
* apply the new isort setting
* remove type ignores for new mypy (#13539)
https://pypi.org/project/mypy/0.981/
* another
* remove soft fork logic at height 2300000. The whole chain can be validated with the post-soft-fork rules. Remove testing of pre-soft-fork rules
* fix test_full_sync
* create new test.simulator module
to allow other packages to use the simulator without it crashing.
also tiny QOL fixes
* change user mode sim plot filter
Now only plots matching the user set fingerprint are loaded, this decreases the chance of user error.
* fix small oops in configure
* oops
* fix lint
* isort, who dis
* move folder over to chia/simulator
also change mypy settings again
* Some work on hints
* More cleanup and test_generator_tools
* Change relevant calls
* More refactor and cleanup
* More refactor
* Small fix
* Create wrapper object PeakPostProcessingResult
* Lint and small fix
* Fix a hint bug
* Fix hint update_wallets
* Fix test_full_sync
* pre-commit
* Start with PR review comments
* More efficient iteration
* Remove tx_removals_additions_and_hints
* Revert mozilla ca
* Fix issue with startup
* add inline executor and an option to run single-threaded
* add option to run test_full_sync in single-thread mode, to include block validation in profiles. Also attempt to speed it up by disabling db_sync
* pre-commit: Add a new hook to run `isort`
* contributing: Add hint about `isort`
* add isort to dev deps, ignore existing .py files, use black profile
* long list to lines not comma delimited
* isort: Update and sort ignore list to match latest `main`
* add bash command line to generate isort extend skip list (#3)
* add bash command line to generate extend skip list
* tidy
* isort: More files to ignore after rebase
* tests: Fix `test_wallet_user_store.py` after rebase
* Some fixes after rebase
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* add tool to test adding every block from a blockchain database, as a sanity check for a new version of the full node
* exercise full_node in artificial sync from scratch test
* add test exercising test_full_sync.py
Matt HauffandGitHub
Kyle AltendorfandGitHub
Richard KissandGitHub
Almog De PazandGitHub
Amine KhaldiandGitHub
Arvid NorbergandGitHub
dustinfaceandGitHub
Jack NelsonandGitHub
Mariano SorgenteandGitHub
Arvid NorbergandGitHub