* make pre_validate_multi_processing() return futures, rather than the awaited and joined results from the thread jobs. This is crucial to allow the validaion jobs to run concurrently with the database interaction in the main thread
* Split up the add_block_batch() function into 3 parts, preparing them to be pipelined in long-sync
* pass AugmentedBlockchain into pre_validate_multiprocessing() and prevalidate_blocks(), to support the augmented blockchain to outlive a single batch
* pipeline pre-validation in sync_from_forkpoint()
* optimize the common case of _reconsider_peak(), to avoid one database lookup
* address review comments
* rename filter_blocks() -> skip_blocks(). It's is only relevant at the start of the sync, add comment and assert
* refactor
* minor fixes
* fix_tests
* add finish sync to add_blocks_in_batches
* add_blocks_in_batches updates wallets
* test fixes
* fix condition in add_block
* add known block to cache
* restore comment
* remove init file from log dir
* fix failed reorg test
* revert known block info change
* assert height in cat test
* wait for wallet height in nft test
* assert sync all height for all wallets
* test_cat_creation assert height
* rename vars
* assert height in process states
* combine current sub slot iterations, current difficulty and previous ses block into ChainState, to make it easier to pass around and update
* address review comments
* address review comments. rename ChainState -> ValidationState. add more comments
* slightly simplify call to pre_validate_blocks_multiprocessing. batch size is always 4, check_filter is always True. Remove the member function wrapper around it in Blockchain, to prepare for being able to pass wrapped blockchains into it
* address review comments
* Apply suggestions from code review (addendum).
---------
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>
change the type of ref_list to List[bytes] instead of List[SerializedProgram]. Also, avoid passing the block generator twice to the worker process for validation
* turn BlockchainInterface into a proper Protocol, to allow mypy to check it
* split out a subset of BlockchainInterface into BlockRecordsInterface
* separate BlockCache into the (simple) production use case and the (complex) test use case of mocking a blockchain object
* temp
* Port `test_cat_creation`
* Port `test_cat_creation_unique_lineage_store`
* Delete `...reuse_address` test
* Port `test_get_wallet_for_asset_id`
* Port `test_cat_doesnt_see_eve`
* Port `test_cat_max_amount_send`
* Port `test_cat_hint`
* Port `test_cat_change_detection`
* Name test amount and fees in test_cat_creation.
* Use set_remainder for first test
* Match the ValueError we're expecting in test_cat_max_amount_send.
* Comment.
---------
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>
* remove block_height_list from BlockGenerator. This field is only used when farming blocks, and we don't use it in production right now. We only use it in the tests, BlockTools. It has been updated with a cleaner and more direct way of setting the block reference list in new blocks
* test
* refactor new_peak_timelord
* fix test
* improve test assertions
* lint
* return after skipping peak
* improve comments
* limit condition to a diff of one block
* remove log
* refactor test
* better conditions
consider overflows as well
* pre commit
* test with overflow
* add test for EOS case
* lint
* comment
* prefer lower iterations peaks
* test full node receiving same weight lower iterations
* lint
* fix reorg test, lint
* fix block tools bug
* use signage point param in test
* use signage point param for overflows as well
* remove unused case
* fix block_tools condition
* lint
* lint
* Add check for peak not none in test
---------
Co-authored-by: Earle Lowe <e.lowe@chia.net>
* Change a bunch of info logs back to warning in crawler, since crawlers are typically run in warning mode, to avoid the info log connection spam
* Ensure that the crawler loop ONLY doesn't start when in tests
* Rename the crawler service to crawler_service_no_loop, so defaults can also be tested WITH a loop
* Test crawler loop modes
* Add message when we received state chaged callback
* Add log to indicate crawling has started and how many peers we're crawling
* Add the concept of 'action scopes'
* Add `WalletActionScope`
* Fix CATWallet pending_change calculation
* Add action_scope: WalletActionScope to all tx endpoints
* Add usage of action scope throughout wallet codebase
* Add the concept of 'action scopes'
* pylint and test coverage
* add try/finally
* add try/except
* Undo giving a variable a name
* Fix CRCAT test
* Fix trade tests
* Fix cat test
* Fix miscellaneous test
* Fix miscellaneous test
* Fix miscellaneous test
* pylint
* Add auto claim test coverage
* Take advantage of extra_spends
* Fix tests
* test fixes
* pylint
* Add the concept of 'action scopes'
* Add `WalletActionScope`
* Fix CATWallet pending_change calculation
* Add action_scope: WalletActionScope to all tx endpoints
* Add the concept of 'action scopes'
* pylint and test coverage
* add try/finally
* add try/except
* Undo giving a variable a name
* Fix CRCAT test
* Fix trade tests
* Fix cat test
* Fix miscellaneous test
* Fix miscellaneous test
* Fix miscellaneous test
* Add auto claim test coverage
* Move capabilities underneath Capability
* Split default capabilities for each service
* Make get_capabilities take in node_type
* Remove optionality
* Refactor get_capability_overrides
* make BlsCache a proper class, rather than a global dict with helper functions
* make the BLS cache a member of the FullNode, rather than a global variable
* add test for BAD_AGGREGATE_SIGNATURE when using the BLS cache
* bump chia_rs to 0.7.0 and transition using sized ints and sized bytes from chia_rs
* transition StructStream to use the version from chia_rs
* Initial port of ConsensusConstants to Rust
* Revert max cost calculation to use int for now
* Remove SizedBytes, since it was moved to chia_rs
* Remove test_struct_stream, since it was moved to chia_rs
* use [0] * 100 instead of bytes(100)
* restore wallet_rpc_api, and make type hints match JsonDict (Any)
* fixup ConsensusConstants; restore return value of calculate_prefix_bits()
* fixup ConsensusConstants; construct int for intermedate cost values
* fixup ConsensusConstants; construct int for intermedate cost values
* fixup: transition StructStream to use the version from chia_rs
* revert changing type annotations for json dict
---------
Co-authored-by: Rigidity <me@rigidnetwork.com>
* Port test_wallet_coinbase
* Port test_wallet_make_transaction
* Port test_wallet_reuse_address
* Port test_wallet_clawback_claim_auto
* Port test_wallet_clawback_clawback
* Port test_wallet_clawback_sent_self
* Port test_wallet_clawback_claim_manual
* Port test_wallet_clawback_reorg
* Rebase fixes and flakiness improvements
* Port test_get_clawback_coins
* Port test_clawback_resync
* Port test_wallet_coinbase_reorg
* Port test_wallet_make_transaction_hop
* Port test_wallet_make_transaction_with_fee
* Port test_wallet_make_transaction_with_memo
* Port test_wallet_create_hit_max_send_amount
* Port test_wallet_prevent_fee_theft
* Port test_wallet_tx_reorg
* Port test_address_sliding_window
* Port test_sign_message
* Port test_wallet_transaction_options
* Remove unneeded check_balances call.
* Introduce tx_amount and use it across test_wallet_clawback_claim_auto.
* Reapply b79ecd2 (got lost in the recent merge).
* Minor improvements to test_wallet_clawback_clawback:
* Introduce tx_amount and test_fees and use them to clarify test conditions.
* Remove no longer needed json.dumps(resp) call.
* Remove unneeded dict() call on a {} constructed dictionary.
* Remove accidental change (addendum to bebc87f).
* More tx_amount and test_fees.
* Remove no longer needed json import.
* Remove unneeded dict() calls on {} constructed dictionaries.
* Introduce tx_amount2 in test_clawback_resync and clarify state transitions.
* Clarify state transitions in test_wallet_make_transaction_hop.
* Simplify transaction_has_failed.
* Just some consistency.
* Remove unneeded check_balances calls.
---------
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>
* Update source to black 24 spec
* update black to 24.1.1
* update pylint to ignore multiple-statements
* update test file
* black updates
* format updates
* Update test_network_protocol_files.py
* Update test_network_protocol_files.py
* undo redundant type-casts from slots, SubEpochSummary and SubEpochData
* undo redundant type-casts from proof-of-space, reward chain and foliage types
* undo redundant type-casts from VDFInfo, VDFProof and ClassgroupElement
* remove unnecessary type annotation
* Don't autopush transactions by default anywhere
* Remove ignore_max_send_amount
* Use TXConfig instead of pending_transactions for coin exclusion in spend_clawback_coins
* Unify transaction pushing into a single method
* Address offline comments by @emlowe
* Rework max send amount to be based on coin quantity rather than amount
* Inadvertent changes
* Test fixes
* Inadvertent change
* Bad import?
* Inadvertent changes?
* readd missing test
Augment conditions_dict_for_solution and conditions_for_solution to accept both programs and serialized programs.
This allows us to use serialized programs without the need to convert them into programs.
* Begin add to support for third-party fee-based harvesters
Signed-off-by: Harold Brenes <h.brenes@chia.net>
* Return FoliageBlockData back to Farmer from FullNode
* Send FoliageBlockData all the way to the harvester, if requested
* Preparing tests for 3rd party harvester support
* Continue writing tests for 3rd party harvester
* Some cleanup. Drafting
* Test full message chain of third-party Harvester reward overwrite.
Added test that validates messages between Farmer, Harvester and FullNode
to ensure that the correct data is used when an overwritten farmer reward
address is given by a third-party Harvester service.
* Changing Harvester signature request flow
to include original data, if requested.
This makes it so that NewSignagePoint unconditionally
always includes the source of the SP hashes.
* Fix incorrect field name
* Harvester can ask for original signing data without
overwriting the reward address also.
* Update test_farmer_handles_farmer_reward_address_override to reflect changes.
- Correct farmer API get the correct data source for the signage point signature request.
- Trivial fixes
* Removing some obsolete stuff
* Trivial
* Fix tests in updated harvester protocol
- Fix including source data when
include_source_signature_data is set.
- Set source data fields optional.
- Update test so that a timelord is started and we get proper signage points
instead of aritficial ones. This helps us
make sure we test for all the source data
cases needed from the full node.
* Refactor harvester source signing data test
- Remove older harvester signing data test
(had missing coverage for data)
- Add harvester fee convention logging
- Begin fee convention logging test
- Refactor fixtures for harvester enhancement protocol tests
Start harvester
* Complete test for harvester fee convention
* Update SignatureRequestSourceData message
- SignatureRequestSourceData is now serialized as enum + bytes.
- Update usage locations for SignatureRequestSourceData.
- Update tests to reflect this
* Missing test from last commit
* Revert to explicit None checks, instead of relying on truthiness
Revert scratch work
* Linting
* adjust for anyio
* Moe linting
* mypy fixes
* Documentation comments
* Move third-party harvester protocol tests
Moved to own directory in order to enable `install_timelord = True`
* mypy fixes
* checkout_blocks_and_plots = True
* Update some log messages, syntax and docs
* Bump timeout in tests
* Update protocol message types for
harvester and farmer for test_missing_messages`
* Regenerate protocol message data for protocol tests
* Remove timelord requirement for tests.
Inject pre-generated signage points and blocks instead.
* Linting
* More linting
* More linting
* Merge from main and lint again
* Remove customized test plots requirements
* Rebase fix
* Coverage update
- Update valid fee quality test to be unit, instead of pseudo-integration.
- New test for invalid fee quality.
- More explicit messaging for invalid fee qualities.
- Explicitly comment out uncovered case in test.
- Update types for mypy.
* Lint
* Lint
* Lint, better logs
* add protocol file tests
* coverage
* Update formatting in fee quality logging
* Rebase fix
* Regenerate protocol_message_bytes-v1.0
---------
Signed-off-by: Harold Brenes <h.brenes@chia.net>
Co-authored-by: Harold Brenes <h.brenes@chia.net>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: Harold Brenes <152330941+haorldbchi@users.noreply.github.com>
Co-authored-by: arvidn <arvid@libtorrent.org>
* update full_node_store to support multiple UnfinishedBlocks with the same reward hash
* extend full node store to support recording outstanding requests for v2 unfinished blocks
* add test for full_node_store
* simplify creation of UnfinishedBlock in test_full_node.py
* extend UnfinishedBlock protocol to support distinguishing between unfinished blocks with the same partial hash but different foliage
* add test for NewUnfinishedBlock and NewUnfinishedBlock2
* when asking the full_node_store for an unfinished block, only by its partial hash; return the highest ranking (lowest foliage tx block hash)
* use rust types for slots, SubEpochSummary and SubEpochData
* use rust type for EndOfSubSlotBundle
* use rust types for SubEpochChallengeSegment, SubEpochSegments and SubSlotData
* use rust types for VDFInfo, VDFProof and ClassgroupElement. Since these types aren't dataclasses, the dataclass.replace() is changed to method replace()
* add test for recursive_replace()
* add comment in recursive_replace() hack
Matt Hauff
Arvid Norberg
Almog De Paz
Kyle Altendorf
Chris Marslender
Rigidity
Earle Lowe
StartToaster
