starred/chia-blockchain
1
0
Fork 0
mirror of https://github.com/Chia-Network/chia-blockchain.git synced 2026-08-24 02:24:23 -05:00
Code Issues Packages Projects Releases Wiki Activity
15,162 Commits 430 Branches 258 Tags
main
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Arvid NorbergandGitHub 6872e75a48 benchmark for blockchain.get_block_generator() (#9999) 2022-02-03 16:09:04 -08:00
Arvid NorbergandGitHub 13bf50833c add tool to test adding every block from a blockchain database (#10076) 
* add tool to test adding every block from a blockchain database, as a sanity check for a new version of the full node

* exercise full_node in artificial sync from scratch test

* add test exercising test_full_sync.py
 2022-02-03 10:31:49 -08:00
Arvid NorbergandGitHub 9153e82b1c avoid computing the header hash, when we already have it available (#10075) 2022-02-02 10:40:41 -08:00
Arvid NorbergandGitHub c33dc256c8 update analyze_blockchain tool to read v2 blockchain databases (#9983) 
* update analyze_blockchain tool to read v2 blockchain databases and introduce option to run all generators in mempool mode. Also update to use the new clvm_rs API

* update gnuplot file for the new output from analyze_blockchain.py
 2022-02-02 10:39:21 -08:00
Arvid NorbergandGitHub 7fb7a916ee Simplify FullNode._replace_proof (#10049) 
* simplify updating of proofs (when compacting them). This also saves two database lookups before the update

* fixup

* insert or ignore duplicate blocks

* when replacing a weight proof with a compact version, we do so by height. Rather than asking for all blocks at the specific height followed by a linear search for the one with the correct header hash, we can simply lookup the correct header hash and just look up that block. This simplifies the logic by dropping the looping over blocks.

* pass in header_hash to _replace_proof() to avoid the height-to-hash lookup
 2022-02-01 22:31:47 -08:00
Arvid NorbergandGitHub 3d063fea8f remove the GeneratorArgs type, and instead include the heights and generator programs as separate lists in BlockGenerator. The heights are not necessary when validating blocks, so this makes it easier to omit them in that case. The heights are only used when generating/farming a block. (#10006) 2022-01-31 18:49:39 -08:00
Arvid NorbergandGitHub 275969ddd1 remove redundant function, calculate_cost_of_program(). The cost is now part of the NPCResult object (#9964) 2022-01-31 18:49:12 -08:00
Arvid NorbergandGitHub e49b11d486 use pytest.fixture instead of pytest_asyncio.fixture in cmd test (#10040) 2022-01-31 18:48:58 -08:00
Arvid NorbergandGitHub 108f72a756 fix DB conversion with duplicate hints in the v1 database (#10041) 2022-01-31 18:48:37 -08:00
Arvid NorbergGitHubdustinface
 		b8ada1ceb7 introduce restrictions on generators at a specific height (#9957) 
* introduce restrictions on generators at a specific height. disallow division on negative numbers and disallow redundant leading zeros on integer condition arguments (produced by a generator)

* use SOFT_FORK_HEIGHT constant

* there is no need to specify height when validating block in block_creation

* Update tests/core/full_node/test_mempool.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
 2022-01-28 12:29:11 -08:00
Arvid NorbergandGitHub 2bb591c681 default log rotation to 50 MB (instead of 20) and make it configurable (#9986) 2022-01-27 15:54:20 -08:00
Arvid NorbergandGitHub a501c53228 default new installs to use v2 blockchain database schema (#9951) 
* default new installs to use v2 blockchain database schema. Leave an option to initialize a v1 database

* rename default database file to say v2
 2022-01-26 17:14:05 -08:00
Arvid NorbergandGitHub 5ebcd82c58 simplify run_block() (#9962) 
* more tests

* only run the generator once, in run_block(). Prior to this patch, we run it once per spent coin + 1
 2022-01-26 16:44:30 -08:00
Arvid NorbergandGitHub 38e8c8c355 simplify cost calculation. It's already computed in rust (#9931) 2022-01-26 09:07:22 -08:00
Arvid NorbergandGitHub 7acdf2d30f avoid converting puzzle_reveal and solution into python representation, just to convert it back again when running it (#9958) 2022-01-26 09:06:41 -08:00
Arvid NorbergandGitHub 234b31f21a move CoinSpend.hints() to a free function in the wallet. It does not belong in the full node and can only be called on coins whose program is trusted (#9953) 2022-01-26 08:52:41 -08:00
Arvid NorbergandGitHub 99eb7d0774 use context manager for get_client() (#9956) 2022-01-26 15:22:32 +02:00
Arvid NorbergandGitHub 1a84ee8db1 fix mempool issue where it would reject a spend bundle that spends an ephemeral coin with an ASSERT_SECONDS_RELATIVE 0 condition. Such condition is a no-op in the main blockchain validation, so the mempool should also allow it. (#9882) 2022-01-25 11:11:19 -08:00
Arvid NorbergandGitHub 5375c1a0af [v2] remove unused counter column in hint table (for v2 schema) (#9778) 
* remove unused counter column in hint table (for v2 schema)

* the new db file is expected to be empty. fail if any tables already exist
 2022-01-25 07:45:54 -08:00
Arvid NorbergandGitHub 8f83ba79bb remove one magic number from weight_proof (#9929) 2022-01-25 07:39:38 -08:00
Arvid NorbergandGitHub 77db90ba6a move member function get_memos() out of SpendBundle and into a free-function living in the wallet code. It does not belong to the full node. Also rename it to indicate that it's expensive (and dangerous) to call (#9907) 2022-01-23 21:45:41 -08:00
Arvid NorbergandGitHub 0554d92870 [v2] build indices last (#9856) 
* drop pragma temp_store and cache_size

* defer creating indices until the conversion is done

* increase commit rate
 2022-01-19 17:52:54 -08:00
Arvid NorbergandGitHub c780f0978c add chia function to convert blockchain database to v2 (#9613) 2022-01-19 11:43:56 -08:00
Arvid NorbergandGitHub fa16829a1f simplify run_generator by just returning the NPCResult. It already has an error field (#9850) 2022-01-18 11:05:57 -08:00
Arvid NorbergandGitHub 2851bedbcc sanity check the blockchain database on startup. If we can't find a peak, we also should not find any coins (#9780) 2022-01-13 14:53:48 -08:00
Arvid NorbergandGitHub 93d967525b revert allow-upgrades configuration option. It makes it significantly more complicated to do proper database migrations (#9768) 2022-01-13 14:53:27 -08:00
Arvid NorbergandGitHub 0ba838b7a8 New clvm generator api (#9645) 
* use the new run_generator2() and run_chia_program() API of clvm_rs

* bump clvm_rs dependency to 0.1.17
 2022-01-11 15:32:43 -08:00
Arvid NorbergandGitHub c143b71c22 log the synchronous mode used to open the blockchain database in (#9781) 2022-01-11 15:32:05 -08:00
Arvid NorbergGitHubKyle Altendorf
 		7ef619274a extend more tests to run with both v1 and v2 blockchain database schema (#9656) 
* extend more tests to run with both v1 and v2 blockchain database schema

* cleanup db_version fixture

* extend test timeouts

* Update tests/core/full_node/config.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update tests/conftest.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* fixup timeout

* fix typo in TODO comment

Co-authored-by: Kyle Altendorf <sda@fstab.net>
 2022-01-10 20:55:40 -08:00
Arvid NorbergandGitHub 659f4f4890 remove (redundant) spent field on CoinRecord. Replace it with a property (#9658) 2021-12-22 18:50:32 -08:00
Arvid NorbergandGitHub 3e895f9b62 remove zero condition cost constants. They only contribute complexity. They must be zero by consensus rules, they won't change. (#9657) 2021-12-22 18:49:52 -08:00
Arvid NorbergandGitHub b2ed21307b add tool to analyze the chain from the blockchain database, and run all generator programs (#9616) 2021-12-21 10:20:17 -08:00
Arvid NorbergGitHubKyle Altendorf
 		8ec2631d70 use context managers for database cursor objects & stream results (#9582) 
* use context managers for database cursor objects, to ensure they are closed. Also stream results back from cursors, rather than reading the full list of results up-front

* Update chia/full_node/block_store.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: Kyle Altendorf <sda@fstab.net>
 2021-12-20 10:39:53 -08:00
Arvid NorbergandGitHub f1dcd36c2a In main chain (#9550) 
* add in_main_chain column to full_blocks table

* add test for block_store in_main_chain

* test block_store invariant
 2021-12-17 14:37:45 -08:00
Arvid NorbergandGitHub 9de8235691 extend chia init to support initializing an (experimental) v2 blockchain database (#9599) 2021-12-17 14:35:43 -08:00
Arvid NorbergandGitHub fa3b90a089 simplify test_block_store (#9594) 2021-12-16 14:53:27 -08:00
Arvid NorbergandGitHub b554b0bfa3 simplify test_block_compression (#9590) 2021-12-16 09:44:10 -08:00
Arvid NorbergandGitHub 31bb2c987a remove block_records table, use the full_blocks table instead (with a new column) (#9510) 2021-12-15 15:25:00 -08:00
Arvid NorbergandGitHub a3c79c939d use zstd compression for full blocks (#9492) 2021-12-13 22:44:29 -08:00
Arvid NorbergandGitHub 2801d96477 unify the name we use for strict/safe mode (#9545) 
* unify the name we use for the special mode we run CLVM in, when accepting transactions into the mempool. We currently call this, strict mode, safe mode, safe and unsafe. Neither of these names are very descriptive. This patch renames this mode to mempool-mode

* import as
 2021-12-13 08:38:35 -08:00
Arvid NorbergandGitHub 5076348f2d when validating blocks in the blockchain, we don't use strict-mode (safe mode) (#9544) 2021-12-11 15:20:09 -08:00
Arvid NorbergandGitHub 35cab7cea2 Block store benchmark (#9520) 
* include SubEpochSummary objects

* make header_hashes authentic. include benchmarks of lookups on block_store as well
 2021-12-09 15:38:59 -08:00
Arvid NorbergGitHubdustinface
 		8db6ee540d [v2] Remove is_peak field (#9487) 
* remove is_peak field, instead, store the peak hash in a single place

* Update chia/full_node/block_store.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update chia/full_node/block_store.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
 2021-12-07 09:55:49 -08:00
Arvid NorbergandGitHub 84c48469ed remove unused field (is_block) from full_blocks and block_records tables (#9481) 2021-12-06 15:39:07 -08:00
Arvid NorbergandGitHub cd80f67d74 [v2] use binary fields in coin_store table (#9455) 
* use binary fields on coin_store table

* fix coin_store benchmark format string
 2021-12-06 11:22:12 -08:00
Arvid NorbergandGitHub a97b03ac56 use binary keys in block_store (#9454) 2021-12-06 11:21:48 -08:00
Arvid NorbergandGitHub 546011381d fix mypy issues in benchmark/block_store.py (#9457) 2021-12-05 09:36:52 -08:00
Arvid NorbergandGitHub 70d31a8c68 V2 database framework (#9442) 
* fix block_store benchmark linting issues

* basic support for database versioning

* update benchmark to run v1 and v2
 2021-12-03 13:52:10 -08:00
Arvid NorbergandGitHub a75b639adf fix typo in block_store benchmark (and missing clvm generator field) (#9433) 2021-12-01 11:31:31 -08:00
Arvid NorbergandGitHub a9d0346fd0 cache heigh-to-hash and height-to-ses maps between runs (#8771) 
* factor out map of block height to block hash and sub epoch summary from blockchain into its own class, BlockHeightMap

* save and restore the height-to-hash map and height-to-sub epoch summary map to flat files, to imrove startup time. Add unit tests

* add back assert
 2021-12-01 09:43:16 -08:00
Arvid NorbergandGitHub e10c58f66c Spent field (#9386) 
* ensure that the spent field is redundant. It can be deduced from (spent_index == 0)

* use spent_index instead of spent, in coin_record database

* since we don't use the spent-field, there's no need to query the database for it
 2021-12-01 09:42:15 -08:00
Arvid NorbergandGitHub b91a0eaddc avoid calling SpendBundle.fees() a second time in peak post processing 2. This is an expensive function, that's why we cache its result (#9434) 2021-12-01 09:41:50 -08:00
Arvid NorbergandGitHub fdaadc1ff9 add benchmark for block_store (#9425) 
* add benchmark for block_store

* use random instead of secrets
 2021-11-30 15:56:11 -08:00
Arvid NorbergandGitHub f030e32edf Name coin fields (#9381) 
* name the fields we pull from the coin_records database

* we don't need to ask for the coin_name in coin_record lookups, we never use it
 2021-11-30 06:43:54 -08:00
Arvid NorbergandGitHub 8d617f0df0 _blockchain_lock_queue is not always initialized by the time we shut down (#9402) 2021-11-29 10:58:14 -08:00
Arvid NorbergandGitHub 7f2be11da7 print database size in CoinStore benchmark (#9419) 2021-11-29 07:48:42 -08:00
Arvid NorbergGitHubneurosis69
 		aacdc03188 Sql index changes (#9349) 
* make the is_peak index smaller

* condition database updates on a configuration option

* drop unused coin_spent index

* drop redundant index on primary key

Co-authored-by: neurosis69 <83925572+neurosis69@users.noreply.github.com>
 2021-11-23 13:23:07 -08:00
Arvid NorbergGitHubxorinox
 		2263f9b399 improve coin_store benchmark (#9354) 
Co-authored-by: xorinox <61642896+xorinox@users.noreply.github.com>
 2021-11-23 09:53:59 -08:00
Arvid NorbergandGitHub 2fb72c1297 tell sqlite to use the puzzle_hash index when looking up coins by puzzle_hash (#9352) 2021-11-23 09:53:33 -08:00
Arvid NorbergandGitHub a29b1158ee fix lgtm issues (#9198) 2021-11-19 11:11:25 -08:00
Arvid NorbergandGitHub 897b7a4f41 catch BaseException (#8933) 2021-10-25 11:23:14 -07:00
Arvid NorbergandGitHub 0cc4251d4f simplify Blockchain.clean_block_records() (#8879) 2021-10-20 10:38:41 -07:00
Arvid NorbergandGitHub 732fb51fe0 add alternative to pkm_pairs_for_conditions_dict() which is a bit more straigh-forward and returns the public keys and messages in the forms we need them to validate them. This is a step towards simplifying the conditions data structures (#8128) 2021-10-19 06:26:53 -07:00
Arvid NorbergandGitHub 6de03d0fe1 Coin store updates (#8818) 
* improve coin store benchmark output

* factor out common row_to_coin() in coin_store, and some other slight cleanups
 2021-10-18 09:33:20 -07:00
Arvid NorbergandGitHub 1f893907b9 make db_sync setting default to FULL. Extend the configuration to cover all of OFF, NORMAL and OFF (#8836) 2021-10-16 09:02:18 -07:00
Arvid NorbergandGitHub dbe4829669 regenerate workflow files. fix test_hint_store.py (#8725) 2021-10-13 16:05:07 -07:00
Arvid NorbergandGitHub bbc2f76a21 remove unnecessary index in CoinStore & add additional benchmarks (#8641) 
* simplify and extend the coin store benchmark to include lookups by coin ID and get_coin_removed_at_height

* remove coin_spent index from coin store table. It's not necessary and slows down updates (and caused one mistaken query plan)
 2021-10-12 09:27:47 -07:00
Arvid NorbergGitHubdustinfaceNino Uzelac
 		61a513869d make synchronous mode in sqlite database connections configurable (#8753) 
* make synchronous mode in sqlite database connections configurabel. Default to enabling synchronous mode, leaving the door open for a more sophisticated heuristic in the future

* Update chia/util/db_synchronous.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update chia/util/initial-config.yaml

Co-authored-by: Nino Uzelac <uzelac.nino@gmail.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: Nino Uzelac <uzelac.nino@gmail.com>
 2021-10-08 06:13:06 -07:00
Arvid NorbergandGitHub 105fe6e1a5 don't create unused indexes in block_records and full_blocks tables (#8755) 2021-10-08 06:12:14 -07:00
Arvid NorbergandGitHub 15da736548 move test utility DBConnection out of test_coin_store and into its own file (#8756) 2021-10-08 06:11:38 -07:00
Arvid NorbergandGitHub f41ad9a2b0 clean up mempool test to use pytest parameters instead of for loops (#8740) 2021-10-08 06:11:04 -07:00
Arvid NorbergandGitHub 4745c60c11 add stress tests with malicious generators (#8570) 
* add tests with malicious generators

* raise limit for stress tests
 2021-09-29 11:41:49 -07:00
Arvid NorbergandGitHub 37c77b32b4 update create_coin hint test to use a list (#8628) 
* update create_coin hint test to use a list

* the hint/memo is always included in the CREATE_COIN conditions now
 2021-09-28 15:46:20 -07:00
Arvid NorbergandGitHub fceb39f783 bump clvm_rs version and add test for the optional 3rd hint parameter to CREATE_COIN (#8566) 2021-09-22 21:01:10 -07:00
Arvid NorbergandGitHub c41ccf6109 improve log formatting (#8543) 
* don't use scientific notation for coin_store timing log

* improve logging of timings in add_spendbundle

* improve timing logging of pre_validate_spendbundle. Pass in spend_name

* improve block validation timing logging

* enable coin_store timing logging unconditionally at DEBUG log level

* validation time formatting

* farmer response time formatting
 2021-09-21 10:48:44 -07:00
Arvid NorbergandGitHub 600a5f90a0 CoinStore benchmark and performance improvements (#8514) 
* add coin_store benchmark

* optimize _set_spent in CoinStore

* simplify _add_coin_record, since we won't use it to replace an entry anymore

* use executemany in add_coin_record and set_spent

* disable sqlite sync
 2021-09-21 10:43:43 -07:00
Arvid NorbergandGitHub 31dab2ae5f Rust only conditions (#8467) 
* no need to check announcements in python, it's all done in rust

* remove python condition parsing, use rust unconditionally

* no need to assert_my_amount, it's done in rust

* assert my puzzlehash is done in rust, no need to check it in python

* parent id is checked in rust, no need to do it in python

* my coin ID is checked in rust, no need to do it in python

* reorder condition checks to have to ones we won't see at the end

* no need for the RUST_CONDITION_CHECKER constant anymore

* update pool tests to use get_name_puzzle_condition
 2021-09-18 10:25:19 -07:00
Arvid NorbergandGitHub f5dca63048 improve CoinStore test (#8495) 
* improve CoinStore test by using pytest parameters (instead of for loops) and use a context manager for the DB connection

* use temporary filename for sqlite db
 2021-09-17 14:09:02 -07:00
Arvid NorbergandGitHub b748bb1356 Disable sqlite sync (#8319) 
* change sqlite pragma synchronous=FULL to pragma synchronous=NORMAL to improve disk I/O performance. In the case of catastrophic (hardware, power or kernel) failure, our database can be rebuilt

* removed redundant database pragmas in wallet
 2021-09-17 10:47:34 -07:00
Arvid NorbergandGitHub b66db03528 log timing of applying additions and removals to the coin store (#8493) 2021-09-17 10:41:23 -07:00
Arvid NorbergandGitHub 55baeb532a remove CoinStore's dependency on FullBlock. Just pass in the parts of the block necessary to add the block. Even though this is a net increase in lines of code (for now), it improves encapsulation of the CoinStore class and enables simpler tests of it (#8496) 2021-09-17 10:40:44 -07:00
Arvid NorbergandGitHub e4b9db08cd execute the sqlite pragmas only once, at the lever where the database is opened. Also, say FULL instead of 2, to make it clearer what the pragma is doing (#5993) 2021-09-15 10:57:13 -07:00
Arvid NorbergandGitHub 0b92c89e7d Conditions parser (#8393) 
* tests for invalid list terminators in conditions

* bump clvm_rs version
 2021-09-08 14:46:25 -07:00
Arvid NorbergandGitHub 6d60bfd58d add debug option to log all SQL commands (#8324) 2021-09-03 14:13:45 -07:00
Arvid NorbergandGitHub e32ddb765b calling SpendBundle.additions() or CoinSpend.additions() runs the CLVM program and parses the conditions outputs. It can be expensive and should not be done willy nilly (#8221) 2021-08-31 08:03:14 -07:00
Arvid NorbergandGitHub 463b971613 break problematic dependency where the streamable infrastructure depends on Program, G1Element, G2Element and PrivateKey. (#8237) 2021-08-31 08:02:40 -07:00
Arvid NorbergandGitHub 3f67425d06 simplify the block_tools test facility. We don't need to parse out the full conditions, just know what the total cost of the generator is (#8236) 2021-08-31 08:01:35 -07:00
Arvid NorbergandGitHub 70fb97c986 make TestCostCalculation deterministic and check against a specific cost (#8274) 2021-08-31 08:01:03 -07:00
Arvid NorbergandGitHub 98b001682f enable the rust condition checker unconditionally in testnet (#8122) 2021-08-24 10:53:46 -07:00
Arvid NorbergandGitHub 1cc047df1d tolerate fields replacing network constants in config.yaml that don't exist, but print warning (#8121) 2021-08-19 11:14:42 -07:00
Arvid NorbergandGitHub cd2f9d9d00 remove commented-out debug breakpoint()s (#8114) 2021-08-18 10:13:03 -07:00
Arvid NorbergandGitHub d81198384f fix mempool TX cache cost (#8054) 
* fix issue where the cost of the mempool TX cache (for spend bundles that can't be included in a block yet) would not be reset when the cache was emptied

* factor out the Pending TX cache from mempool, to allow unit testing
 2021-08-17 23:09:15 -07:00
Arvid NorbergandGitHub 5ee182932e Rust conditions (#7167) 
* use rust implementation of condition checker

* enable rust condition checker at specific block height

* bump switch-over height
 2021-08-17 14:57:41 -07:00
Arvid NorbergandGitHub 750a2759a3 test agg sig conditions (#7962) 
* agg sig conditions always take two args

* extend test a bit, add comment

* unit test agg sig parser where args list is not null terminated

* fix formatting
 2021-08-10 15:26:29 -07:00
Arvid NorbergandGitHub 7927a45945 remove initial transaction freeze feature (#7922) 2021-08-03 10:21:37 -07:00
Arvid NorbergandGitHub d380957460 remove trailing whitespace in generated workflow files (#7921) 2021-08-02 17:14:25 -07:00
Arvid NorbergandGitHub 485fab80b7 don't save unknown conditions (#7548) 
* don't save unknown conditions

* remove UNKNOWN enum from ConditionOpcode enum
 2021-08-02 14:40:32 -07:00
Arvid NorbergandGitHub 258de4522f remove unnecessary indentation in block_body_validation (#7916) 2021-08-02 14:39:17 -07:00
Arvid NorbergandGitHub c6bf1910d0 extend initial freeze test (#7580) 2021-07-27 10:30:48 -07:00
Arvid NorbergGitHubRichard KissAdam Kelly
 		51a7adf966 add mempool tests (#7519) 
* add mempool tests

* Update tests/core/full_node/test_mempool.py

Co-authored-by: Richard Kiss <him@richardkiss.com>

* Update tests/core/full_node/test_mempool.py

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>

* Update tests/core/full_node/test_mempool.py

Co-authored-by: Richard Kiss <him@richardkiss.com>

* Update tests/core/full_node/test_mempool.py

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>

Co-authored-by: Richard Kiss <him@richardkiss.com>
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
 2021-07-16 11:43:13 -07:00