Matt Hauff and GitHub
40db4635a8
[LABS-245] Enable PEP604 Ruff rules ( #20269 )
...
* Enable PEP604 Ruff rules
* Fix harcoded signature in test
* Hack CLVMStreamable test with note to fast follow
2025-11-18 12:34:00 -08:00
Florin Chirica and GitHub
7d077cdc46
DL Code Cleanup ( #19782 )
...
* DL code cleanup
* Lint.
2025-07-14 22:25:57 -04:00
Kyle Altendorf
fb8de7a1b3
Merge commit 'dac837e79961fa5ce9611d5531d789211031c465' into catchup/long_lived_datalayer_merkle_blob_from_main_dac837e79961fa5ce9611d5531d789211031c465
2025-07-11 22:35:57 -04:00
Matt Hauff and GitHub
f782fa8341
[CHIA-1315] Port (wallet) DataLayer endpoints to @marshal ( #19689 )
...
* Port `create_new_dl`
* Forgot default_raise
* Port `dl_track_new`
* Port `dl_stop_tracking`
* Port `dl_latest_singleton`
* Port `dl_singletons_by_root`
* fix test
* Port `dl_update_root`
* Port `dl_update_multiple`
* fix tests
* Port `dl_history`
* Port `dl_owned_singletons`
* Port `dl_get_mirrors`
* Make `Mirror` streamable (and fix test)
* Port `dl_new_mirror`
* push=True
* Port `dl_delete_mirror`
* Port `dl_verify_proof`
* Fix test
* fix import
2025-06-27 11:29:24 -07:00
Kyle Altendorf
668548cd33
Merge commit 'da20e4145fb6279d6880fd998b62456237e2f700' into catchup/long_lived_datalayer_merkle_blob_from_main_da20e4145fb6279d6880fd998b62456237e2f700
2025-06-23 22:03:50 -04:00
6050235bdb
[CHIA-3043] Add (hopefully) non-controversial ruff rules ( #19684 )
...
* Add (hopefully) non-controversial ruff rules
* mypy-exclusions.txt
* PIE
* Fix pre-commit?
* PYI
* RSE
* S + ignores
* Whitespace fix
* Re-add build-init-files.py
* Add comment about security ignores
* Update mypy-exclusions.txt
Co-authored-by: Kyle Altendorf <sda@fstab.net >
* Fix the executable status of build-init-files.py
* Set executable bit for tools/run_block.py and readd shebang
* Fix new errors
---------
Co-authored-by: Kyle Altendorf <sda@fstab.net >
2025-06-11 08:36:08 -07:00
+1
85c47c6649
catchup: into long_lived/datalayer_merkle_blob from main @ 329ce24c89 ( #19351 )
...
* update pin to chia-blockchain-gui (#19292 )
* Update the anchor against the checkpoint.
* mypy annotation for `test_dos.py` (#19036 )
* remove unused function, get_block_path()
* add type annotation to test_dos.py
* use chia_rs directly for foliage (#19308 )
* use chia_rs directly for weight proof (#19315 )
* build(deps): bump ruff from 0.8.5 to 0.9.6 (#19260 )
* build(deps): bump ruff from 0.8.5 to 0.9.6
Bumps [ruff](https://github.com/astral-sh/ruff ) from 0.8.5 to 0.9.6.
- [Release notes](https://github.com/astral-sh/ruff/releases )
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md )
- [Commits](https://github.com/astral-sh/ruff/compare/0.8.5...0.9.6 )
---
updated-dependencies:
- dependency-name: ruff
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
* Update source to ruff 0.9.6
* Apply suggestions from code review
Co-authored-by: Kyle Altendorf <sda@fstab.net >
---------
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Earle Lowe <e.lowe@chia.net >
Co-authored-by: Earle Lowe <30607889+emlowe@users.noreply.github.com >
Co-authored-by: Kyle Altendorf <sda@fstab.net >
* build(deps): bump mypy from 1.14.1 to 1.15.0 (#19319 )
* build(deps): bump mypy from 1.14.1 to 1.15.0
Bumps [mypy](https://github.com/python/mypy ) from 1.14.1 to 1.15.0.
- [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md )
- [Commits](https://github.com/python/mypy/compare/v1.14.1...v1.15.0 )
---
updated-dependencies:
- dependency-name: mypy
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
* adjust accordingly
* remove another
* and windows
---------
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net >
* build(deps): bump pyinstaller from 6.11.1 to 6.12.0 (#19318 )
Bumps [pyinstaller](https://github.com/pyinstaller/pyinstaller ) from 6.11.1 to 6.12.0.
- [Release notes](https://github.com/pyinstaller/pyinstaller/releases )
- [Changelog](https://github.com/pyinstaller/pyinstaller/blob/develop/doc/CHANGES.rst )
- [Commits](https://github.com/pyinstaller/pyinstaller/compare/v6.11.1...v6.12.0 )
---
updated-dependencies:
- dependency-name: pyinstaller
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* use chia_rs directly for sized bytes (#19326 )
* build(deps): bump keyring from 25.5.0 to 25.6.0 (#19150 )
Bumps [keyring](https://github.com/jaraco/keyring ) from 25.5.0 to 25.6.0.
- [Release notes](https://github.com/jaraco/keyring/releases )
- [Changelog](https://github.com/jaraco/keyring/blob/main/NEWS.rst )
- [Commits](https://github.com/jaraco/keyring/compare/v25.5.0...v25.6.0 )
---
updated-dependencies:
- dependency-name: keyring
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* build(deps): bump filelock from 3.16.1 to 3.17.0 (#19164 )
Bumps [filelock](https://github.com/tox-dev/py-filelock ) from 3.16.1 to 3.17.0.
- [Release notes](https://github.com/tox-dev/py-filelock/releases )
- [Changelog](https://github.com/tox-dev/filelock/blob/main/docs/changelog.rst )
- [Commits](https://github.com/tox-dev/py-filelock/compare/3.16.1...3.17.0 )
---
updated-dependencies:
- dependency-name: filelock
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* build(deps): bump pre-commit from 4.0.1 to 4.1.0 (#19166 )
Bumps [pre-commit](https://github.com/pre-commit/pre-commit ) from 4.0.1 to 4.1.0.
- [Release notes](https://github.com/pre-commit/pre-commit/releases )
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md )
- [Commits](https://github.com/pre-commit/pre-commit/compare/v4.0.1...v4.1.0 )
---
updated-dependencies:
- dependency-name: pre-commit
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* build(deps): bump types-setuptools from 75.6.0.20241126 to 75.8.0.20250225 (#19320 )
build(deps): bump types-setuptools
Bumps [types-setuptools](https://github.com/python/typeshed ) from 75.6.0.20241126 to 75.8.0.20250225.
- [Commits](https://github.com/python/typeshed/commits )
---
updated-dependencies:
- dependency-name: types-setuptools
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* build(deps): bump boto3 from 1.35.90 to 1.37.1 (#19322 )
Bumps [boto3](https://github.com/boto/boto3 ) from 1.35.90 to 1.37.1.
- [Release notes](https://github.com/boto/boto3/releases )
- [Commits](https://github.com/boto/boto3/compare/1.35.90...1.37.1 )
---
updated-dependencies:
- dependency-name: boto3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* use chia_rs directly for pool target (#19332 )
* address issue introduced in #19207 (#19328 )
address issue introduced in #19207 where empty transaction blocks would not have None transactions_block_generator
* CA Cert updates (#19312 )
adding ca updates
Co-authored-by: ChiaAutomation <automation@chia.net >
* bump chia_rs to 0.21.0 (#19337 )
* bump chia_rs to 0.21.0
* update mempool tests with the new, stricter, rules
* use chia_rs directly for reward chain block (#19336 )
* cache `get_unspent_lineage_info()` (#19335 )
* no need to pass get_unspent_lineage_info_from_puzzle_hash function into create_bundle_from_mempool(), we already have it as a member now
* cache results from get_lineage_info_for_puzzle_hash()
* fix
* fixup
---------
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: wjblanke <wjb98672@gmail.com >
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org >
Co-authored-by: Arvid Norberg <arvid@libtorrent.org >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Earle Lowe <e.lowe@chia.net >
Co-authored-by: Earle Lowe <30607889+emlowe@users.noreply.github.com >
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: ChiaAutomation <automation@chia.net >
Co-authored-by: Zachary Brown <z.brown@chia.net >
2025-03-05 11:52:01 -05:00
Kyle Altendorf and GitHub
2a53889d9c
correct datalayer side value types ( #19330 )
2025-03-05 09:30:51 -05:00
Kyle Altendorf
7a6e9b3d12
Merge commit '329ce24c89e29f8c7f0d4a8bb4d04fc2b0438d95' into catchup/long_lived_datalayer_merkle_blob_from_main_329ce24c89e29f8c7f0d4a8bb4d04fc2b0438d95
2025-03-04 16:43:20 -05:00
Kyle Altendorf and GitHub
d50b6e0883
use chia_rs directly for sized bytes ( #19326 )
2025-02-28 09:35:25 -08:00
Kyle Altendorf and GitHub
1567fa3df6
use chia_rs proof of inclusion for datalayer ( #19327 )
2025-02-27 14:19:38 -05:00
Kyle Altendorf and GitHub
c09d63ccb0
extract sibling hash and side methods as functions ( #19324 )
2025-02-26 10:52:21 -05:00
Kyle Altendorf and GitHub
5c68281a81
more datalayer rust transition prep ( #19306 )
...
* more datalayer rust transition prep
* tidy
2025-02-26 07:57:50 -05:00
Kyle Altendorf
b54c22faf0
Merge commit 'e446d6b726a03f6d917083b4d039abfec503e54e' into catchup/long_lived_datalayer_merkle_blob_from_main_e446d6b726a03f6d917083b4d039abfec503e54e
2025-02-25 11:30:14 -05:00
Kyle Altendorf and GitHub
e446d6b726
use chia_rs directly for sized ints ( #19303 )
...
* use chia_rs directly for sized ints
* Update data_layer_rpc_api.py
2025-02-24 14:33:09 -08:00
Kyle Altendorf and GitHub
849eef1b0a
remove ProofOfInclusion.as_program() ( #19305 )
2025-02-22 11:03:44 -05:00
Kyle Altendorf and GitHub
63d35aa560
Change delta format ( #19175 )
...
Change the delta file format to diff only against the previous
generation, as opposed to all existing generations. This maintains
backwards compatibility for now, but adds to the DB only the minimum
amount of hashes so we can sync. Once all clients update to this new
delta format, it will no longer be needed to store all the hashes into
the DB.
2025-01-31 09:46:05 -05:00
Florin Chirica
d07eefce8e
Add comments.
2025-01-30 19:59:18 +02:00
Kyle Altendorf
4d5f168ef5
make ProofOfInclusion.root_hash a method, from a property
2025-01-24 09:50:18 -05:00
Florin Chirica
11e1c623bb
Lint.
2024-10-30 02:28:44 +02:00
Florin Chirica
7765bbd504
Lint.
2024-10-29 06:01:22 +02:00
Florin Chirica and GitHub
81083b2176
Merge branch 'long_lived/datalayer_merkle_blob' into fc.dl_integration
2024-10-29 05:20:52 +02:00
Florin Chirica
de52d3899e
Migrate old DB.
2024-10-25 17:57:41 +03:00
Kyle Altendorf and GitHub
9c8557a1f8
Remove Python 3.8 support and update source to 3.9 standards ( #18687 )
...
* manual python 3.8 removals
* poetry
* pyupgrade
* black
* isort
* manually tidy imports
* work around shadowed type
* catch up build network protocol files for pyupgrade changes
* placate pylint
* re-enable 20.04
* placate flake8
2024-10-16 14:21:15 -07:00
Kyle Altendorf and GitHub
a6a27bfe8e
optimize datalayer hashing ( #18510 )
2024-09-11 09:41:54 -07:00
a4da91a86d
Changed: Enhance DataLayer Plugin Registration System for Improved Third-Party Integration ( #17711 )
...
* feat: load plugins from file
* feat: add tests
* feat: add file based datalayr plugin support
* fix: isort
* match mozilla-ca repo with main
* chore: fix linting
* add additional test coverage
* Update chia/util/misc.py
Co-authored-by: Kyle Altendorf <sda@fstab.net >
* move utility to datalayer folder
* fix lint
* use PluginRemote schema
* remove check
* remove check
* Update chia/data_layer/util/plugin.py
Co-authored-by: Kyle Altendorf <sda@fstab.net >
* load plugin using yaml.safe_load instead of json.load
* add default value to headers if doesnt exist
* fix: syntax error
* add log exception
* Update chia/data_layer/util/plugin.py
Co-authored-by: Kyle Altendorf <sda@fstab.net >
---------
Co-authored-by: Kyle Altendorf <sda@fstab.net >
2024-07-16 11:05:11 -07:00
Kyle Altendorf and GitHub
7de41fc9b4
tidy import aiosqlite as aiosqlite ( #18278 )
2024-07-15 13:36:01 -07:00
Kyle Altendorf and GitHub
cf7b54966e
root hash, does None mean empty or unspecified? ( #18244 )
...
* mark some cases where a none root hash means unspecified, not empty
* propagate, hint for help tracking, etc
* cp
* enum trick
* tidy
* tidy
* fixup
2024-07-11 13:44:36 -07:00
a58cc334a4
clvm mypie ( #16898 )
...
* clvm mypie
* update exclusions
* wip
* some
* undo for now
* more
* more
* drop the program usage in datalayer tests
* ignore tree hash errors
* ignore enough in _sexp_replace...
* again
* almost
* Fix test_singleton.py
* dl fixup
* did fixup
* touchup
* nft fixup
* fixup tests
* Revert "fixup tests"
This reverts commit 5df1a705ec .
* or this i guess
* blah
* Fix get_owner_did
* tidy
* blah
* `.as_atom()`
* `.as_int()`
* Update chia/wallet/nft_wallet/nft_puzzles.py
* `rrrff`
* stop using `CastableType`
* undo stuff around using `CastableType`
* `announcement_assertions: List[Program]`
* Update chia/types/blockchain_format/program.py
* more
* clvm==0.9.10
---------
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org >
Co-authored-by: Matt <quexington@gmail.com >
2024-05-22 12:28:07 -05:00
Kyle Altendorf and GitHub
98df4ce222
change datalayer python and logging from tree id to store id ( #17864 )
...
* change python and logging from tree id to store id
* always more
* oops
* again
* and...
* comment
* more
2024-05-10 13:45:45 -07:00
Amine Khaldi and GitHub
5350428eb1
Speedup leaf_hash in data_layer_util.py ( #17759 )
...
Speedup leaf_hash in data_layer_util.py.
2024-03-25 08:45:52 -07:00
Amine Khaldi and GitHub
cd6f4f56de
Speedup key_hash in data_layer_util.py ( #17758 )
...
Speedup key_hash in data_layer_util.py.
2024-03-25 08:45:37 -07:00
368fecc17e
DL multiple batch updates. ( #17414 )
...
* DL multiple batch updates.
* Lint.
* Lint.
* Add RPCs.
* Lint
* Lint
* Lint
* Add test.
* Lint.
* Add publish pending root option.
* Improve test.
* Fix cmd.
* Lint.
* Fix test
* Revert
* Add test.
* More tests.
* Improve test.
* Apply suggestions from code review
Co-authored-by: Kyle Altendorf <sda@fstab.net >
* Lint.
* Test.
* Latest local root.
* Change publish to submit.
* More submit instead of publish.
* Coverage.
* Lint.
* Improve coverage.
* Lint.
* Lint.
---------
Co-authored-by: Kyle Altendorf <sda@fstab.net >
2024-02-23 09:56:00 -08:00
fa220629ec
DL RPC pagination. ( #17347 )
...
* DL RPC pagination.
* Lint.
* Lint.
* Add RPCs.
* Lint.
* Lint.
* Add test.
* Lint.
* Add root_hash.
* Coverage.
* Fix bug.
* Update chia/cmds/data.py
Co-authored-by: Kyle Altendorf <sda@fstab.net >
* Dedupe.
* Lint.
* Add reference dict to test.
* Fix test.
* Fix test.
* Fix test.
* Fix test.
* Lint.
* Fix test
* Lint
* Remove short option.
* Add test.
* Lint.
* Update tests/core/data_layer/test_data_rpc.py
Co-authored-by: Kyle Altendorf <sda@fstab.net >
---------
Co-authored-by: Kyle Altendorf <sda@fstab.net >
2024-02-14 11:08:20 -08:00
63417e765a
Add support for generating and verifying DataLayer Proofs of Inclusions ( #16845 )
...
* initial implementaion of get_proof and verify_proof for datalayer
* Add wallet RPC and stop using offer blobs
* Don't require DL wallet to validate DL proof
* Convert to dl_verify_proof to free function
* More updates to DL proofs get/verify
* More updates to DL proofs get/verify
* More updates to DL proofs get/verify
* remove some unused verification functions
* oops deleted some important things
* fix type
* Add tests for DL proofs
* WIP using streamable classes for RPC
* Tests pass with new serializable classes
* use clvm tree hash calls instead of magic numbers and std_hash
* expanded tests to include more ways to call the same thing
* minor tests updates from feedback
* add in CLI tests
* Add separate create_multiple_keys_option
* Add helper and fixup key option
* Update tests/core/data_layer/test_data_rpc.py
Co-authored-by: Kyle Altendorf <sda@fstab.net >
* Add layer of indirection for the proof response
---------
Co-authored-by: Kyle Altendorf <sda@fstab.net >
2024-02-05 08:35:28 -08:00
Florin Chirica and GitHub
41c69f5144
DL Compress get_keys_values output by hash. ( #17243 )
...
* DL Compress get_keys_values output by hash.
* Lint.
* Lint.
* Lint.
* Remove get_node from delete.
2024-01-12 09:41:00 -06:00
Florin Chirica and GitHub
ebf0c00a21
DL: Use unsubscribe queue to relax subscriptions lock. ( #17068 )
2023-12-19 10:59:27 -06:00
Kyle Altendorf and GitHub
90c1441af0
apply pyupgrade --py38-plus --keep-runtime-typing ( #14646 )
2023-10-24 12:01:23 -05:00
Kyle Altendorf and GitHub
1bbfcfaf89
add new plugins: config allowing for custom headers ( #16194 )
...
* add new `plugins:` config allowing for custom headers
* add a test
2023-09-06 17:42:25 -05:00
a4471bfa8a
pass DataLayer root and generation instead of requerying ( #15613 )
...
* Cache root changes in insert_batch loop
* Reduce selects for root and generation
* remove sql logging, extracted to https://github.com/Chia-Network/chia-blockchain/pull/15690
* remove optimization extracted to https://github.com/Chia-Network/chia-blockchain/pull/15691
* remove optimization extracted to https://github.com/Chia-Network/chia-blockchain/pull/15728
* Delete test_batch_speed.py
benchmark provided in another already merged pr
* add `Root.to_row()`
* shift optimization to `DataStore.get_keys_values_dict()`
* rename to `intermediate_root`
* less tuple returning
* Revert "shift optimization to `DataStore.get_keys_values_dict()`"
This reverts commit b12ae08e3b .
* add `test_data_store.test_insert_batch_reference_and_side()`
---------
Co-authored-by: Kyle Altendorf <sda@fstab.net >
2023-07-19 13:47:17 -05:00
Kyle Altendorf and GitHub
f66b521067
Add DataLayer clear pending roots interfaces ( #15516 )
...
* add clear pending roots cli and rpc
* add tests
* drop extracted changes
* remove extracted change
* tidy
* add --yes
* handle todo
* more todo
2023-06-29 20:49:54 -05:00
433a414496
Add plugin_info and healthz endpoints to s3 plugin service, add chia data plugins check cli ( #15028 )
...
* Add plugin_id and healthz endpoints to s3 plugin service
* Add `chia data plugins check` cli
* plugin_id -> plugin_info
* updates based on reviews
* Use gather to run the lookups
* Update chia/data_layer/data_layer.py
Co-authored-by: Kyle Altendorf <sda@fstab.net >
* Fix typo
* short_help -> help
---------
Co-authored-by: Kyle Altendorf <sda@fstab.net >
2023-04-18 18:45:35 -05:00
Kyle Altendorf and GitHub
c678f00acb
cleanup adjacent string literals ( #14650 )
2023-02-22 17:16:07 -06:00
ed55423db6
Add atari get sync status RPC. ( #13440 )
...
* Add atari get sync status RPC.
* Try to fix test.
* Try to fix test.
* Some fixes.
* Update tests/core/data_layer/test_data_rpc.py
Co-authored-by: Kyle Altendorf <sda@fstab.net >
Co-authored-by: Kyle Altendorf <sda@fstab.net >
2022-10-28 15:05:03 -05:00
Kyle Altendorf and GitHub
5b39550f73
convert datalayer to DBWrapper2 ( #13582 )
...
* convert datalayer to DBWrapper2 (all write)
* more read, less write
* remove unneeded connection managers
* and... close it
* data store now creates its own wrapper
* Drop unused hint DataLayer.batch_update_db_wrapper
* require named arguments for most of `DBWrapper2.create()`
2022-10-03 17:50:12 -05:00
Amine Khaldi
98237e6227
Reflect the changes from main on atari related code.
2022-08-30 22:35:57 +01:00
Amine Khaldi
fd471e457f
Reflect the changes from main on atari related code.
2022-08-30 20:31:51 +01:00
Kyle Altendorf
2fbfe49605
Merge branch 'long_lived/atari' into atari-cancel_offer_rpc
2022-08-22 12:11:45 -04:00
Kyle Altendorf and GitHub
782f7cc41c
Merge pull request #13080 from Chia-Network/atari-fixup_fee_optionality
...
atari - fixup fee optionality
2022-08-20 09:20:27 -04:00
Kyle Altendorf
ebd4dfeaaa
atari - fixup fee optionality
2022-08-19 17:09:15 -04:00