Commit Graph
239 Commits
Author SHA1 Message Date
Arvid NorbergandGitHub 5ae87189c4 Rust coin (#11807)
* use rust implementation of Coin class, from chia_rs

* bump chia_rs dependency
2022-06-13 07:21:04 -05:00
ab53d48995 Request header blocks, and new rate limits (#11636)
* new blob block api method integrated into wallet

* direct msg streaming of headers, rename, tests

* perform_handshake call fix

* updated trusted sync with new block header calls

* add max blocks limit to fetch

* added tests for rejected block header msgs

* avoid parsing transactions info if not required

* avoid looking up capabilities setting

* move block tests out of a class

* test fix

* Merge changes

* added docs and increased rate limits

* increased block header request interval from 32 to 128

* remove fetching hashes and use height range

* fetching by height in db v2

* update capabilities, other fixes

* fixed range block header call

* Add type hints

* Start work on optimizing fetch_last_tx_from_peer

* Huge speedup in trusted wallet sync

* Revert unintentional changes

* Fix trade issue

* Improve the code

* Str format

* Optimize handling of farming rewards

* Fix bug

* Performance fixes

* Optimizations to wallet syncing

* Don't return all coins in respond_additions

* Revert concurrency numbers

* More optimization of the caches

* Small optimization in coin_added

* Optimize request_additions significantly by using a cache

* fixes from feedback

* capabilities check fixes

* Increase rate limits to allow 250tps in verification requests

* Start work on rate limits

* New rate limit versioning support

* Revert unrelated changes

* revert return False

* Lint

* Revert cbi

* try tests with trusted peer

* Revert unrelated wallet changes

* Revert more debug changes

* Add test and throw on an error if not found

* Reject invalid requests

* Revert bad change with uint32, and change warning to info

* Parametrize wallet sync test

* Merge and LGTM

* More clean way to choose peers

* Fix lint

* add the new RejectBlockHeaders, RequestBlockHeaders and RespondBlockHeaders to the network protocol regression test and regenerate test files

* Rate limit diffs only

* Improve performance

* Simpler

* Lint

Co-authored-by: Sebastjan <trepca@gmail.com>
Co-authored-by: arvidn <arvid@libtorrent.org>
2022-06-11 01:35:41 -05:00
1aeeffedbe update unknown condition test (#11746)
* since condition opcode 1 is about to become well defined, use some other opcode to test unknown conditions

* Update tests/core/full_node/test_mempool.py

Co-authored-by: Richard Kiss <him@richardkiss.com>

Co-authored-by: Richard Kiss <him@richardkiss.com>
2022-06-03 12:36:17 -05:00
Arvid NorbergandGitHub 3c2fc1947d upgrade chia-rs to streamable support (#11503)
* upgrade chia-rs to streamable support

* bump chia_rs dependency
2022-06-01 12:56:11 -05:00
8b2f7ec24c Add Windows CI (#10681)
* Add Windows CI

* rebuild workflows

* rebuild workflows

* try different activation

* catch up windows workflow template

* debug dir

* hard code dev extra on windows

* it is pytest

* .exe

* -m pytest for windows

* --module pytest

* Revert "debug dir"

This reverts commit 74b5bbfd92.

* debug pkg_resources import on windows

* debug tests.conftest import on windows

* rebuild workflows

* debug

* debug oops

* debug more

* debug more

* Update pytest.ini

* Update pytest.ini

* fix it

* Revert "debug more"

This reverts commit 367ed61dfa.

* Revert "debug more"

This reverts commit 772b6707c6.

* Revert "debug oops"

This reverts commit 5065c1e172.

* Revert "debug"

This reverts commit ecf721e453.

* Revert "debug tests.conftest import on windows"

This reverts commit ea71a51625.

* Revert "debug pkg_resources import on windows"

This reverts commit eb6afb1dfb.

* more warning ignore

* they're regexes...

* --numprocesses 0 for windows ci

* specify each test file individually since pytest on windows does not seem to handle the wildcards

* relative...

* select most recent supported python on windows, support INSTALL_PYTHON_VERSION

* sort test files when generating workflows

* report full python version on windows, actually use the selected version for the env

* oops

* debug dir

* skip timelord-requiring jobs on windows

* oops

* handle wildcard expansion for coverage on windows

* rebuild workflows

* download blocks and plots on windows

* correct and and or in yaml

* -d for Install.ps1

* git config --global core.autocrlf false

* add missing windows plot_sync workflow

* no check resource usage on windows

* delete outdated windows workflows that need the timelord

* catch up windows workflow template

* try again without xdist

* rebuild workflows

* Revert "try again without xdist"

This reverts commit 4bedf5f047.

* maybe now no xdist on windows

* maybe now no xdist on windows

* rebuild workflows

* debug chiapos==1.0.9

* Revert "debug chiapos==1.0.9"

This reverts commit 4b374e128a.

* debug blspy==1.0.9

* catch up windows workflow template

* rebuild workflows

* Various windows CI fixes

* Windows CI fixes and workflow updates

* ignore another warning for the keyring

* pre-commit fixup

* rebuild workflows

* catchup windows test workflows with long_lived/** and release/** setup

* Update test_full_sync.py

* Try without parallelization

* Some other tweaks for windows

* some more timeout tweaks

* more timeout tweaks for windows

* rebuild workflows

* Adjusting more timeouts

* Skip ipv6 test on windows

* More timeout tweaks

* More timeout adjustments

* More time out adjustments

* more windows tweaks

* Try running some things with -n 0

* Fix up windows test template and re-gen workflows

* Try this again

* Try all windows tests with -n 0

* misc update

* Undo some timeout changes

* Adjust timeout

* Increase timeout

* Coverage combine should combine the files in the current directory

* Trying retry test on windows

* timeout adjustment

* use tmp_path

* Few minor updates based on review

* lint fix

* Update tests/build-workflows.py

* Update tests/core/server/test_dos.py

* Try windows with pytest xdist n=2

* Check windows with xdist n=3

* Trying n=2 again

* Adjust timeout

* Update testconfig.py

* Update build-workflows.py

* Update testconfig.py

* Update .pre-commit-config.yaml

* Update testconfig.py

* Update build-workflows.py

* Update build-test-macos

* Update build-test-ubuntu

* Update build-test-windows

* skip

* Apply suggestions from code review

* tidy

* Update test_pool_config.py

Co-authored-by: Earle Lowe <e.lowe@chia.net>
Co-authored-by: Earle Lowe <30607889+emlowe@users.noreply.github.com>
2022-05-31 14:04:45 -05:00
Mariano SorgenteandGitHub 8dc1d66cd2 Increase from 180 to 300 (#11707) 2022-05-31 13:40:08 -05:00
Kyle AltendorfandGitHub 12ec1a2b3e add tests.util.misc.assert_maximum_duration for benchmark assertions (#11589)
* add tests.util.misc.assert_maximum_duration for benchmark assertions

* fix some imports

* more

* lint

* correct duration

* hint fixup

* rename to caller_file_and_line()

* set default timer to thread_time()

* extract gc manager and default to disabling

* self calibrate overhead

* wrap none-ness behind .results() method

* add messages

* from __future__ import annotations

* yield out self

* final from typing_extensions

* split it

* just use Future

* tweak

* tweak

* correction in loosely hint checked file

* all future indexing in hints

* union for the hints

* rename

* assert_runtime()

* another rename...

* message -> label

* label all results with test names

* oops
2022-05-23 12:50:48 -07:00
Arvid NorbergandGitHub 3d8081cc0d restore missing hints being stored as None (instead of 0-length bytes) (#11568) 2022-05-23 10:42:40 -07:00
b644f37277 Fix a few additional flaky tests (#11597)
* Fix a few additional flaky tests

* Lint

* Update tests/core/full_node/stores/test_full_node_store.py

Co-authored-by: Jeff <jeff@chia.net>

* Fix merge conflict

* Remove unnecessary function

Co-authored-by: Jeff <jeff@chia.net>
2022-05-23 07:55:05 -07:00
Mariano SorgenteandGitHub 766e1c21f5 Fix several flaky tests (#11576)
* Increase some of the timeouts to reduce flakiness

* Experiment with CI for flakiness

* Force more runs

* Add argument name

* Fix flaky mempool test

* Attempt to fix another flaky test

* Fix wallet retry test

* Missing arg

* Lint

* Increase benchmark test time

* Debug

* Debug

* Simplify retry test

* Lint

* More lint

* Return false instead of assert

* No need to check disconnect twice (since we don't break, this was flaky)

* Remove useless changes

* Accidental changes revert

* Accidental changes revert 2

* Accidental changes revert 3

* Revert fixture
2022-05-23 07:54:38 -07:00
Arvid NorbergandGitHub 51fc5f6e88 remove the cache from CoinStore. It appears the cost of maintaining the cache outweighs the gains. (#11540) 2022-05-16 13:28:42 -07:00
Arvid NorbergandGitHub 3795a52152 fix some LGTM issues (#11453) 2022-05-12 15:17:24 -07:00
Mariano SorgenteandGitHub e0df18f85d Ms.hint refactor (#11222)
* Some work on hints

* More cleanup and test_generator_tools

* Change relevant calls

* More refactor and cleanup

* More refactor

* Small fix

* Create wrapper object PeakPostProcessingResult

* Lint and small fix

* Fix a hint bug

* Fix hint update_wallets

* Fix test_full_sync

* pre-commit

* Start with PR review comments

* More efficient iteration

* Remove tx_removals_additions_and_hints

* Revert mozilla ca

* Fix issue with startup
2022-05-05 08:19:57 -07:00
Arvid NorbergandGitHub 3b4cbd42ce lookup coins in large batches (#11328)
* lookup coins in large batches, rather than one at a time, in block_body_validation

* make get_coin_records use the cache. and make the blockchain change tracking also use the new batch lookup

* make building the sql query a bit more elegant

* fix assert

* fix test_blockchain assert tests

* add type annotation

* await aiosqlite.Cursor.execute() immediately
2022-04-28 19:34:43 -07:00
ca4881ff3e Fix incorrect return in "rollback_to_block" (#10954)
* Add comments and fix a coin subscription bug

* Updated rollback test

* simplify test

* expand test

Co-authored-by: Mariano Sorgente <sorgente711@gmail.com>
2022-04-20 11:12:00 -07:00
Arvid NorbergandGitHub 743758c788 apply stricter mempool conditions (no garbage at the end of condition arguments). Fixup test (#10737) 2022-04-20 11:10:30 -07:00
2f9e718073 Ms.fast test blockchain (#11051)
* more work on test blockchain

* Optimize test_blockchain.py

* Fix weight proof bug

* Rename variable

* first rc_sub_slot hash bug

* New plots

* try with a new ID

* Run without cache

* Address test blocks and plots preparation.

* Update constant in test_compact_protocol().

* Update this constant too.

* Revert accidental altering of the gui submodule in ae7e3295f280a591e76c4dffdea75fb74ea5de6f.

* Fix benchmark test

* Revert mozilla-ca change

* Rebase on main

Co-authored-by: almog <almogdepaz@gmail.com>
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>
2022-04-08 09:57:59 -07:00
Amine KhaldiandGitHub a2fa8dda01 Prepare test blocks and plots only for tests that need them. This saves us a couple more hours of CI running time. (#10975) 2022-04-05 10:34:57 -07:00
Mariano SorgenteandGitHub 35fb7d341c Faster full node tests (#10986)
* Start fast full node tests

* Perf improvement on send_transaction

* Major performance improvement for mempool test

* Speed up another test

* Speed up mempool tests startup

* Lint

* Debug tests

* Try function scope for wallet_nodes

* Update comment
2022-04-04 13:49:33 -07:00
Arvid NorbergandGitHub fe77c69018 run_generator2 rust call and compact conditions data structure (#8862)
* use run_generator2 rust call and compact spend bundle conditions data structure pervasively.

* address review comments
2022-04-04 11:53:13 -07:00
Mariano SorgenteandGitHub 890c7d3754 Fix remaining linting issues (#10962)
* FIx remaining linting issues

* Revert type:ignore

* Revert token_bytes change
2022-04-02 13:22:55 -07:00
Arvid NorbergandGitHub 647cf6b52b fix performance tests (#10983) 2022-04-01 10:12:10 -04:00
Arvid NorbergandGitHub 8833cc351c reorg fixes (#10943)
* when going through a reorg, maintain all chain state until the very end, when the new fork has been fully validated and added

* when rolling back the chain, also rollback the height-to-hash map

* add tests
2022-03-31 08:26:23 -07:00
Mariano SorgenteandGitHub 908f186c1e Also throw DB error on double spending a coin (#10947)
* Throw error on double spending a coin

* Throw error on double spending a coin

* Improve test
2022-03-31 08:25:57 -07:00
Arvid NorbergandGitHub bf15087c79 run more tests in parallel on CI (#10960)
* run more tests in parallel on CI

* fix test_farmer_get_harvesters to wait for plots to be loaded before asking about them
2022-03-31 08:23:59 -07:00
Arvid NorbergandGitHub 40f2d42c73 bump timing threashold for mempool performance test (#10953) 2022-03-30 10:01:23 -07:00
Adam KellyandGitHub c5426068aa Consolidate test fixtures (#10778)
* Rename confusing fixtures, especially ones with the same name but different implementation

* revert premature fixture rename: two_wallet_nodes_start_height_1

* Consolidate test fixtures
2022-03-29 13:25:30 -07:00
Arvid NorbergandGitHub a100dda37e new DBWrapper supporting concurrent readers (#10166)
* new DBWrapper supporting concurrent readers

* adress review comments

* fixup default database version, when file doesn't exist

* remove unused argument
2022-03-28 11:58:00 -07:00
7ea074c9e7 fixup and enable condition checking tests (#10888)
* fixup and enable tests for the edge cases of absolute timestamp and absolute height conditions in mempool_manager

* Update chia/full_node/full_node_api.py

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
2022-03-25 18:49:11 -07:00
Kyle AltendorfandGitHub 2eb4fdeaee Disable the pytest-monitor plugin in CI if not checking results (#10837)
* disable the pytest-monitor plugin if not reporting results

pytest-monitor uses multiprocessing and has caused multiple confusing
issues.  Perhaps it can be adjusted to not use multiprocessing, but
for now lets just isolate the oddities to where we actually use it.

* use a template for resource usage check, similar to timelord install

* hint testconfig.custom_vars
2022-03-24 09:30:42 -07:00
Arvid NorbergandGitHub b5d51c38f8 run tests in parallel in CI (#10499) 2022-03-22 15:04:07 -07:00
Adam KellyandGitHub b8e1f3b9a9 Rename confusing fixtures, especially ones with the same name but dif… (#10772)
* Rename confusing fixtures, especially ones with the same name but different implementation

* rename test_environment to test_plot_environment

* Make it so setup_two_nodes is no longer the name of a fixture and a utility function

* revert premature fixture rename: two_wallet_nodes_start_height_1
2022-03-22 11:24:55 -07:00
Arvid NorbergandGitHub 0e956c8b5a run benchmarks separately (#10754)
* run benchmarks separately

* only run benchmarks once, with the most recent python version we support
2022-03-17 08:36:51 -07:00
Adam KellyandGitHub 7bbfa30a24 Ak.setup nodes (#10619)
* Remove unused test code

* Centralize fixture uses of setup_n_nodes

* Centralize fixure uses of setup_two_nodes

* Break up setup_nodes into setup_services, for individial services, and setup_nodes, for initializing different simulator configurations

* Sort imports
2022-03-16 11:47:54 -07:00
JeffandGitHub fc453e637c Fixed test failures on Windows. (#10740) 2022-03-15 09:59:27 -07:00
Kyle AltendorfandGitHub c63324abe9 remove some event_loop() fixtures (#10420)
* remove event_loop() fixtures

* flake8

* flake8

* remove sys.exit() from daemon shutdown

* bump full node test timeout.  a lot...  to see.

* fixup some tests

* back to module scope event loop fixture for test_full_node.py

* Update test_full_node.py

* Iterator...

* for the whole directory

* some fixtures back to module scope for reduced runtime

* back to 40 minute workflow timeouts

* these are being addressed separately
2022-03-14 06:08:35 -07:00
Adam KellyandGitHub 70639be906 Ak.convert fixtures (#10612)
* Use bt fixture

* rebase

* Use local_hostname

* flake8

* Remove set_shared_instance

* Remove unneeded comments

* Revert unrelated change

* Add back type signature - rebase issue

* Correct type for _configure_legacy_backend

* See what's going on during CI mypy run

* github workflows

* mypy typing

* Remove legacy Keyring create method

* Start daemon first

* Set chia-blockchain-gui to hash in main

* Fix two test files that were not importing self_hostname

* self_hostname fixture

* Convert all class fixtures to top level functions
2022-03-11 16:26:54 -08:00
Adam KellyandGitHub ff324095cc Remove block tools and keychain globals (#10358)
* Use bt fixture

* rebase

* Use local_hostname

* Fix test_json (inheritance from unittest.TestCase)

* Use correct BlockTools fixture for test_simulation

* Pass bt fixture into cost calculation tests

* flake8

* Add missing parameters to test functions

* Fix from rebase issues

* Remove set_shared_instance

* Update comment

* Remove unneeded comments

* Remove unused code

* Remove unused code, run `multiprocessing.set_start_method("spawn")` at correct time.

* Revert unrelated change

* Set daemon_port. Teardown services in correct order.
    BIG thanks to Mariano Sorgente for debugging help.

* Add back type signature - rebase issue

* Apply review fixes from Jeff

* Document why we need a later pytest-asyncio version

* Correct type for _configure_legacy_backend

* See what's going on during CI mypy run

* github workflows

* mypy typing

* Remove legacy Keyring create method

* Start daemon first

* Shutdown daemon coroutine properly

* Remove un-needed daemon_port argument

* Set chia-blockchain-gui to hash in main

* Remove connect_to_daemon_port

* Remove code that set "daemon_port" before calling `setup_daemon`

* Remove self_hostname fixture and extra self_hostname global

* Fix two test files that were not importing self_hostname

* self_hostname fixture

* Remove more unused test code

* Simplify  fixture
2022-03-10 11:06:49 -08:00
Arvid NorbergandGitHub 0e29dbc6d4 Bump CAT wallet test timeout to 40 minutes (#10605) 2022-03-09 03:41:18 -08:00
Kyle AltendorfandGitHub 6ff6fbe7de Fixup some hidden test errors (#10442) 2022-03-08 08:49:20 -08:00
Arvid NorbergandGitHub b76bf7b8a3 fix updating of sub-epoch-summary map (part of BlockHeightMap) when the entirey change, including genesis changes (#10486) 2022-03-02 12:50:06 -08:00
Arvid NorbergandGitHub d80293e390 more test fixes (#10395)
* WIP: fix more hard coded listen sockets in tests

* correct comment

* remove seemingly redundant event_loop fixtures (that sometimes seem to cause problems)

* when a test fails, the error is the most important thing to print

* fix mempool tests to me truly independent, by giving each test case a fresh block, and fresh reward coins to use for the test
2022-02-23 07:51:18 -08:00
Arvid NorbergandGitHub 97ef7b275a use free listen ports when running tests (#10307) 2022-02-22 16:22:11 -08:00
Arvid NorbergandGitHub 48a1c26232 make mempool tests independent, i.e. possible to run individually (#10317) 2022-02-20 21:30:07 -08:00
Kyle AltendorfandGitHub a17e5fc100 Split full node tests directory for parallelism (#10108)
* Split full node tests directory for parallelism

* reduce full node job timeout to 40, duplicate config to new directory

* rebuild workflows

* add missing __init__.py

* update some imports

* oops

* isort

* Revert "isort"

This reverts commit 03d8360342.

* update isort exclude list
2022-02-18 07:51:52 -08:00
Arvid NorbergandGitHub 34159d3529 optimize get_block_generator() (#10104)
* optimize get_block_generator()

* add a v1 compatible get_generator() to speed up get_block_generator() with v1 databases. Add test. Add error log in case generator_from_block() fails.

* speed up test_full_block_utils
2022-02-17 07:32:29 -08:00
Arvid NorbergandGitHub ce533879e2 run pytest in strict asyncio mode. add missing @pytest.mark.asyncio (#10230) 2022-02-16 11:01:04 -05:00
Kyle AltendorfandGitHub 884ebd902b remove chia.util.clvm pass-through file (#10130)
* remove chia.util.clvm pass-through file

* in tests too
2022-02-14 12:52:14 -08:00
Mariano SorgenteandGitHub 7fa1861def Ms.wallet refactor (#10146)
* wallet fixes

* Don't show false positive synched

* Code cleanup and lint

* Fixes

* Revert issue

* Fix reorg issue

* WIP wallet node

* More wallet refactor

* More wallet changes

* More

* Fix subscription bugs

* Fix some tests

* Fix pool tests

* More tweaks

* Lint and small issues

* call update_ui at the correct points

* Small changes

* new peak queue

* Fix peer height issue

* Rollback more for safety, and tweak logging

* Small WSM style fixes

* Change fork point in long_sync

* More fixes with real world testing

* Fix reversed filter

* Fix function name

* Fix coin store bug properly

* Raise CancelledError

* Fix rollback issue

* Lint

* Small fix

* Fix CAT issue

* Fix test trades race condition

* Fix test trades race condition

* Try to reduce flakiness

* Test coin store and fix additional method

* Improve flakiness
2022-02-14 11:28:36 -08:00
wjblankeandGitHub 419f88f296 updated soft fork to 2300000 (#10170) 2022-02-09 18:02:06 -08:00