0ae3b2823f
* Begin add to support for third-party fee-based harvesters Signed-off-by: Harold Brenes <h.brenes@chia.net> * Return FoliageBlockData back to Farmer from FullNode * Send FoliageBlockData all the way to the harvester, if requested * Preparing tests for 3rd party harvester support * Continue writing tests for 3rd party harvester * Some cleanup. Drafting * Test full message chain of third-party Harvester reward overwrite. Added test that validates messages between Farmer, Harvester and FullNode to ensure that the correct data is used when an overwritten farmer reward address is given by a third-party Harvester service. * Changing Harvester signature request flow to include original data, if requested. This makes it so that NewSignagePoint unconditionally always includes the source of the SP hashes. * Fix incorrect field name * Harvester can ask for original signing data without overwriting the reward address also. * Update test_farmer_handles_farmer_reward_address_override to reflect changes. - Correct farmer API get the correct data source for the signage point signature request. - Trivial fixes * Removing some obsolete stuff * Trivial * Fix tests in updated harvester protocol - Fix including source data when include_source_signature_data is set. - Set source data fields optional. - Update test so that a timelord is started and we get proper signage points instead of aritficial ones. This helps us make sure we test for all the source data cases needed from the full node. * Refactor harvester source signing data test - Remove older harvester signing data test (had missing coverage for data) - Add harvester fee convention logging - Begin fee convention logging test - Refactor fixtures for harvester enhancement protocol tests Start harvester * Complete test for harvester fee convention * Update SignatureRequestSourceData message - SignatureRequestSourceData is now serialized as enum + bytes. - Update usage locations for SignatureRequestSourceData. - Update tests to reflect this * Missing test from last commit * Revert to explicit None checks, instead of relying on truthiness Revert scratch work * Linting * adjust for anyio * Moe linting * mypy fixes * Documentation comments * Move third-party harvester protocol tests Moved to own directory in order to enable `install_timelord = True` * mypy fixes * checkout_blocks_and_plots = True * Update some log messages, syntax and docs * Bump timeout in tests * Update protocol message types for harvester and farmer for test_missing_messages` * Regenerate protocol message data for protocol tests * Remove timelord requirement for tests. Inject pre-generated signage points and blocks instead. * Linting * More linting * More linting * Merge from main and lint again * Remove customized test plots requirements * Rebase fix * Coverage update - Update valid fee quality test to be unit, instead of pseudo-integration. - New test for invalid fee quality. - More explicit messaging for invalid fee qualities. - Explicitly comment out uncovered case in test. - Update types for mypy. * Lint * Lint * Lint, better logs * add protocol file tests * coverage * Update formatting in fee quality logging * Rebase fix * Regenerate protocol_message_bytes-v1.0 --------- Signed-off-by: Harold Brenes <h.brenes@chia.net> Co-authored-by: Harold Brenes <h.brenes@chia.net> Co-authored-by: Kyle Altendorf <sda@fstab.net> Co-authored-by: Harold Brenes <152330941+haorldbchi@users.noreply.github.com> Co-authored-by: arvidn <arvid@libtorrent.org>
Test CI Job generation
The CI jobs for these tests are managed by build-workflows.py.
If you add a test file, or move one to another directory, please run build-workflows.py.
Tests are recognized by the file glob test_*.py.
Changing the contents of a file does not require running build-workflows.py.
We currently use github actions. Default runners have two vcpus. The workflows are located in ../.github/workflows/.
The inputs to build-workflows.py are the templates in runner-templates, the file testconfig.py in this directory, and the optional config.py files in some test subdirectories.
Files in the template directory ending in include.yml are included in jobs based on the per-directory settings.
The generated workflows are output to ../.github/workflows/.
Each subdirectory below the directories root_test_dirs in testconfig.py becomes a job in the github workflow matrix.
If your jobs run too long, simply move some tests into new subdirectories and run build-workflows.py.
A workflow built from a parent directory does not include the tests in its subdirectories.
The subdirectory jobs do not include the tests from their parents.
testconfig.py
In the top tests directory, testconfig.py contains the application settings and the per-directory default settings.
config.py
Each directory has an optional config.py file, which can override the per-directory default settings.
Per directory settings defaults:
parallel = False
checkout_blocks_and_plots = True
install_timelord = True
job_timeout = 30
Parallel test execution
If you are certain that all the tests in a directory can run in parallel, set parallel = True in config.py inside that directory.
Optional job stages
Set checkout_blocks_and_plots to False to omit checking out the test-cache repo.
Set install_timelord to False to omit the step of installing a Time Lord for your directory's job.
Job Timeout
Set job_timeout to the number of minutes you want the CI system to wait before it kills your job.
Add two or three minutes to allow for job setup.