* new blob block api method integrated into wallet
* direct msg streaming of headers, rename, tests
* perform_handshake call fix
* updated trusted sync with new block header calls
* add max blocks limit to fetch
* added tests for rejected block header msgs
* avoid parsing transactions info if not required
* avoid looking up capabilities setting
* move block tests out of a class
* test fix
* Merge changes
* added docs and increased rate limits
* increased block header request interval from 32 to 128
* remove fetching hashes and use height range
* fetching by height in db v2
* update capabilities, other fixes
* fixed range block header call
* Add type hints
* Start work on optimizing fetch_last_tx_from_peer
* Huge speedup in trusted wallet sync
* Revert unintentional changes
* Fix trade issue
* Improve the code
* Str format
* Optimize handling of farming rewards
* Fix bug
* Performance fixes
* Optimizations to wallet syncing
* Don't return all coins in respond_additions
* Revert concurrency numbers
* More optimization of the caches
* Small optimization in coin_added
* Optimize request_additions significantly by using a cache
* fixes from feedback
* capabilities check fixes
* Increase rate limits to allow 250tps in verification requests
* Start work on rate limits
* New rate limit versioning support
* Revert unrelated changes
* revert return False
* Lint
* Revert cbi
* try tests with trusted peer
* Revert unrelated wallet changes
* Revert more debug changes
* Add test and throw on an error if not found
* Reject invalid requests
* Revert bad change with uint32, and change warning to info
* Parametrize wallet sync test
* Merge and LGTM
* More clean way to choose peers
* Fix lint
* add the new RejectBlockHeaders, RequestBlockHeaders and RespondBlockHeaders to the network protocol regression test and regenerate test files
* Rate limit diffs only
* Improve performance
* Simpler
* Lint
Co-authored-by: Sebastjan <trepca@gmail.com>
Co-authored-by: arvidn <arvid@libtorrent.org>
* Hint and lightly refactor chia.server.upnp.UPnP
* Remove chia.server.upnp from mypy.ini lax rules
* black
* Revert the pointless int for shutdown
* isort
* from __future__ import annotations
* port 0 to fix flakiness
* Try fixing setup_full_system
* Try fixing setup_full_system, and lint
* More attempts to fix
* No more calls to get random ports in setup_nodes
* Revert accidental changes
* Timelord extra arg
* Try with port 0
* Fix daemon test, and lint
* Try without 0.0.0.0
* Back to 0.0.0.0
* Try a few timelord changes to get test running
* Increase timeout again
* Use the correct interface to get the port
* INFO logging to debug issue
* Revert "INFO logging to debug issue"
This reverts commit 7c379e5cca.
* Fix advertised port log
* Add extra log
* Logging back
* Rollback the timelord changes
* Try port 0 timelord
* Revert "Try port 0 timelord"
This reverts commit 4997faf3b2.
* Try full green, change ordering
* Remove unused var
* speed up simulation and cleanup
* Now try without the port config
* Fix a flaky call to get_event_loop
* Try getting the port dynamically
* No dynamic port
* Try changing the ordering
* Try adding a sleep
* Back to what works
* Timelord before vdf clients
* Dynamic port for 1st timelord
* Revert "Dynamic port for 1st timelord"
This reverts commit 0f322a15b7.
* Revert "Timelord before vdf clients"
This reverts commit 3286c34696.
* Revert "Back to what works"
This reverts commit 30380dffb7.
* Revert "Try adding a sleep"
This reverts commit 9212b665a6.
* Revert "Try changing the ordering"
This reverts commit a62597d70d.
* Revert "No dynamic port"
This reverts commit 5d2e15749b.
* Revert "Try getting the port dynamically"
This reverts commit ef9cd75679.
* Revert "Fix a flaky call to get_event_loop"
This reverts commit 01a000fdfb.
* Try one to 0
* Just not 0
* Don't get port dynamically
* Cleanup a bit
* Fix
* Some cleanup work
* Some cleanup work
* Fix daemon test
* Cleanup
* Remove arguments
* streamable: Merge `strictdataclass` into `Streamable` class
* tests: Test not supported streamable types
* streamable: Reorder decorators
* streamable: Simplify streamable decorator and force correct usage/syntax
* streamable: Just move some stuff around in the file
* streamable: Improve syntax error messages
* mypy: Drop `type_checking.py` and `test_type_checking.py` from exclusion
* streamable: Use cached fields instead of `__annotations__`
This is now possible after merging `__post_init__` into `Streamable`
* Introduce `DefinitionError` as `StreamableError`
* `/t` -> ` `
* protocols|server: Define new harvester plot refreshing protocol messages
* protocols: Bump `protocol_version` to `0.0.34`
* tests: Introduce `setup_farmer_multi_harvester`
Allows to run a test setup with 1 farmer and mutiple harvesters.
* plotting: Add an initial plot loading indication to `PlotManager`
* plotting|tests: Don't add removed duplicates to `total_result.removed`
`PlotRefreshResult.removed` should only contain plots that were loaded
properly before they were removed. It shouldn't contain e.g. removed
duplicates or invalid plots since those are synced in an extra sync step
and not as diff but as whole list every time.
* harvester: Reset `PlotManager` on shutdown
* plot_sync: Implement plot sync protocol
* farmer|harvester: Integrate and enable plot sync
* tests: Implement tests for the plot sync protocol
* farmer|tests: Drop obsolete harvester caching code
* setup: Add `chia.plot_sync` to packages
* plot_sync: Type hints in `DeltaType`
* plot_sync: Drop parameters in `super()` calls
* plot_sync: Introduce `send_response` helper in `Receiver._process`
* plot_sync: Add some parentheses
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* plot_sync: Additional hint for a `Receiver.process_path_list` parameter
* plot_sync: Force named parameters in `Receiver.process_path_list`
* test: Fix fixtures after rebase
* tests: Fix sorting after rebase
* tests: Return type hint for `plot_sync_setup`
* tests: Rename `WSChiaConnection` and move it in the outer scope
* tests|plot_sync: More type hints
* tests: Rework some delta tests
* tests: Drop a `range` and iterate over the list directly
* tests: Use the proper flags to overwrite
* test: More missing duplicates tests
* tests: Drop `ExpectedResult.reset`
* tests: Reduce some asserts
* tests: Add messages to some `assert False` statements
* tests: Introduce `ErrorSimulation` enum in `test_sync_simulated.py`
* tests: Use `secrects` instead of `Crypto.Random`
* Fixes after rebase
* Import from `typing_extensions` to support python 3.7
* Drop task name to support python 3.7
* Introduce `Sender.syncing`, `Sender.connected` and a log about the task
* Add `tests/plot_sync/config.py`
* Align the multi harvester fixture with what we do in other places
* Update the workflows
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Add mempool_max_total_cost to RPC
* Add signage_point event
* Fix incorrect crawler RPC port lookup
* Set up initial timelord RPC server + finished_pot_challenge event
* Add new compact proof event
* Add skipping/new_peak to track when fastest or not
* Check for None on change_data
* Add skipping_peak + new_peak to changes for metrics
* Convert chain to value
* Rename iters
* Timelord RPC to 8557 - 8556 is used in simulation tests
* Make tests work with RPC server on timelord
* Change event name to finished_pot
* Use broadcast_farmer object
* Move state changed for `finished_pot` after proofs_finished.append
* Fix type on ips var + add vdf_info and vdf_proof
* fix event name on the state_changed function
* Start fixing other issues
* Fork point, sync cancelling, random peer
* Reduce logging
* Improve performance and have a fallback peer for fetching
* Disconnect not synced peers
* Make sure try catch doesn't fail
* Fix lint
* Update chia/wallet/wallet_node.py
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Pylint has a bug so ignore pylint for this line
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* shutdown the weight proof process pool
* use a context manager for the weight proof process pool executor
* record of the debug code
* mostly cleaned up
* suppress sync task cancellation propagation when awaited while closing
* breakup multi-second WeightProofHandler.validate_weight_proof with async sleeps
* move awaiting of sync task until after existing cancellation
* properly handle shutdown file with a new instance each time and a context manager
* cleanup
* Add crawler RPC server
* Generate private keypair for crawler
* Bring over cleanup changes from the last closed PR
* Update the crawler RPC information to be its own subsection within seeder
* Add sleep before crawling to let the daemon connection get set up
* Wait for the actual callback to not be None, instead of just a random sleep interval
* Rework crawler/dns seeder to use the daemon + normal chia start process rather than the old system intended for the standalone repo
* Update configure testnet to work with seeder config
* Add back the crawler/seeder options from the standalone version
* Remove the check for none/sleep. Not needed when this is started by the daemon
* Add real data to the get_peer_counts endpoint
* Lint
* Fix calls to configure from init
* Turns out we still might sometimes move too quick before daemon/state changed callback is ready
* Add peer counts in the state_changed callback method
* Add a setting for peer_connect_timeout in the seeder: section so we can control it just for crawler
* start_seeder
* Pass config/root_path to the DNSServer so it can also use the configured crawler DB Path
* change in () instead of if/or
* Remove unnecessary return
* wallet changes from pac
* cat changes
* pool tests
* pooling tests passing
* offers
* lint
* mempool_mode
* black
* linting
* workflow files
* flake8
* more cleanup
* renamed
* remove obsolete test, don't cast announcement
* memos are not only bytes32
* trade renames
* fix rpcs, block_record
* wallet rpc, recompile settlement clvm
* key derivation
* clvm tests
* lgtm issues and wallet peers
* stash
* rename
* mypy linting
* flake8
* bad initializer
* flaky tests
* Make CAT wallets only create on verified hints (#9651)
* fix clvm tests
* return to log lvl warn
* check puzzle unhardened
* public key, not bytes. api caching change
* precommit changes
* remove unused import
* mypy ci file, tests
* ensure balance before creating a tx
* Remove CAT logic from full node test (#9741)
* Add confirmations and sleeps for wallet (#9742)
* use pool executor
* rever merge mistakes/cleanup
* Fix trade test flakiness (#9751)
* remove precommit
* older version of black
* lint only in super linter
* Make announcements in RPC be objects instead of bytes (#9752)
* Make announcements in RPC be objects instead of bytes
* Lint
* misc hint'ish cleanup (#9753)
* misc hint'ish cleanup
* unremove some ci bits
* Use main cached_bls.py
* Fix bad merge in main_pac (#9774)
* Fix bad merge at 71da0487b9
* Remove unused ignores
* more unused ignores
* Fix bad merge at 3b143e7050
* One more byte32.from_hexstr
* Remove obsolete test
* remove commented out
* remove duplicate payment object
* remove long sync
* remove unused test, noise
* memos type
* bytes32
* make it clear it's a single state at a time
* copy over asset ids from pacr
* file endl linter
* Update chia/server/ws_connection.py
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: Matt Hauff <quexington@gmail.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* farmer: Run a start background task to wait if there are no keys yet
* farmer: Run the handshake in a background task if there are no keys yet
* tests: Yield the service in farmer/harvester setups
* tests: Introduce `start_service` in farmer/harvester setups
* server: Make `ChiaServer` and `Service` restartable
* tests: Test the delayed start/handshake in `test_farmer_harvester.py`
* tests: Move `test_farmer_harvester.py` to `tests/core/farmer_harvester`
Attempt to avoid issues due to the global block tools / shared keychain
between tests
* farmer: Return `bool` in `Farmer.setup_keys`
* tests: Add `__init__.py` to `tests/farmer_harvester`
* Rework sized bytes for type hinting compatibility
* add a bunch of type: ignores
* this will be handled elsewhere
* noqa E501 instead of changing code
* normalize comment plurality
* @classmethod
* Revert "@classmethod"
This reverts commit 95db80e339.
* add ignore in benchmarks
* just E501 again...
* add some new type: ignores
* initial hack
* crawler
* add pytz
* Checkpoint.
* Catch some bugs.
* Localhost dig working.
* Checkpoint: return only high quality nodes.
* Statistics.
* Try improving finding reliable nodes.
* Bug.
* Move db to memory.
* Timestamp in the last 5 days.
* Increase crawl parameters, 180+ connections per sec.
* Bug.
* Optimize for DNS traffic.
* Prepare for hosting.
* Minimum height.
* Typo.
* Try catch everything.
* dnslib.
* Add db, format code.
* nits.
* No connections for the dns server.
* Rename src -> chia
* Fix some issues with v1.1
* Crawler task pool.
* Optimize closing connections.
* Split crawler and dns server.
* Install instructions.
* Catch startup bug.
* Try a big timeout for lock aquire.
* lint.
* Lint.
* Initial commit extended stats.
* Simplify code.
* Config.
* Correct stats.
* Be more restrictive in crawling.
* Attempt to fix stats bug.
* Add other peers port to config.
* Update README for the config.
* Simplify crawl task.
* Fix bug on restarts.
* Prevent log spamming.
* More spam prevention.
* Fix bug.
* Ipv6 (#1)
* Enable ipv6.
* Fix bug.
* Use numeric codes for QTYPE.
* ANY working.
* More spam prevention.
* Try to improve IPv6 selection.
* Log IPv6 available.
* Try to crawl more aggresive for v6.
* rename dns.py to crawler_dns.py so it doesn't conflict with imported package names
* Remove pytz package off dependencies
* Tidy-up ws_connection.py
* Fix spelling
* Reinstate chia-blockchain readme, with additional lines pertaining to the DNS introducer & crawler
* More detailed info in the README wrt Chia Seeder
* Nit
* More memetic naming of Chia Seeder
* Nit
* Add entry points
* Add entry in packages
* Patch some methods on the upstream server
* Update peer record fields
* Standard library imports first
* Crawler API check
* Reconcile crawl store
* Account for crawler_db_path in config
* Await crawl store load DB and load reliable peers
* Updates to crawler
* Rename to dns_server
* Crawler-specific overrides for the chia server
* Edit comment
* Undo changes to ChiaServer in view of crawler-specific overrides introduced in previous commit
* Nit
* Update service groups
* Expand name maps, mostly
* Fix the init config
* Remove unused import
* total_records unused at this stage
* Remove ios_reliable in peer_reliability table
* Remove row[20] entry
* Split overly long line
* Fix
* Type hint for ns_records
* Reconcile mismatch btw type int and uint64
* Type annotations in crawler
* Check whether crawl store is set
* Remove upnp_list
* Lint
* Chia Seeder CLI
* Lint
* Two white spaces
* 3rd party package import
* Cleaner way to handle overrides for ChiaServer method
* Address linter warnings
* Rename
* Nits
* Fix
* Change port #
* Most chia_seeder commands up and running
* Rename
* Progress of sorts
* Fix
* Improve legibility
* Fix naming
* Fix setup.py
* Lint
* None -> ''
* Remove whitespace
* Rename
* Log ipv6 better. (#9227)
* Log ipv6 better.
* Lint.
* -
* Undo GUI changes
* Another attempt
* GUI changes
Co-authored-by: Yostra <straya@chia.net>
Co-authored-by: Florin Chirica <fchirica96@gmail.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
* Address deficiency discussed in #8552, add ability to resolve to IPv6
addresses for hostnames.
* If there is no prefer_ipv6 in the config, set it True (per @hoffmang9)
and write it back to config.yaml
* Pass prefer_ipv6 flag to get_host_addr, which required a little digging
for it in a few places that call get_host_addr.
* Update a couple things for consistency
* Move the load_config into Wallet's __init__ so it doesn't get called so
many times as it would in has_full_node.
* Pass None into get_host_addr if there's no preference in config, so we have only that one place where the coded default lives. also fix an oversight where we were building a PeerInfo from a PeerInfo in some cases.
* Change the default here to match the default coded into util/network.py. It seems that github testers can't handle trying to use IPv6 and this may be easier for average users (sadly)
* A test to see if manually creating the server on :: (IP6_ANY) lets tests connect to localhost with IPv6 on
* Revert back to IPv4 default and remove the override inserted into
TCPSite for testing.
* Don't test for ip6-localhost, as it's not on all systems.
* Bah. Forced formatting of commented code...
* Add a type annotation for the addrset variable
* If we don't quote the socket enums, pylint gets upset because it has issues figuring out where/how they're defined. So, quote them here.
Co-authored-by: Chris Ross <cross+chia@distal.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Serialize/deserialize peer data alongside existing sqlite implementation (to be removed)
* Simplified AddressManagerStore. No longer uses sqlite and is no longer async.
* Removed aiosqlite usage from AddressManagerStore.
Added PeerStoreResolver class to determine the appropriate location for "peers.dat"
Updated initial-config.yaml to include "peers_file_path" default, replacing "peer_db_path" (similar change for "wallet_peers_path")
* Minor comment changes/additions
* Added migration from sqlite peer db.
Made AddressManagerStore's serialization async as it was right at the edge of blocking for too long.
* Minor tweaks to checking for migration
* Removed AddressManagerSQLiteStore class scaffolding
* makePeerDataSerialization now returns bytes instead of a PeerDataSerialization object
* Async file I/O for write_file_async using aiofiles
Added more tests
* Separate out the synchronous part of move_file
* Renamed write_file to files since we're opening up the capabilities a bit
* Update references to write_file
* Renamed test_write_file to test_files
* Tests covering move_file and move_file_async
* Minor refinements to behavior and tests
* Use aiofiles for reading peers.dat
* Added missing mypy typing info for aiofiles. Also added types-PyYAML to dev_dependencies so that `mypy chia tests` doesn't require running with --install-types
* Add types-aiofiles to the linting workflow
* Directory perms can now be passed into write_file_async.
Added an explicit f.flush() followed by os.fsync() after writing the temp file contents.
* Prority locking to consensus
* Remove pstats
* Linting
* Do some stuff outside of lock
* Fix startup
* Add log timings
* Try some different locking
* Add limit
* catch excp
* CLVM inside lock
* Try using a semaphore instead
* use events for lock queue
* test
* Add logging for message types
* type
* remove seed
* check new peak waiters
* correct FullNodeAPI self.full_node.new_peak._waiters typo
* correct logging string typos
* only warn about new_peak Waiters if there is at least 1
* remove no-longer-accepted parameter to FullNode.peak_post_processing()
* only warn about respond_transaction Waiters if there is at least 1
* lint
* Change some constants
* Small fix and logging changes
* Put message types outside
* Change some log levels so we can test with info
* More logging
* Increase rate limits but decrease paralelism
* tweaks
* Log dropped tx
* Fix pool rpc test
* Test fixes
* Mempool optimization
* Remove from seen if fails
* Increase queue sizes
* Message types info
* More test and logging
* Small changes to networking just in case
* Decrease logging
* Decrease logging even further
* Decrease logging even further even further
* Decrease logging 3
* Transaction queue
* Don't cancel tasks or close connection
* Cancel tasks on disconnect (for shutdown purposed)
* Fix typo
* Catch cancelled
* Do multiple at a time
* More accurate farmer response time
* More efficiently create tasks
* Increase queue size and priority by fee
* Revert priority
* Don't re-request too many times for dropped TX
* Handle cancelled error so we don't go into a bad state
* Catch cancelled in syncing tasks
* Reduce new_peak_sem to improve performance
* Less bytes conversion
* Missing file, and 2 workers for CLVM
* Validate BLS in a new thread
* tests
* Change semaphore constants
* correct a cancellation triggered exception and assertion
* Fix send_transaction, dont use BaseException, fix tests
* Fix more tests
* only log transaction handler cancellation in debug
* typing in log
* move unfinished validation to diff proc
* it is asyncio.CancelledError
* Add a test for bad signature
* Fix more tests, reduce logging, lint
* One more lint
* blockchain tests, pass bytes directly, single call
* Try to fix rl_wallet failures
* Fix mempool test
* catch everything
* Don't test RL wallet
* Fix more tests and return error code
* Improve error handling in multiprocess
* Add pre-validation time
* Add pre-validation time in logs, and revert pytest.ini changes
* Add log correctly
* Ms.bls cache experiment (#9115)
* Logging for cache
* Less logging
* Return to original plan
* Clean up
* Remove coment
* Remove log
* formalize LockQueue shutdown
* Comments
* Fix blockchain test
* Improve cache
* Remove logs
* Fix sign_coin_spends
* Fix pool wallet
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: Yostra <straya@chia.net>
* Before calling gethostbyname(), check to see if we already have an address.
That prevents an unnecessary call, and also allows for specifying IPv6
addresses which otherwise cause exceptions here.
* Also recognize addresses when passed to timelord spawn_process, so that
IPv6 addresses can be used.
* Missed importing PeerInfo
* Adjust style to match requirements and existing code
* Cast PeerInfo port to uint16
* add coin_store benchmark
* optimize _set_spent in CoinStore
* simplify _add_coin_record, since we won't use it to replace an entry anymore
* use executemany in add_coin_record and set_spent
* disable sqlite sync
* Check types of messages returned from other peers
* Check message type data structures at startup, check that no peeer messages that expect replies are broadcast, and dynamically check return types of messages that do expect replies.
* Type hint protocol message type check functions
* lint
* typo
* add replay_type decorator
* add api reply decorations
* typo
* Don't check non full-node broadcast messages
* Distinguish internal vs peer protocol error
Added Changelog
Moved static check to import time
Moved protocol timeouts to chia/protocols/protocol_timing.py
Comment typos
Rename create_request -> send_request
Comment that INVALID_PROTOCOL_MESSAGE is bannable, not temporary
* Call static check at module import time
* Rename message_response_ok
* Improve protocol checking for outgoing message validation
* Type-o
* reset submodule
Co-authored-by: almog <almogdepaz@gmail.com>
* fullnode related changes from wallet protocol and unharded derivation
* limit total subscriptions per peer
* reset counter on disconnect
* dict not a set
* check membership
* remove unused tests, lint
Kyle AltendorfandGitHub
dustinfaceandGitHub
Amine Khaldi
Earle Lowe
Jack NelsonandGitHub
Chris MarslenderandGitHub
Earle LoweandGitHub
JeffandGitHub
Arvid NorbergandGitHub
