Commit Graph
14659 Commits
Author SHA1 Message Date
dustinfaceandGitHub b89d958108 server: Drop reply_to_request and use send_message instead (#9327)
It's doing the same thing.
2021-11-24 11:28:24 -08:00
dustinfaceandGitHub a95dfba70b plotting|util|tests: Introduce chia.util.generator_tools.list_to_batches (#9304)
* plotting|util: Introduce `chia.util.generator_tools.list_to_batches`

* tests: Test `list_to_batches` in  `test_list_to_batches.py`

* util|tests: Return an empty iterator for empty input lists

* tests: Test list sizes from 1-10 in `test_valid`
2021-11-24 11:27:31 -08:00
Chris MarslenderandGitHub 280cc7f694 Fix installers (#9378)
* Update GUI back to main

* Pin specific versions of electron-package and electron-osx-sign

* Make sure ssh git urls actually use https

* Update azure pipeline to use https instead of ssh for git

* Update windows pipelines to use https instead of ssh

* Fix messed up insteadOf rules on windows + azure mac pipeline
2021-11-24 12:00:09 -06:00
William Blanke 7418142fb3 updated gui to d4ee0c701a4726c0aecea3d7857b698af154cc64 2021-11-23 21:15:28 -08:00
William Blanke 2d14b1ac1f updated gui to 0d79b544267509c41b7da9a00c1150f2886b72cd 2021-11-23 20:19:20 -08:00
Kyle Altendorf 46a5592943 tweak 2021-11-23 16:58:41 -05:00
Kyle Altendorf cdf6477857 rework block farming for data layer wallet 2021-11-23 16:30:25 -05:00
aacdc03188 Sql index changes (#9349)
* make the is_peak index smaller

* condition database updates on a configuration option

* drop unused coin_spent index

* drop redundant index on primary key

Co-authored-by: neurosis69 <83925572+neurosis69@users.noreply.github.com>
2021-11-23 13:23:07 -08:00
Kyle AltendorfandGitHub fbc6c738c8 Install .[dev] in CI tests (#9251)
* Install .[dev] in CI tests

* only install pytest-monitor on linux, as before

* add -d to install.sh for dev extras

* handle invalid options to install.sh

* rebuild workflows
2021-11-23 12:45:23 -08:00
1073f9d5dd mypy: warn_unused_ignores = true (and remove some) (#9085)
* mypy: warn_unused_ignores = true (and remove some)

* catch up

* type checking tweaks

* precommit uses mypy --no-warn-unused-ignores

* revert a bit

* precommit explanatory comment

* Update .pre-commit-config.yaml

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2021-11-23 12:44:53 -08:00
2263f9b399 improve coin_store benchmark (#9354)
Co-authored-by: xorinox <61642896+xorinox@users.noreply.github.com>
2021-11-23 09:53:59 -08:00
Arvid NorbergandGitHub 2fb72c1297 tell sqlite to use the puzzle_hash index when looking up coins by puzzle_hash (#9352) 2021-11-23 09:53:33 -08:00
Florin ChiricaandGitHub f4f58b6878 Test serialisation of network protocol messages. (#9347)
* Start with farmer protocol.

* Also check bytes.

* Add full node protocol.

* Remove from_bytes() in hardcoded data file.

* Add all protocol data.

* Ignore flake8.

* Run pre-commit.

* Lint.

* Add randomness to bytes32 and uints.

* Run pretest.
2021-11-23 09:52:42 -08:00
Kyle Altendorf 239243a0f2 more block progression help 2021-11-22 23:11:04 -05:00
Kyle Altendorf cb2b256724 update database layer clvm sha256tree 2021-11-22 22:25:48 -05:00
Kyle Altendorf 5f0e38c8a7 install pytest for pre-commit mypy 2021-11-22 22:09:39 -05:00
Kyle Altendorf 04969f1a71 Merge branch 'main' into atari 2021-11-22 22:04:41 -05:00
Kyle Altendorf c562115d55 Fixup optionals 2021-11-22 21:54:28 -05:00
Kyle Altendorf 57bf7eab55 fix block progression in tests 2021-11-22 21:06:53 -05:00
2c4aaa1ec9 Resolve v6 addresses (#8861)
* Address deficiency discussed in #8552, add ability to resolve to IPv6
addresses for hostnames.

* If there is no prefer_ipv6 in the config, set it True (per @hoffmang9)
and write it back to config.yaml

* Pass prefer_ipv6 flag to get_host_addr, which required a little digging
for it in a few places that call get_host_addr.

* Update a couple things for consistency

* Move the load_config into Wallet's __init__ so it doesn't get called so
many times as it would in has_full_node.

* Pass None into get_host_addr if there's no preference in config, so we have only that one place where the coded default lives.  also fix an oversight where we were building a PeerInfo from a PeerInfo in some cases.

* Change the default here to match the default coded into util/network.py.  It seems that github testers can't handle trying to use IPv6 and this may be easier for average users (sadly)

* A test to see if manually creating the server on :: (IP6_ANY) lets tests connect to localhost with IPv6 on

* Revert back to IPv4 default and remove the override inserted into
TCPSite for testing.

* Don't test for ip6-localhost, as it's not on all systems.

* Bah.  Forced formatting of commented code...

* Add a type annotation for the addrset variable

* If we don't quote the socket enums, pylint gets upset because it has issues figuring out where/how they're defined.  So, quote them here.

Co-authored-by: Chris Ross <cross+chia@distal.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
2021-11-22 14:34:03 -08:00
chuwtandGitHub e66e6505fa full_node: break when having 3 peaks (#9101) 2021-11-22 13:17:58 -08:00
Kyle AltendorfandGitHub 05f67167fe Correct some type hints adjacent to bytes32 (#8957)
* Correct some type hints adjacent to bytes32

Note that these errors won't show up until the sized bytes are
rewritten to "support" hinting.  They are too dynamic at the moment.

* drop some bytes32 hints with unneeded Optional

* Apply suggestions from code review

* remove code change, this pr is hints only
2021-11-22 13:16:52 -08:00
almog 7f148dab35 lint 2021-11-22 21:26:59 +02:00
Kyle Altendorf a6e58c9855 Merge branch 'main' into atari 2021-11-22 14:24:00 -05:00
Kyle Altendorf ed806e1d23 mypy 2021-11-22 12:27:07 -05:00
Kyle Altendorf 1237ba6b2a flake8 2021-11-22 12:26:05 -05:00
3e2dc4d52f add get_version to daemon wss endpoint (#8710)
* add get_version to damon wss endpoint

* add unit test for daemon::get_version

* use fixture for get_daemon

* fix flake8 warning in test

* Update server.py

Co-authored-by: wjblanke <wjb98672@gmail.com>
2021-11-19 22:43:44 -08:00
Earle LoweandGitHub 0d84578999 EL.Show NFT contract address on plot check (#8910)
* Show NFT contract address on plot check

* Update string for consistency with plotnft show
2021-11-19 11:27:49 -08:00
dustinfaceandGitHub 584000f0b3 rpc|tests: Introduce RPC endpoint /get_routes (#9084)
* rpc: Move `RPCServer._wrap_http_handler` to `chia.rpc.util`

* rpc: Introduce `RPCServer.get_routes`

* rpc|tests: Introduce RPC endpoint `/get_routes`
2021-11-19 11:20:56 -08:00
Kyle AltendorfandGitHub 420f9f066b Avoid exception on shutdown from RpcServer._state_changed() (#9108)
```
2021-11-03T19:09:09.603 full_node chia.rpc.rpc_server     : WARNING  Sending data failed. Exception Traceback (most recent call last):
  File "/farm/chia-blockchain/chia/rpc/rpc_server.py", line 69, in _state_changed
    await self.websocket.send_str(dict_to_json_str(payload))
AttributeError: 'NoneType' object has no attribute 'send_str'
.
```
2021-11-19 11:18:45 -08:00
Kyle AltendorfandGitHub 37e6a5f83c Initialize FullNode._transaction_queue_task and avoid AttributeError on shutdown (#9182)
```python-traceback
Traceback (most recent call last):
  File "/farm/chia-blockchain/venv/bin/chia_full_node", line 33, in <module>
    sys.exit(load_entry_point('chia-blockchain', 'console_scripts', 'chia_full_node')())
  File "/farm/chia-blockchain/chia/server/start_full_node.py", line 60, in main
    return run_service(**kwargs)
  File "/farm/chia-blockchain/chia/server/start_service.py", line 255, in run_service
    return asyncio.run(async_run_service(*args, **kwargs))
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
    self._run_once()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1823, in _run_once
    event_list = self._selector.select(timeout)
  File "/usr/lib/python3.8/selectors.py", line 468, in select
    fd_event_list = self._selector.poll(timeout, max_ev)
  File "/farm/chia-blockchain/chia/server/start_service.py", line 193, in _accept_signal
    self.stop()
  File "/farm/chia-blockchain/chia/server/start_service.py", line 210, in stop
    self._node._close()
  File "/farm/chia-blockchain/chia/full_node/full_node.py", line 704, in _close
    self._transaction_queue_task.cancel()
AttributeError: 'FullNode' object has no attribute '_transaction_queue_task'
```
2021-11-19 11:17:32 -08:00
Adam KellyandGitHub 1f0cd65f63 Show active network (#9194) 2021-11-19 11:16:35 -08:00
Kyle AltendorfandGitHub d45f8db533 Strip IPv6 [] correctly even with non-4-character segments (#9249)
* Strip IPv6 [] correctly even with non-4-character segments

For example, the existing code failed to properly strip
`[____:____:____:600:____:a2:____:____]` resulting in the trailing `]`
below.  (four character segments and node id redacted with `_`)

```
FULL_NODE ____:____:____:600:____:a2:____:____]   8444/8444  ________... Nov 12 20:46:47      1.5|18.9
                                                 -SB Height:  1131954    -Hash: 344cbef3...
```

* Simply IPv6 strip

Technically this hazards stripping multiple brackets from either end or a right bracket from the left, etc.  Perhaps this is ok.
2021-11-19 11:14:57 -08:00
Kyle AltendorfandGitHub 4b389c8fc0 Timed assertion output simplification (#9290)
* add values to time_out_assert_custom_interval() message

* hide time out assert helpers from stack traces

Get:

```
>       await time_out_assert(15, dl_wallet_0.get_unconfirmed_balance, 101)
E       AssertionError: Timed assertion timed
```

Instead of:
```
>       await time_out_assert(15, dl_wallet_0.get_unconfirmed_balance, 101)

tests/wallet/db_wallet/test_dl_wallet.py:114:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/time_out_assert.py:25: in time_out_assert
    await time_out_assert_custom_interval(timeout, 0.05, function, value, *args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

timeout = 15, interval = 0.05
function = <bound method DataLayerWallet.get_unconfirmed_balance of <chia.data_layer.data_layer_wallet.DataLayerWallet object at 0x7fbcbc35c7c0>>
value = 101, args = (), kwargs = {}, start = 1637077216.2218025, f_res = 0

    async def time_out_assert_custom_interval(timeout: int, interval, function, value=True, *args, **kwargs):
        start = time.time()
        while time.time() - start < timeout:
            if asyncio.iscoroutinefunction(function):
                f_res = await function(*args, **kwargs)
            else:
                f_res = function(*args, **kwargs)
            if value == f_res:
                return None
            await asyncio.sleep(interval)
>       assert False, "Timed assertion timed out"
E       AssertionError: Timed assertion timed out

tests/time_out_assert.py:21: AssertionError
```
2021-11-19 11:14:18 -08:00
dustinfaceandGitHub f8e8edc567 server: Refine return value of WSChiaConnection.send_message (#9305) 2021-11-19 11:13:50 -08:00
JeffandGitHub c4e14f5c78 Peer db new serialization (#9079)
* Serialize/deserialize peer data alongside existing sqlite implementation (to be removed)

* Simplified AddressManagerStore. No longer uses sqlite and is no longer async.

* Removed aiosqlite usage from AddressManagerStore.
Added PeerStoreResolver class to determine the appropriate location for "peers.dat"
Updated initial-config.yaml to include "peers_file_path" default, replacing "peer_db_path" (similar change for "wallet_peers_path")

* Minor comment changes/additions

* Added migration from sqlite peer db.
Made AddressManagerStore's serialization async as it was right at the edge of blocking for too long.

* Minor tweaks to checking for migration

* Removed AddressManagerSQLiteStore class scaffolding

* makePeerDataSerialization now returns bytes instead of a PeerDataSerialization object

* Async file I/O for write_file_async using aiofiles
Added more tests

* Separate out the synchronous part of move_file

* Renamed write_file to files since we're opening up the capabilities a bit

* Update references to write_file

* Renamed test_write_file to test_files

* Tests covering move_file and move_file_async

* Minor refinements to behavior and tests

* Use aiofiles for reading peers.dat

* Added missing mypy typing info for aiofiles. Also added types-PyYAML to dev_dependencies so that `mypy chia tests` doesn't require running with --install-types

* Add types-aiofiles to the linting workflow

* Directory perms can now be passed into write_file_async.
Added an explicit f.flush() followed by os.fsync() after writing the temp file contents.
2021-11-19 11:12:58 -08:00
Arvid NorbergandGitHub a29b1158ee fix lgtm issues (#9198) 2021-11-19 11:11:25 -08:00
Kyle Altendorf f7cdc301c4 mypy 2021-11-19 11:01:22 -05:00
Kyle AltendorfandGitHub b11a2e9255 Merge pull request #9288 from Chia-Network/atari-data_layer_mark 2021-11-19 09:57:36 -05:00
Kyle Altendorf f2c5ed3b4a lint 2021-11-19 09:37:11 -05:00
Kyle Altendorf 50daa1f236 black 2021-11-19 09:33:45 -05:00
matt 6ca556b902 fix bugs and test reclaim 2021-11-19 14:32:21 +00:00
Kyle AltendorfandGitHub 4a08f852dd Merge pull request #9312 from Chia-Network/atari-maybe_fix_db_locking 2021-11-18 15:12:23 -05:00
Kyle Altendorf 5fe0703158 speedup / more fix 2021-11-17 22:18:38 -05:00
Kyle Altendorf 6e1b03747f speedup / more fix 2021-11-17 21:45:56 -05:00
Kyle AltendorfandGitHub 1634acf8aa Allow .add_interested_puzzle_hash() to handle db transaction 2021-11-17 20:34:55 -05:00
Kyle Altendorf 8a000294c1 add db wallet workflows 2021-11-17 20:32:32 -05:00
matt 7e16168713 fix DLO test and add uncurry functions 2021-11-17 16:30:27 +00:00
dustinfaceandGitHub 7178d47d3a tests|github: Improve PlotManager tests + move them to test_plot_manager.py (#8875)
* tests: Improve `PlotManager` tests + move them to `test_plot_manager.py`

* github|tests: Enable `plotting` tests and generate the workflow files
2021-11-17 06:54:25 -08:00
Kyle Altendorf 24dc3bc5d7 add pytest data_layer mark 2021-11-16 10:03:26 -05:00