Syncing on Windows really slow

[ImmaZoni]

Syncing on Windows takes about 6 Hours, on mac with the same internet connection this sync takes about 30 min. When starting on windows you get the following output. Which may indicate that only a single thread is being used for the syncing of the node, causing it to take significantly longer than needed. Issue has also been reported by a user aswell.

2022-03-29 22:43:19 Subspace
2022-03-29 22:43:19 ✌️  version 0.1.0-042302e-x86_64-windows-msvc
2022-03-29 22:43:19 ❤️  by Subspace Labs <https://subspace.network>, 2021-2022
2022-03-29 22:43:19 📋 Chain specification: Subspace testnet
2022-03-29 22:43:19 🏷  Node name: Void-1
2022-03-29 22:43:19 👤 Role: AUTHORITY
2022-03-29 22:43:19 💾 Database: RocksDb at C:\Users\Justin\AppData\Local\subspace-node-windows-x86_64-snapshot-2022-mar-09.exe\data\chains\subspace_test\db\full
2022-03-29 22:43:19 ⛓  Native runtime: subspace-100 (subspace-1.tx1.au1)
2022-03-29 22:43:50 Last archived block 265113
2022-03-29 22:43:50 Archiving already produced blocks 265114..=265202
2022-03-29 22:43:50 🏷  Local node identity is: 12D3KooWPJ3JoaHSG3fGBxHnkn5ASWqbUzFHAGdZk2kANp2t6bCA
2022-03-29 22:43:52 🧑‍🌾 Starting Subspace Authorship worker
2022-03-29 22:43:52 📦 Highest known block at #265302
2022-03-29 22:43:52 〽️ Prometheus exporter started at 127.0.0.1:9615
2022-03-29 22:43:52 Multi-threaded server is not available on Windows. Falling back to single thread.
2022-03-29 22:43:52 Listening for new connections on 0.0.0.0:9944.
2022-03-29 22:43:53 🔍 Discovered new external address for our node: /ip4/71.209.182.243/tcp/30333/p2p/12D3KooWPJ3JoaHSG3fGBxHnkn5ASWqbUzFHAGdZk2kANp2t6bCA
2022-03-29 22:43:57 ⚙️  Syncing, target=#336948 (24 peers), best: #265348 (0x12c1…3949), finalized #0 (0x332e…4a74), ⬇ 3.5MiB/s ⬆ 18.8kiB/s
2022-03-29 22:44:02 ⚙️  Syncing 12.9 bps, target=#336949 (25 peers), best: #265413 (0xcf82…8227), finalized #0 (0x332e…4a74), ⬇ 108.2kiB/s ⬆ 22.9kiB/s

Edit: Issue is also being tracked for users on the Forum here: https://community.subspace.network/t/multi-threaded-server-is-not-available-on-windows-falling-back-to-single-thread/46

[nazar-pc]

Yes, I think Multi-threaded server is not available on Windows. Falling back to single thread. is the root cause. Is CPU power significantly different between those machines?

[ImmaZoni]

Yes, I think Multi-threaded server is not available on Windows. Falling back to single thread. is the root cause. Is CPU power significantly different between those machines?

Macbook: 8 core (16 hyperthreaded) - 8gb ram
Windows: 4core (8 hyperthreaded) - 16gb ram

So they are a little different but not by too much.

[i1i1]

Should be fixed by this:
paritytech/jsonrpc#670

[i1i1]

The issue is still there unfortunately. Will dig into it more in the future.

[nazar-pc]

The reason seems to be runtime API calls. Each call takes ~5ms. In snapshot-2022-mar-09 most runtime calls were in verification stage and verification in sc-consensus-subspace took ~40ms for one block. Block import in sc-consensus-subspace took ~12ms.

Verification is theoretically parallelizable (though not sure if it actually is in Substrate right now), but block import is sequential. With 15ms per block we can do at most 40 blocks/s, I see ~19 blocks/s in practice, but there must be some extra overhead in Substrate itself.

Now need to figure out whether that overhead comes from WASM VM instantiation or state access or something else entirely. But I'm fairly confident this is a Substrate-level issue, not Subspace's fault as such.

[nazar-pc]

With native runtime verification in sc-consensus-subspace takes 0ms and block import in sc-consensus-subspace takes 1ms. Clearly WASM runtime overhead.

[nazar-pc]

This was resolved by updating documentation first and by fixing the code with Substrate upgrade afterwards, nothing left to do here.