Question: safety of multiple HTTPProviders

[menaitm]

• Version: 5.31.1
• Python: 3.9

Example

A single python process which can connect to multiple RPC endpoints may look like this:

chain_a = Web3(Web3.HTTPProvider(RPC_NODE_ETHEREUM))
chain_b = Web3(Web3.HTTPProvider(RPC_NODE_ARBITRUM))
chain_c = Web3(Web3.WebsocketProvider(RPC_NODE_OPTIMISM))

chain_a.eth.get_balance()
chain_b.eth.get_balance()
chain_c.eth.get_balance()

However, the docs state you should not create multiple HTTPProviders:

Note that you should create only one HTTPProvider per python process, as the HTTPProvider recycles underlying TCP/IP network connections, for better performance.

Questions

1. Is this still true? I see the session cache uses the endpoint_uri as the key, so does this comment actually mean that you should only create one HTTPProvider per RPC URI per python process?
2. Is there a workaround that doesn't require using multiple WebsocketProviders instead?
3. Does this apply to AsyncHTTPProvider too?
4. Does the request/response model of the WebsocketProvider mean you should not use asyncio.gather() to fire off multiple requests concurrently (to prevent race conditions when receiving messages in a different order back from the node)?

Thanks

[menaitm]

I'm not sure if this relates to the session caching that was recently changed here: #1847 (comment)

[kclowes]

1. Is this still true? I see the session cache uses the endpoint_uri as the key, so does this comment actually mean that you should only create one HTTPProvider per RPC URI per python process?

Currently it is, but only if you're using the same URL. The PR linked is a fix to allow multiple Web3 instances with the same URL too.

2. Is there a workaround that doesn't require using multiple WebsocketProviders instead?

Not that I know of, but there might be.

3. Does this apply to AsyncHTTPProvider too?

I believe so. The PR mentioned above should fix that though, and again, only if you're using the same provider URL.

4 Does the request/response model of the WebsocketProvider mean you should not use asyncio.gather() to fire off multiple requests concurrently (to prevent race conditions when receiving messages in a different order back from the node)?

Right. You should not use asyncio.gather with the websocket provider.

[kclowes]

After thinking further about allowing multiple web3 instances with the same provider type and URL, we decided to highlight and discourage that use case in the docs. The fix introduces a lot of complexity for what seems like a pretty small edge case.

@menaitm - if you (or anyone else following along) have a use case for this feature though that they'd like to advocate for, we'd be happy to reconsider.