Future of cloud-based auto-providers

[pipermerriam]

What was wrong?

• Recently Infura changed it's API to require an API key.
• There are new cloud based JSON-RPC providers showing up.

The maintenance overhead of this API is not something I'd like the core development team to end up spending their time on, but there is value in having a way for users to quickly connect to a reliable cloud based provider.

How can it be fixed?

I see two main options.

1. Use something like setuptools entry_points to allow 3rd party providers to hook into the web3 namespace so that a user would install web3-infura and in doing so, be able to import the web3 providers from web3.auto.infura.*
2. Just relegate all of the cloud ones to 3rd party packages.

Downsides I see with the entry points based approach is that it makes the imports look like they come from the main web3 library when actually you're importing 3rd party code. This has the potential to mask bad actors, or worse, have malicious code insert itself masquerading as "infura" but actually doing something malicious. This is a major downside.

Because of this I'm inclined to go with option two and to deprecate our existing web3.auto.infura imports and move them into our own web3-infura package that we maintain. We could even go as far as providing a template for these repos to make it easier for people like nodesmith to manage their own packages.

Last, we could have a curated list of these packages listed in the docs to make them more discoverable.

[wolovim]

Found my way here after a PR was opened to add documentation for another remote node provider here: #1692.

I like the idea of deprecating native Infura-specific APIs in favor of a more even playing field. Would be lovely to handle all providers via generic env vars, e.g., WEB3_REMOTE_NODE_URL, ...SCHEMA, ...KEY, but may be unrealistic.

[pipermerriam]

I think it is reasonable to define a URI scheme for providers that would allow something like Web3.from_provider_uri(...) however I think this is still different from the auto style imports which are supposed to just work.

I'm still in favor of splitting off and releasing a web3-infura library that can serve as a template for other hosted providers who wish to provide similar ease-of-onboarding along with listing any other libraries like web3-quicknode if they choose to make it in the documentation.

[wolovim]

Closing since the decision has been made to move to 3rd party packages. Newer issue here: #1736.