Implement exponential backoff logic for request retries for http providers.

- update docs
- decrease default backoff
- fix lint
- add newsfragment

Author: Haoran Wang

docs/internals.rst

@@ -146,8 +146,10 @@ Retry Requests for HTTP Providers
 
 ``HTTPProvider`` and ``AsyncHTTPProvider`` instances retry certain requests by default
 on exceptions. This can be configured via configuration options on the provider
-instance. Below is an example showing the default options for the retry configuration
-and how to override them.
+instance. The retry mechanism employs an exponential backoff strategy, starting from
+the initial value determined by the ``backoff_factor``, and doubling the delay with
+each attempt, up to the ``retries`` value. Below is an example showing the default
+options for the retry configuration and how to override them.
 
 .. code-block:: python
 
@@ -165,8 +167,8 @@ and how to override them.
             # number of retries to attempt
             retries=5,
 
-            # how long to wait between retries
-            backoff_factor=0.5,
+            # initial delay multiplier, doubles with each retry attempt
+            backoff_factor=0.125,
 
             # an in-house default list of retryable methods
             method_allowlist=REQUEST_RETRY_ALLOWLIST,

newsfragments/3358.breaking.rst

@@ -0,0 +1 @@
+Employ an exponential backoff strategy using the ``backoff_factor`` in ``ExceptionRetryConfiguration`` for ``HTTPProvider`` and ``AsyncHTTPProvider``. Reduce the default initial delay to ``0.125`` seconds.

web3/providers/rpc/async_rpc.py

@@ -123,7 +123,7 @@ async def _make_request(self, method: RPCEndpoint, request_data: bytes) -> bytes
                 except tuple(self.exception_retry_configuration.errors):
                     if i < self.exception_retry_configuration.retries - 1:
                         await asyncio.sleep(
-                            self.exception_retry_configuration.backoff_factor
+                            self.exception_retry_configuration.backoff_factor * 2**i
                         )
                         continue
                     else:

web3/providers/rpc/rpc.py

@@ -130,7 +130,9 @@ def _make_request(self, method: RPCEndpoint, request_data: bytes) -> bytes:
                     )
                 except tuple(self.exception_retry_configuration.errors) as e:
                     if i < self.exception_retry_configuration.retries - 1:
-                        time.sleep(self.exception_retry_configuration.backoff_factor)
+                        time.sleep(
+                            self.exception_retry_configuration.backoff_factor * 2**i
+                        )
                         continue
                     else:
                         raise e

web3/providers/rpc/utils.py

@@ -84,7 +84,7 @@ def __init__(
         self,
         errors: Sequence[Type[BaseException]] = None,
         retries: int = 5,
-        backoff_factor: float = 0.5,
+        backoff_factor: float = 0.125,
         method_allowlist: Sequence[str] = None,
     ):
         super().__init__(