IndexError on Connection Failure

[erewok]

Version:
redis-py: 4.3.3
Redis Version: 6.0.14

Platform:

• Kubernetes pod
• Docker image
• Python3.9

Description:

We are seeing IndexError when we reboot a redis server and our async Redis connection is closed and reconnect happens. The IndexError is located here in the codebase.

Traceback is like this:

  File "/usr/local/lib/python3.9/site-packages/sfdi_events_synd/clients/redis.py", line 81, in ping
    return await client.ping()
  File "/usr/local/lib/python3.9/site-packages/redis/asyncio/client.py", line 473, in execute_command
    conn = self.connection or await pool.get_connection(command_name, **options)
  File "/usr/local/lib/python3.9/site-packages/redis/asyncio/connection.py", line 1700, in get_connection
    await connection.connect()
  File "/usr/local/lib/python3.9/site-packages/redis/asyncio/connection.py", line 706, in connect
    raise ConnectionError(self._error_message(e))
  File "/usr/local/lib/python3.9/site-packages/redis/asyncio/connection.py", line 765, in _error_message
    f"Error {exception.args[0]} connecting to {self.host}:{self.port}. "
IndexError: tuple index out of range

It looks like our exception_args is empty?

        if len(exception.args) == 1:
            return f"Error connecting to {self.host}:{self.port}. {exception.args[0]}."
        else:
            return (
                f"Error {exception.args[0]} connecting to {self.host}:{self.port}. "
                f"{exception.args[0]}."
            )

We have been experimenting with ConnectionPooling and reconnect strategies, and this came up this morning on an experimental attempt to drop connections and reconnect in code that looks roughly like this:

@asynccontextmanager
async def redis_connection(pool):
    """
    Context manager for automatically closing connections on finish
    """
    connection = redis.Redis(
        connection_pool=pool, auto_close_connection_pool=False
    )
    yield connection
    await connection.close()


async def ping(pool):
    async with redis_connection(pool) as client:
        return await client.ping()

As noted above, we run this while rebooting our server to see if we can simulate failing connections and reconnect. We were expecting to see a ConnectionError instead of an IndexError.

[sileht]

In my case, the exception object was a https://docs.python.org/3/library/exceptions.html#ConnectionResetError

[sileht]

asyncio can raise the ConnectionResetError without creating an instance here:
https://github.com/python/cpython/blob/a31173c5ceb1708df687f942d714bdecae7cb759/Lib/asyncio/sslproto.py#L568

[erewok]

Wow. Nice find! That's super interesting.