Skip to content

_read_loop() inside nats/aio/client.py could become a never yield forever loop (nats-py-2.3.1.tar.gz, ssl) #490

@3figs

Description

@3figs

What version were you using?

nats-py-2.3.1.tar.gz

What environment was the server running in?

alpine linux - 3.18.2

Is this defect reproducible?

Yes, 1 out 4 of changing certs. after nats-server restarts, client can fall into forever loop.

Given the capability you are leveraging, describe your expectation?

Under no circumstance would _read_loop fall into a non-yield forever loop. It blocked everything.

Given the expectation, what is the defect you are observing?

Hi,
Our ssl nats client could fall into a forever loop when certs changed (nat-server restarted). Eventually we located the forever loop was inside _read_loop() inside aio/client/py:

while True:
   try:
        should_bail = self.is_closed or self.is_reconnecting
        if should_bail or self._transport is None:
               break
       if self.is_connected and self._transport.at_eof():
              err = errors.UnexpectedEOF()
              await self._error_cb(err)
              await self._process_op_err(err)
              break
       b = await self._transport.read(DEFAULT_BUFFER_SIZE) <-----
       await self._ps.parse(b)

b is always empty, eof is true. When this issue happened, all other asyncio tasks were blocked because this loop never yield.

We hit this issue when certs changes . After certs changed, nats-server restarts first. The nats-client using old certs should detect the failure and eventually reconnect using new certs. But nats-client falls into a forever loop and blocks other tasks from running. This issue can be seen around 1 out 4 tries.

Have anyone ever hit similar issue? It seems a bug to me. Under no circumstance would _read_loop fall into a non-yield forever loop.

Thanks
Steven

Metadata

Metadata

Assignees

No one assigned

    Labels

    defectSuspected defect such as a bug or regression

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions