'NoneType' object has no attribute 'add_reader' problem

[vichargrave]

I'm using the latest Pyshark to read pcap files and dump their contents to stdout. Sometimes I'm able to read all the packets from a given file with no problems. But every 3rd or 4th time I run my program with the same input file I get this error:

'NoneType' object has no attribute 'add_reader'

Using traceback I consistently get this output when this exception is raised:

Traceback (most recent call last):
File "/Users/vich/src/espcap/capture/espcap_file.py", line 62, in capture
dump_packets(capture, file_date_utc)
File "/Users/vich/src/espcap/capture/espcap_file.py", line 32, in dump_packets
for packet in capture:
File "/usr/local/anaconda/lib/python2.7/site-packages/pyshark/capture/capture.py", line 173, in _packets_from_tshark_sync
self._get_packet_from_stream(tshark_process.stdout, data, psml_structure=psml_structure))
File "/usr/local/anaconda/lib/python2.7/site-packages/trollius/base_events.py", line 350, in run_until_complete
return future.result()
File "/usr/local/anaconda/lib/python2.7/site-packages/trollius/futures.py", line 286, in result
raise self._exception
AttributeError: 'NoneType' object has no attribute 'add_reader'

I get this on MacOS and Linux, although more so on Linux. 'espcap.py' is the name of my application. At any rate what could be causing this?

[emunicio]

same problem here!

[KimiNewt]

Sorry, I'm in an exams period so have little time to check up on issues (will end shortly!).

Can you try running "my_cap_file.set_debug()" on your Capture object before reading packets and see if anything comes up?

[emunicio]

When setting debug, it seems the only additional given information is:

"** (process:23220): WARNING **: Preference "column.hidden" has been converted to "gui.column.hidden"
Save your preferences to make this change permanent.

** (process:23220): WARNING **: Preference "column.format" has been converted to "gui.column.format"
Save your preferences to make this change permanent "

Same problem here. I thought I could catch the error and just continue the parsing with the next package by changing _packets_from_tshark_sync but that didn't work either.

set_debug() didn't show anything additionally to what was seen before. Due to the classified data I can't post the dump ... :/

[ZeeZerEn]

Same problem :/

[heicci]

I get this too with python 2.7.8 on linux when I iterate through packet capture

[GinesGarcia]

i'm getting this error too when iterate over a .pcap file, any solutions??

thanks in advance.

[mrmiker11]

I found a solution to this, I too was receiving this error. It turns out it is an issue with the latest upgrade of trollius (v2.0). I first tried to downgrade pyshark with no luck, then I downgraded trollius and everything is working great. Here are the steps I took. I have not yet upgraded pyshark back to the newest version, but to get things working for sure I would suggest trying this out.

Sudo pip uninstall pyshark
Suod pip install pyshark==0.3.5
Sudo pip uninstall trollius
Sudo pip install trollius==1.0.4

[ZeeZerEn]

thank you @mrmiker11.

[KimiNewt]

Like @mrmiker11 said, I can only solve this by downgrading trollius. The new version of it is littered with bugs (including straight-up not working in Py3.5). I've specifically marked the old version in the setup now, and it will remain there until the bugs are fixed (or in 20xx where we drop Py2 support).

I apologize for late responses and not handling the many issues, I'm in school and busy with midterms and the like. I will find time to check out pull requests if there are any.

[git-blame]

I get the same error on FreeBSD. It seems that the older version of trollius doesn't help.

  File "/usr/local/lib/python2.7/site-packages/pyshark/capture/capture.py", line 173, in _packets_from_tshark_sync
    self._get_packet_from_stream(tshark_process.stdout, data, psml_structure=psml_structure))
  File "/usr/local/lib/python2.7/site-packages/trollius/base_events.py", line 300, in run_until_complete
    return future.result()
  File "/usr/local/lib/python2.7/site-packages/trollius/futures.py", line 287, in result
    raise self._exception
AttributeError: 'NoneType' object has no attribute 'add_reader'

pip list
pyshark (0.3.6.1)
trollius (1.0.4)

[vichargrave]

You need Pyshark 0.3.5.

…

-- vic

Sent from my iPhone

On Nov 24, 2016, at 10:00 AM, Chinh Nguyen ***@***.***> wrote: I get the same error on FreeBSD. It seems that the older version of trollius doesn't help. File "/usr/local/lib/python2.7/site-packages/pyshark/capture/capture.py", line 173, in _packets_from_tshark_sync self._get_packet_from_stream(tshark_process.stdout, data, psml_structure=psml_structure)) File "/usr/local/lib/python2.7/site-packages/trollius/base_events.py", line 300, in run_until_complete return future.result() File "/usr/local/lib/python2.7/site-packages/trollius/futures.py", line 287, in result raise self._exception AttributeError: 'NoneType' object has no attribute 'add_reader' pip list pyshark (0.3.6.1) trollius (1.0.4) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[git-blame]

Maybe Pyshark 0.3.5 will help, but it looks like there is a problem in trollius. I'm looking at the _UnixReadPipeTransport class and I see this:

def _read_ready(self):
    try:
        data = wrap_error(os.read, self._fileno, self.max_size)
    except (BlockingIOError, InterruptedError):
        pass
    except OSError as exc:
        self._fatal_error(exc, 'Fatal read error on pipe transport')
    else:
        if data:
            self._protocol.data_received(data)
        else:
            if self._loop.get_debug():
                logger.info("%r was closed by peer", self)
            self._closing = True
            self._loop.remove_reader(self._fileno)
            self._loop.call_soon(self._protocol.eof_received) 
            self._loop.call_soon(self._call_connection_lost, None) <========= Uh oh

def pause_reading(self):
    self._loop.remove_reader(self._fileno)

def resume_reading(self):
    self._loop.add_reader(self._fileno, self._read_ready)

Note that _call_connection_lost sets _loop to None.

In the StreamReader class there's this:

def _maybe_resume_transport(self):
    if self._paused and len(self._buffer) <= self._limit:
        self._paused = False
        self._transport.resume_reading()

So there is likely a race condition where sometimes resume_reading() is called after _call_connect_lost() has already been invoked leading to trying to call add_reader on a None loop.

The fix looks easy but I believe trollius is no longer being maintained. Here is a different resume_reading implementation in trollius for socket transports:

def resume_reading(self):
    if not self._paused:
        raise RuntimeError('Not paused')
    self._paused = False
    if self._closing:   <============== sanity check
        return
    self._loop.add_reader(self._sock_fd, self._read_ready)
    if self._loop.get_debug():
        logger.debug("%r resumes reading", self)

Pyshark fails on file capture because tshark exits closing the pipe. On live capture, we run forever or when packet_count is reached, pyshark kills tshark. So we never have the same condition.