AttributeError: 'cimpl.Message' object has no attribute 'timestamp'

[drazenzubovic]

We encounter issues when running an integration_test.py on macOS. They could be related to the incompatibility with the c implementation of Kafka in librdkafka and the python wrapper.

System components:
OS: Sierra 10.12.1
Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 26 2016, 12:10:39)
librdkafka: stable 0.9.2 (bottled)
confluent-kafka (0.9.2)

Issue 1:
('==============================', 'Verifying Consumer', '==============================') Traceback (most recent call last): File "/Users/drazen/Documents/audi/workspace/confluent-kafka-python-master/examples/integration_test.py", line 465, in <module> verify_consumer() File "/Users/drazen/Documents/audi/workspace/confluent-kafka-python-master/examples/integration_test.py", line 259, in verify_consumer tstype, timestamp = msg.timestamp() AttributeError: 'cimpl.Message' object has no attribute 'timestamp'

in code snippet ...

tstype, timestamp = msg.timestamp()

Issue 2:
('==============================', 'Verifying stats_cb', '==============================') Traceback (most recent call last): File "/Users/drazen/Documents/audi/workspace/confluent-kafka-python-master/examples/integration_test.py", line 477, in <module> verify_stats_cb() File "/Users/drazen/Documents/audi/workspace/confluent-kafka-python-master/examples/integration_test.py", line 393, in verify_stats_cb c = confluent_kafka.Consumer(**conf) cimpl.KafkaException: KafkaError{code=_INVALID_ARG,val=-186,str="Property "stats_cb" must be set through dedicated .._set_..() function"}

in code snippet ...

conf = {'bootstrap.servers': bootstrap_servers,
'group.id': uuid.uuid1(),
'session.timeout.ms': 6000,
'error_cb': error_cb,
'stats_cb': stats_cb,
'statistics.interval.ms': 200,
'default.topic.config': {
'auto.offset.reset': 'earliest'
}}
c = confluent_kafka.Consumer(**conf)

Once the conflicting code is commented out, the rest of example will run through.

[edenhill]

It looks like it is picking up an older version of the Python client. Check your module and virtualenv paths.

[drazenzubovic]

The Python client should be ok afict.

I deliberately do not use virtualenv to avoid any library confusion.
There are only two Python installed. The Apple original (do not touch) Python 2.7.10 in /System/Frameworks ... and my Python 2.7.12 in /Library/Frameworks/...

$ python --version
Python 2.7.12

$ which python
/Library/Frameworks/Python.framework/Versions/2.7/bin/python

$ set |grep PATH
PATH=/Library/Frameworks/Python.framework/Versions/2.7/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin

Confluent kafka_python client is the latest ...

$ pip list |grep confluent
confluent-kafka (0.9.2)

... as well as the librdkafka from brew ...

$ brew info librdkafka
librdkafka: stable 0.9.2 (bottled), HEAD
The Apache Kafka C/C++ library
https://github.com/edenhill/librdkafka
/usr/local/Cellar/librdkafka/0.9.2 (14 files, 1.4M) *
Poured from bottle on 2016-11-22 at 16:41:18
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/librdkafka.rb
==> Dependencies
Build: pkg-config ✔
Required: lzlib ✔, openssl ✔

[edenhill]

What does this give you: python -c 'from confluent_kafka import version ; print(version())'

[drazenzubovic]

$ python -c 'from confluent_kafka import version ; print(version())'
('0.9.2', 590336)

[edenhill]

Oh, sorry, my bad, timestamp() was added after the 0.9.2 release and is only available on master branch.
(fyi: master still reports version 0.9.2)

[edenhill]

So this probably means that you are running examples/integration_test.py from master branch, but it uses the 0.9.2 release confluent_kafka module.
If this is the case, try (from the master checkout): pip install . (maybe prefix with sudo, depending on your setup)

[drazenzubovic]

Yes, your assumption is correct :) The code is indeed the examples/integration_test.py from the master branch. As it seems, it is a minor version/timing mismatch which will be fixed with the next release. And thanks for the prompt response!

I actually stumbled over it by trying to figure out if librdkafka and Python wrapper would have "Confluent Metric Interceptor" as java does. Python is namely somewhat simper to run the producer and consumer.

[edenhill]

Good, so it seems things are under control :)
Confluent Metric Interceptors are not yet available for librdkafka-based clients, but it is on the backlog.