Consumer error when connecting with SASL_PLAINTEXT

[aravinda92]

Description

Hi,
I'm using a dockerized kafka consumer to connect to KAFKA Broker. Can you check below log and let me know whats the problem ?

test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.195|SASLREFRESH|rdkafka#consumer-1| [thrd:app]: Kerberos ticket refreshed in 129ms
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.195|BROKER|rdkafka#consumer-1| [thrd:app]: GroupCoordinator: Added new broker with NodeId -1
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.196|BROKER|rdkafka#consumer-1| [thrd:app]: sasl_plaintext://10.48.148.44:9092/bootstrap: Added new broker with NodeId -1
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.196|BRKMAIN|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator: Enter main broker thread
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.196|BRKMAIN|rdkafka#consumer-1| [thrd::0/internal]: :0/internal: Enter main broker thread
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.196|BRKMAIN|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Enter main broker thread
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.196|INIT|rdkafka#consumer-1| [thrd:app]: librdkafka v1.4.0 (0x10400ff) rdkafka#consumer-1 initialized (builtin.features snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,sasl_oauthbearer, GCC GXX INSTALL GNULD LDS C11THREADS LIBDL PLUGINS SSL SASL_CYRUS HDRHISTOGRAM SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER CRC32C_HW, debug 0x202)
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.197|CONNECT|rdkafka#consumer-1| [thrd:main]: sasl_plaintext://10.48.148.44:9092/bootstrap: Selected for cluster connection: coordinator query (broker has 0 connection attempt(s))
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.200|CONNECT|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Received CONNECT op

test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.200|STATE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Broker changed state INIT -> TRY_CONNECT
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.200|CONNECT|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: broker in state TRY_CONNECT connecting
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.200|STATE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.201|CONNECT|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Connecting to ipv4#10.48.148.44:9092 (sasl_plaintext) with socket 10
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.212|CONNECT|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Connected to ipv4#10.48.148.44:9092
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.212|CONNECTED|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Connected (#1)
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.212|FEATURE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.212|STATE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Broker changed state CONNECT -> APIVERSION_QUERY
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.224|PROTOERR|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Protocol parse failure for ApiVersion v3 at 3/6 (rd_kafka_handle_ApiVersion:1911) (incorrect broker.version.fallback?)
test_kafka-consumer.1.ow47pujnt7r
u@testdev1 | %7|1605786768.224|PROTOERR|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: ApiArrayCnt -1 out of range
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.224|APIVERSION|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: ApiVersionRequest v3 failed due to UNSUPPORTED_VERSION: retrying with v0
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.231|FEATURE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.231|AUTH|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Auth in state APIVERSION_QUERY (handshake supported)
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.231|STATE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Broker changed state APIVERSION_QUERY -> AUTH_HANDSHAKE
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.232|SASLMECHS|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Broker supported SASL mechanisms: GSSAPI
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.232|AUTH|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Auth in state AUTH_HANDSHAKE (handshake supported)
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.232|STATE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Broker changed state AUTH_HANDSHAKE -> AUTH_LEGACY
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.232|SASL|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Initializing SASL client: service name kafka, hostname 10.48.148.44, mechanisms GSSAPI, provider Cyrus
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.234|SASL|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: My supported SASL mechanisms: DIGEST-MD5 EXTERNAL NTLM CRAM-MD5 LOGIN PLAIN ANONYMOUS
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %2|1605786768.234|LIBSASL|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: No worthy mechs found

How to reproduce

Dockerfile

FROM python:3.6-slim
ENV DEBIAN_FRONTEND=noninteractive
RUN  apt-get update && apt-get install -y apt-utils  && apt-get -yqq install  libpq-dev gcc ca-certificates git build-essential libssl-dev libsasl2-dev libsasl2-modules krb5-user

WORKDIR /test
RUN git clone https://github.com/edenhill/librdkafka.git
WORKDIR librdkafka
RUN  git checkout tags/v1.4.0 && ./configure && make && make install && ldconfig

WORKDIR /test
RUN git clone https://github.com/confluentinc/confluent-kafka-python.git
WORKDIR /test/confluent-kafka-python/
RUN  git checkout tags/v1.4.0
RUN python setup.py build
RUN python setup.py install

Checklist

Please provide the following information:

• confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): v1.4.0 (tried with v1.5.2). you can check the version in Dockerfile too
• Apache Kafka broker version:
• Client configuration:
  { 'bootstrap.servers': "broker_ip_port", 'group.id': "group_id", 'debug': 'broker,security', 'api.version.request': False, 'sasl.mechanism': 'GSSAPI', 'sasl.kerberos.service.name': 'kafka', 'security.protocol': "SASL_PLAINTEXT, 'sasl.kerberos.keytab': 'path/to/keytab', 'sasl.kerberos.principal': 'ketab_user@realm' }
• Operating system: Linux (Using docker image)
• Provide client logs (with 'debug': '..' as necessary)
• [] Provide broker log excerpts
• Critical issue

[edenhill]

[thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: My supported SASL mechanisms: DIGEST-MD5 EXTERNAL NTLM CRAM-MD5 LOGIN PLAIN ANONYMOUS
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %2|1605786768.234|LIBSASL|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: No worthy mechs found

The first log line is what cyrus/libsasl2 modules are installed on your client system, the second line is saying that it couldnt find a match for GSSAPI (your security.protocol) in that list.
You need to install the libsasl2-modules-gssapi-mit debian package.

[aravinda92]

Thank you. It worked