Skip to content

External Access to Apache Kafka on OpenShift/OKD from the .NET client. Is it possible? #1485

@Serg675

Description

@Serg675

Description

Hello,
I need to produce to Kafka on OpenShift/OKD cluster deployed using https://strimzi.io.
The specific of this setup is that Kafka brokers are not accessible from outside of the cluster. Kafka can be accessed via route service only which is TLS encrypted.
I try to access Kafka using 1.5.3 version of the client on Windows 10 and Windows Server 2019.
I use information from this source https://blog.mimacom.com/strimzi-okd/ from "External Access of Kafka" chapter.
I have extracted ca.crt as described in mentioned article. The problem is that this certificate has public key only. No private key.
Java client and Kafka consumer and producer console scripts work fine with .JKS trusted store created with this exported certificate (as depicted in the article). But Kafka .NET client does not.
I use following setting:

<add key="bootstrap.servers" value="kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxx.com:443"/>
<add key="ssl.endpoint.identification.algorithm" value="none" />
<add key="security.protocol" value="SSL" />
<add key="ssl.certificate.location" value="ca.crt" />
<add key="ssl.key.password" value="password" />

I read these settings from the app.config into Dictionary<string,string> and feed it to ProducerBuilder. Works fine for all other setting (not SSL related). So I'm sure that settings are adopted by ProducerBuilder.

I get SSL handshake failed error.

%3|1608580696.853|FAIL|rdkafka#producer-2| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxx]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxx.com:443/bootstrap: SSL handshake failed: Disconnected: connecting to a PLAINTEXT broker listener? (after 479ms in state CONNECT)
%3|1608580696.853|ERROR|rdkafka#producer-2| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxx]: 1/1 brokers are down

Could you please answer below questions?

  1. Is private key mandatory to configure SSL?
  2. If not then what is right configuration to use public key only?

Thank you in advance.

Adding console output with debug=protocol,security,broker

%7|1608583003.853|OPENSSL|rdkafka#producer-1| [thrd:app]: librdkafka built with OpenSSL version 0x1000211f
%7|1608583004.287|CERTROOT|rdkafka#producer-1| [thrd:app]: 74/74 certificate(s) successfully added from Windows Certificate Root store
%7|1608583004.287|SSL|rdkafka#producer-1| [thrd:app]: Loading public key from file ca.crt
%7|1608583004.290|BRKMAIN|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Enter main broker thread
%7|1608583004.290|BROKER|rdkafka#producer-1| [thrd:app]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Added new broker with NodeId -1
%7|1608583004.290|CONNECT|rdkafka#producer-1| [thrd:app]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Selected for cluster connection: bootstrap servers added (broker has 0 connection attempt(s))
%7|1608583004.290|BRKMAIN|rdkafka#producer-1| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Enter main broker thread
%7|1608583004.290|INIT|rdkafka#producer-1| [thrd:app]: librdkafka v1.5.3 (0x10503ff) rdkafka#producer-1 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer, SSL ZLIB SNAPPY SASL_SCRAM PLUGINS HDRHISTOGRAM, debug 0x282)
%7|1608583004.290|CONNECT|rdkafka#producer-1| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Received CONNECT op
%7|1608583004.291|STATE|rdkafka#producer-1| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Broker changed state INIT -> TRY_CONNECT
%7|1608583004.291|CONNECT|rdkafka#producer-1| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: broker in state TRY_CONNECT connecting
%7|1608583004.291|STATE|rdkafka#producer-1| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
%7|1608583004.295|OPENSSL|rdkafka#producer-2| [thrd:app]: librdkafka built with OpenSSL version 0x1000211f
%7|1608583004.302|CERTROOT|rdkafka#producer-2| [thrd:app]: 74/74 certificate(s) successfully added from Windows Certificate Root store
%7|1608583004.304|SSL|rdkafka#producer-2| [thrd:app]: Loading public key from file ca.crt
%7|1608583004.306|BRKMAIN|rdkafka#producer-2| [thrd::0/internal]: :0/internal: Enter main broker thread
%7|1608583004.306|BROKER|rdkafka#producer-2| [thrd:app]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Added new broker with NodeId -1
%7|1608583004.307|CONNECT|rdkafka#producer-2| [thrd:app]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Selected for cluster connection: bootstrap servers added (broker has 0 connection attempt(s))
%7|1608583004.307|BRKMAIN|rdkafka#producer-2| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Enter main broker thread
%7|1608583004.307|INIT|rdkafka#producer-2| [thrd:app]: librdkafka v1.5.3 (0x10503ff) rdkafka#producer-2 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer, SSL ZLIB SNAPPY SASL_SCRAM PLUGINS HDRHISTOGRAM, debug 0x282)
%7|1608583004.307|CONNECT|rdkafka#producer-2| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Received CONNECT op
%7|1608583004.307|STATE|rdkafka#producer-2| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Broker changed state INIT -> TRY_CONNECT
%7|1608583004.308|CONNECT|rdkafka#producer-2| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: broker in state TRY_CONNECT connecting
%7|1608583004.308|STATE|rdkafka#producer-2| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
%7|1608583004.338|CONNECT|rdkafka#producer-2| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Connecting to ipv4#111.111.111.111:443 (ssl) with socket 1960
%7|1608583004.338|CONNECT|rdkafka#producer-1| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Connecting to ipv4#111.111.111.111:443 (ssl) with socket 1484
%7|1608583004.575|CONNECT|rdkafka#producer-1| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Connected to ipv4#111.111.111.111:443
%7|1608583004.586|CONNECT|rdkafka#producer-2| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Connected to ipv4#111.111.111.111:443
%7|1608583004.815|FAIL|rdkafka#producer-1| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: SSL handshake failed: Disconnected: connecting to a PLAINTEXT broker listener? (after 524ms in state CONNECT) (_SSL)
%3|1608583004.816|FAIL|rdkafka#producer-1| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: SSL handshake failed: Disconnected: connecting to a PLAINTEXT broker listener? (after 524ms in state CONNECT)
%7|1608583004.817|STATE|rdkafka#producer-1| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: Broker changed state CONNECT -> DOWN
%3|1608583004.817|ERROR|rdkafka#producer-1| [thrd:ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcimcluster.sapienc]: 1/1 brokers are down
%3|1608583004.817|ERROR|rdkafka#producer-1| [thrd:app]: rdkafka#producer-1: ssl://kafka-prod-kafka-bootstrap-dev.apps.okdcluster.xxxxxxxxxxxxxxx.com:443/bootstrap: SSL handshake failed: Disconnected: connecting to a PLAINTEXT broker listener? (after 524ms in state CONNECT)

Checklist

Please provide the following information:

  • A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.
  • Confluent.Kafka nuget version.
  • Apache Kafka version.
  • Client configuration.
  • Operating system.
  • Provide logs (with "debug" : "..." as necessary in configuration).
  • Provide broker log excerpts.
  • Critical issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions