Kafka 0.11.5 Will not reconnect-> 3/3 brokers are down error #584
Description
Description
I am using
Confluent.Kafka 0.11.5
Confluent.Kafka.Avro 0.11.5
Confluent.SchemaRegistry 0.11.5
Confluent.Apache.Avro 1.7.7.5
Client Config: - Took out names of bootstrap.Servers to be server1, server2, server3. We are adding group Id manually
<!-- consumer settings. use <add />, <remove />, <clear /> as usual -->
<consumer>
<add key='bootstrap.servers' value='server1, server2, server3' />
<add key='enable.auto.commit' value='true' />
<add key='auto.commit.interval.ms' value='1000' />
<add key='fetch.message.max.bytes' value='262144000' />
<add key='receive.message.max.bytes' value='300000000' />
<!-- if group.id is set here it will override whatever is specified in yaml or wherever that is -->
<!--<add key='group.id' value='' />-->
</consumer>
Operating System:
Windows Server 2012 R2
Periodically the Kafka brokers are restarted. When they are restarted our Kafka Consumer Service Running on a different box never is able to connect again. The error says 3/3 brokers are down.
I understand that there is a config to suppress these messages. However, I am NOT asking how to suppress the messages.
[Error] [TID:14] 8/6/2018 10:06:09 PM: akka://Cluster/user/genericcommander/consumerGeneric, OnError in Kafka generic consumer sender Confluent.Kafka.Consumer, Reason: 3/3 brokers are down
[Error] [TID:14] 8/6/2018 10:16:09 PM: akka://Cluster/user/genericcommander/consumerGeneric, OnError in Kafka generic consumer sender Confluent.Kafka.Consumer, Reason: 3/3 brokers are down
[Error] [TID:14] 8/6/2018 10:26:09 PM: akka://Cluster/user/genericcommander/consumerGeneric, OnError in Kafka generic consumer sender Confluent.Kafka.Consumer, Reason: 3/3 brokers are down
We tried restarting the windows service that it is running on and it still doesn't work. We can stop the windows Services and start them an hour later and it still doesn't work. We can ever copy the DLLs to another Server and run them and it connects, so we know for sure messages are coming thru.
The solution that finally worked is we restarted the Server and then Kafka reconnected and worked correctly.
We would like to NOT restart the Server everytime we restart the Kafka brokers.
It seems like it is caching the Consumer is down and just won't reconnect. Is this a DLL Version issue or is that someway I can change the code?
Any insight would be helpful