Skip to content

Can not fetch data from kafka #326

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
2 tasks
joukosusi opened this issue Mar 6, 2018 · 3 comments
Closed
2 tasks

Can not fetch data from kafka #326

joukosusi opened this issue Mar 6, 2018 · 3 comments

Comments

@joukosusi
Copy link

joukosusi commented Mar 6, 2018 •
edited
Loading

Description

I can not fetch data from kafka and found several point about this

  1. logs below show that function committed and get_watermark_offsets worked and returned valid value:

%7|1520322063.946|OFFSET|rdkafka#consumer-1| [thrd:app]: ssl://192.168.4.65:8093/1: OffsetRequest (v0, opv 0) for 1 topic(s) and 1 partition(s)
%7|1520322063.946|OFFSET|rdkafka#consumer-1| [thrd:app]: ssl://192.168.4.65:8093/1: OffsetRequest (v0, opv 0) for 1 topic(s) and 1 partition(s)

%7|1520322064.689|OFFSETFETCH|rdkafka#consumer-1| [thrd:main]: ssl://192.168.4.65:8093/1: OffsetFetchResponse: TOPIC_TEST [0] offset 4269
%7|1520322064.689|OFFFETCH|rdkafka#consumer-1| [thrd:main]: ssl://192.168.4.65:8093/1: OffsetFetch for 1/1 partition(s) returned Success

  1. error "no offset stored" occured

%7|1520322068.399|COMMIT|rdkafka#consumer-1| [thrd:main]: OffsetCommit internal error: Local: No offset stored
%7|1520322068.399|COMMIT|rdkafka#consumer-1| [thrd:main]: OffsetCommit for -1 partition(s): cgrp auto commit timer: returned: Local: No offset stored

  1. i checked server logs and didn't find logs about 'group.test' like this:

[2018-03-06 15:15:01,505] INFO [GroupCoordinator 0]: Preparing to restabilize group group.test with old generation 12367 (kafka.coordinator.GroupCoordinator)
[2018-03-06 15:15:25,853] INFO [GroupCoordinator 0]: Stabilized group group.test generation 12368 (kafka.coordinator.GroupCoordinator)

  1. i can fetch data by using pykafka

how can i solve this problem???

log detail

%7|1520327542.149|MEMBERID|rdkafka#consumer-1| [thrd:app]: Group "group.test": updating member id "(not-set)" -> ""
%7|1520327542.150|BROKER|rdkafka#consumer-1| [thrd:app]: ssl://192.168.2.6:8092/bootstrap: Added new broker with NodeId -1
%7|1520327542.150|BROKER|rdkafka#consumer-1| [thrd:app]: ssl://192.168.2.6:8093/bootstrap: Added new broker with NodeId -1
%7|1520327542.150|BRKMAIN|rdkafka#consumer-1| [thrd::0/internal]: :0/internal: Enter main broker thread
%7|1520327542.150|STATE|rdkafka#consumer-1| [thrd::0/internal]: :0/internal: Broker changed state INIT -> UP
%7|1520327542.150|BROADCAST|rdkafka#consumer-1| [thrd::0/internal]: Broadcasting state change
%7|1520327542.150|BROKER|rdkafka#consumer-1| [thrd:app]: ssl://192.168.2.6:8094/bootstrap: Added new broker with NodeId -1
%7|1520327542.150|BRKREASSIGN|rdkafka#consumer-1| [thrd:main]: Group "group.test" management reassigned from broker (none) to :0/internal
%7|1520327542.150|CGRPSTATE|rdkafka#consumer-1| [thrd:main]: Group "group.test" changed state init -> wait-broker (v1, join-state init)
%7|1520327542.150|BROADCAST|rdkafka#consumer-1| [thrd:main]: Broadcasting state change
%7|1520327542.150|BRKASSIGN|rdkafka#consumer-1| [thrd:main]: Group "group.test" management assigned to broker :0/internal
%7|1520327542.150|BRKMAIN|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/bootstrap: Enter main broker thread
%7|1520327542.150|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "group.test" received op OFFSET_COMMIT (v0) in state wait-broker (join state init, v1 vs 0)
%7|1520327542.150|CONNECT|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/bootstrap: broker in state INIT connecting
%7|1520327542.150|COMMIT|rdkafka#consumer-1| [thrd:main]: Group "group.test": unable to OffsetCommit in state wait-broker: manual: coordinator (:0/internal) is unavailable: retrying later
%7|1520327542.150|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "group.test" received op SUBSCRIBE (v0) in state wait-broker (join state init, v1 vs 0)
%7|1520327542.150|SUBSCRIBE|rdkafka#consumer-1| [thrd:main]: Group "group.test": subscribe to new subscription of 1 topics (join state init)
%7|1520327542.150|UNSUBSCRIBE|rdkafka#consumer-1| [thrd:main]: Group "group.test": unsubscribe from current unset subscription of 0 topics (leave group=no, join state init, v1)
%7|1520327542.150|GRPLEADER|rdkafka#consumer-1| [thrd:main]: Group "group.test": resetting group leader info: unsubscribe
%7|1520327542.150|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "group.test" changed join state init -> wait-unassign (v1, state wait-broker)
%7|1520327542.150|UNASSIGN|rdkafka#consumer-1| [thrd:main]: Unassign not done yet (0 wait_unassign, 0 assigned, 1 wait commit): unassign (no previous assignment)
%7|1520327542.150|CONNECT|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/bootstrap: Connecting to ipv4#192.168.2.6:8093 (ssl) with socket 14
%7|1520327542.150|STATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/bootstrap: Broker changed state INIT -> CONNECT
%7|1520327542.150|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: Broadcasting state change
%7|1520327542.154|CONNECT|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/bootstrap: Connected to ipv4#192.168.2.6:8093
%7|1520327542.156|BRKMAIN|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/bootstrap: Enter main broker thread
%7|1520327542.156|CGRPQUERY|rdkafka#consumer-1| [thrd:main]: Group "group.test": no broker available for coordinator query: intervaled in state wait-broker
%7|1520327542.156|BRKMAIN|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/bootstrap: Enter main broker thread
%7|1520327542.157|CONNECT|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/bootstrap: broker in state INIT connecting
%7|1520327542.156|CONNECT|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/bootstrap: broker in state INIT connecting
%7|1520327542.157|CONNECT|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/bootstrap: Connecting to ipv4#192.168.2.6:8094 (ssl) with socket 16
%7|1520327542.157|CONNECT|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/bootstrap: Connecting to ipv4#192.168.2.6:8092 (ssl) with socket 15
%7|1520327542.157|STATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/bootstrap: Broker changed state INIT -> CONNECT
%7|1520327542.157|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: Broadcasting state change
%7|1520327542.157|STATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/bootstrap: Broker changed state INIT -> CONNECT
%7|1520327542.157|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: Broadcasting state change
%7|1520327542.161|CONNECT|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/bootstrap: Connected to ipv4#192.168.2.6:8094
%7|1520327542.165|CONNECT|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/bootstrap: Connected to ipv4#192.168.2.6:8092
%7|1520327542.194|CONNECTED|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/bootstrap: Connected (#1)
%7|1520327542.194|FEATURE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1520327542.194|STATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/bootstrap: Broker changed state CONNECT -> APIVERSION_QUERY
%7|1520327542.194|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: Broadcasting state change
%7|1520327542.198|FEATURE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/bootstrap: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime
%7|1520327542.199|STATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/bootstrap: Broker changed state APIVERSION_QUERY -> UP
%7|1520327542.199|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: Broadcasting state change
%7|1520327542.199|METADATA|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/bootstrap: Request metadata for brokers only: connected
%7|1520327542.201|CONNECTED|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/bootstrap: Connected (#1)
%7|1520327542.201|FEATURE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1520327542.201|STATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/bootstrap: Broker changed state CONNECT -> APIVERSION_QUERY
%7|1520327542.201|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: Broadcasting state change
%7|1520327542.203|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8093/bootstrap: ===== Received metadata: connected =====
%7|1520327542.203|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8093/bootstrap: ClusterId: sRli_no0R7eJoYM4pejDug, ControllerId: 1
%7|1520327542.203|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8093/bootstrap: 3 brokers, 0 topics
%7|1520327542.203|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8093/bootstrap: Broker #0/3: 192.168.2.6:8094 NodeId 2
%7|1520327542.203|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8093/bootstrap: Broker #1/3: 192.168.2.6:8093 NodeId 1
%7|1520327542.203|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8093/bootstrap: Broker #2/3: 192.168.2.6:8092 NodeId 0
%7|1520327542.203|UPDATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/bootstrap: NodeId changed from -1 to 2
%7|1520327542.203|UPDATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/2: Name changed from ssl://192.168.2.6:8094/bootstrap to ssl://192.168.2.6:8094/2
%7|1520327542.203|LEADER|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/2: Mapped 0 partition(s) to broker
%7|1520327542.203|UPDATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/bootstrap: NodeId changed from -1 to 0
%7|1520327542.203|UPDATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/0: Name changed from ssl://192.168.2.6:8092/bootstrap to ssl://192.168.2.6:8092/0
%7|1520327542.203|LEADER|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/0: Mapped 0 partition(s) to broker
%7|1520327542.203|UPDATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/bootstrap: NodeId changed from -1 to 1
%7|1520327542.203|UPDATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/1: Name changed from ssl://192.168.2.6:8093/bootstrap to ssl://192.168.2.6:8093/1
%7|1520327542.203|CLUSTERID|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8093/1: ClusterId update "" -> "sRli_no0R7eJoYM4pejDug"
%7|1520327542.203|SUBSCRIPTION|rdkafka#consumer-1| [thrd:main]: Group "group.test": no topics in metadata matched subscription
%7|1520327542.203|BRKREASSIGN|rdkafka#consumer-1| [thrd:main]: Group "group.test" management reassigned from broker :0/internal to ssl://192.168.2.6:8093/1
%7|1520327542.203|BRKUNASSIGN|rdkafka#consumer-1| [thrd:main]: Group "group.test" management unassigned from broker handle :0/internal
%7|1520327542.203|BRKASSIGN|rdkafka#consumer-1| [thrd:main]: Group "group.test" management assigned to broker ssl://192.168.2.6:8093/1
%7|1520327542.203|CGRPQUERY|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8093/1: Group "group.test": querying for coordinator: intervaled in state wait-broker
%7|1520327542.203|CONNECTED|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/2: Connected (#1)
%7|1520327542.203|FEATURE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/2: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1520327542.203|STATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/2: Broker changed state CONNECT -> APIVERSION_QUERY
%7|1520327542.203|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: Broadcasting state change
%7|1520327542.203|LEADER|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/1: Mapped 0 partition(s) to broker
%7|1520327542.203|STATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/1: Broker changed state UP -> UPDATE
%7|1520327542.203|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: Broadcasting state change
%7|1520327542.203|STATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: ssl://192.168.2.6:8093/1: Broker changed state UPDATE -> UP
%7|1520327542.203|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8093/bootstrap]: Broadcasting state change
%7|1520327542.206|FEATURE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/0: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime
%7|1520327542.206|STATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/0: Broker changed state APIVERSION_QUERY -> UP
%7|1520327542.206|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: Broadcasting state change
%7|1520327542.206|METADATA|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: ssl://192.168.2.6:8092/0: Request metadata for brokers only: connected
%7|1520327542.207|FEATURE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/2: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime
%7|1520327542.207|STATE|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/2: Broker changed state APIVERSION_QUERY -> UP
%7|1520327542.207|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: Broadcasting state change
%7|1520327542.207|METADATA|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/2: Request metadata for brokers only: connected
%7|1520327542.207|METADATA|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8094/bootstrap]: ssl://192.168.2.6:8094/2: Skipping metadata request: connected: full request already in-transit
%7|1520327542.207|CGRPCOORD|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8093/1: Group "group.test" coordinator is 192.168.2.6:8092 id 0
%7|1520327542.207|CGRPCOORD|rdkafka#consumer-1| [thrd:main]: Group "group.test" changing coordinator -1 -> 0
%7|1520327542.207|BRKREASSIGN|rdkafka#consumer-1| [thrd:main]: Group "group.test" management reassigned from broker ssl://192.168.2.6:8093/1 to ssl://192.168.2.6:8092/0
%7|1520327542.207|BRKUNASSIGN|rdkafka#consumer-1| [thrd:main]: Group "group.test" management unassigned from broker handle ssl://192.168.2.6:8093/1
%7|1520327542.207|BRKASSIGN|rdkafka#consumer-1| [thrd:main]: Group "group.test" management assigned to broker ssl://192.168.2.6:8092/0
%7|1520327542.207|CGRPSTATE|rdkafka#consumer-1| [thrd:main]: Group "group.test" changed state wait-broker -> wait-broker-transport (v1, join-state wait-unassign)
%7|1520327542.207|BROADCAST|rdkafka#consumer-1| [thrd:main]: Broadcasting state change
%7|1520327542.207|CGRPSTATE|rdkafka#consumer-1| [thrd:main]: Group "group.test" changed state wait-broker-transport -> up (v1, join-state wait-unassign)
%7|1520327542.207|BROADCAST|rdkafka#consumer-1| [thrd:main]: Broadcasting state change
%7|1520327542.207|CGRPQUERY|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8094/2: Group "group.test": querying for coordinator: intervaled in state up
%7|1520327542.207|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "group.test" received op OFFSET_COMMIT (v0) in state up (join state wait-unassign, v1 vs 0)
%7|1520327542.207|OFFSET|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: Enqueue OffsetCommitRequest(v1, 1/1 partition(s))): manual
%7|1520327542.209|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: ===== Received metadata: connected =====
%7|1520327542.209|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: ClusterId: sRli_no0R7eJoYM4pejDug, ControllerId: 1
%7|1520327542.209|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: 3 brokers, 0 topics
%7|1520327542.209|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: Broker #0/3: 192.168.2.6:8094 NodeId 2
%7|1520327542.209|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: Broker #1/3: 192.168.2.6:8093 NodeId 1
%7|1520327542.209|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: Broker #2/3: 192.168.2.6:8092 NodeId 0
%7|1520327542.209|SUBSCRIPTION|rdkafka#consumer-1| [thrd:main]: Group "group.test": no topics in metadata matched subscription
%7|1520327542.210|CGRPCOORD|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8094/2: Group "group.test" coordinator is 192.168.2.6:8092 id 0
%7|1520327542.215|COMMIT|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: OffsetCommit for 1 partition(s): manual: returned: Success
%7|1520327542.215|UNASSIGN|rdkafka#consumer-1| [thrd:main]: Group "group.test": unassign done in state up (join state wait-unassign): without new assignment: OffsetCommit done
%7|1520327542.215|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "group.test" changed join state wait-unassign -> init (v1, state up)
%7|1520327542.215|JOIN|rdkafka#consumer-1| [thrd:main]: Group "group.test": join with 0 (1) subscribed topic(s)
%7|1520327542.215|METADATA|rdkafka#consumer-1| [thrd:main]: Hinted cache of 1/1 topic(s) being queried
%7|1520327542.215|CGRPMETADATA|rdkafka#consumer-1| [thrd:main]: consumer join: metadata for subscription only available for 0/1 topics (-1ms old)
%7|1520327542.215|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: Request metadata for 1 topic(s): consumer join
%7|1520327542.215|JOIN|rdkafka#consumer-1| [thrd:main]: Group "group.test": postponing join until up-to-date metadata is available
%7|1520327542.219|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: ===== Received metadata (for 1 requested topics): consumer join =====
%7|1520327542.219|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: ClusterId: sRli_no0R7eJoYM4pejDug, ControllerId: 1
%7|1520327542.219|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: 3 brokers, 1 topics
%7|1520327542.219|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: Broker #0/3: 192.168.2.6:8094 NodeId 2
%7|1520327542.219|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: Broker #1/3: 192.168.2.6:8093 NodeId 1
%7|1520327542.219|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: Broker #2/3: 192.168.2.6:8092 NodeId 0
%7|1520327542.219|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: Topic #0/1: TOPIC_TEST with 1 partitions
%7|1520327542.219|METADATA|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: 1/1 requested topic(s) seen in metadata
%7|1520327542.219|SUBSCRIPTION|rdkafka#consumer-1| [thrd:main]: Group "group.test": effective subscription list changed from 0 to 1 topic(s):
%7|1520327542.219|SUBSCRIPTION|rdkafka#consumer-1| [thrd:main]: Topic TOPIC_TEST with 1 partition(s)
%7|1520327542.219|REJOIN|rdkafka#consumer-1| [thrd:main]: Group "group.test": subscription updated from metadata change: rejoining group
%7|1520327542.219|GRPLEADER|rdkafka#consumer-1| [thrd:main]: Group "group.test": resetting group leader info: Group rejoin
%7|1520327542.219|REJOIN|rdkafka#consumer-1| [thrd:main]: Group "group.test" rejoining in join-state init without an assignment
%7|1520327542.219|JOIN|rdkafka#consumer-1| [thrd:main]: Group "group.test": join with 1 (1) subscribed topic(s)
%7|1520327542.219|CGRPMETADATA|rdkafka#consumer-1| [thrd:main]: consumer join: metadata for subscription is up to date (0ms old)
%7|1520327542.219|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "group.test" changed join state init -> wait-join (v1, state up)
%7|1520327542.219|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: Broadcasting state change
%7|1520327542.223|JOINGROUP|rdkafka#consumer-1| [thrd:main]: JoinGroup response: GenerationId 0, Protocol , LeaderId (me), my MemberId , 0 members in group: Broker: Invalid session timeout
%7|1520327542.223|REQERR|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: JoinGroupRequest failed: Broker: Invalid session timeout: explicit actions 0x0
%7|1520327542.223|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "group.test" changed join state wait-join -> init (v1, state up)
%7|1520327543.149|COMMIT|rdkafka#consumer-1| [thrd:main]: OffsetCommit internal error: Local: No offset stored
%7|1520327543.149|COMMIT|rdkafka#consumer-1| [thrd:main]: OffsetCommit for -1 partition(s): cgrp auto commit timer: returned: Local: No offset stored
%7|1520327543.149|UNASSIGN|rdkafka#consumer-1| [thrd:main]: Group "group.test": unassign done in state up (join state init): without new assignment: OffsetCommit done (__NO_OFFSET)
%7|1520327544.149|COMMIT|rdkafka#consumer-1| [thrd:main]: OffsetCommit internal error: Local: No offset stored
%7|1520327544.149|COMMIT|rdkafka#consumer-1| [thrd:main]: OffsetCommit for -1 partition(s): cgrp auto commit timer: returned: Local: No offset stored
%7|1520327544.149|UNASSIGN|rdkafka#consumer-1| [thrd:main]: Group "group.test": unassign done in state up (join state init): without new assignment: OffsetCommit done (__NO_OFFSET)
%7|1520327545.149|JOIN|rdkafka#consumer-1| [thrd:main]: Group "group.test": join with 1 (1) subscribed topic(s)
%7|1520327545.149|CGRPMETADATA|rdkafka#consumer-1| [thrd:main]: consumer join: metadata for subscription is up to date (2929ms old)
%7|1520327545.149|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "group.test" changed join state init -> wait-join (v1, state up)
%7|1520327545.149|COMMIT|rdkafka#consumer-1| [thrd:main]: OffsetCommit internal error: Local: No offset stored
%7|1520327545.149|COMMIT|rdkafka#consumer-1| [thrd:main]: OffsetCommit for -1 partition(s): cgrp auto commit timer: returned: Local: No offset stored
%7|1520327545.149|UNASSIGN|rdkafka#consumer-1| [thrd:main]: Group "group.test": unassign done in state up (join state wait-join): without new assignment: OffsetCommit done (__NO_OFFSET)
%7|1520327545.150|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: Broadcasting state change
%7|1520327545.154|JOINGROUP|rdkafka#consumer-1| [thrd:main]: JoinGroup response: GenerationId 0, Protocol , LeaderId (me), my MemberId , 0 members in group: Broker: Invalid session timeout
%7|1520327545.154|REQERR|rdkafka#consumer-1| [thrd:main]: ssl://192.168.2.6:8092/0: JoinGroupRequest failed: Broker: Invalid session timeout: explicit actions 0x0
%7|1520327545.154|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "group.test" changed join state wait-join -> init (v1, state up)
%7|1520327546.149|COMMIT|rdkafka#consumer-1| [thrd:main]: OffsetCommit internal error: Local: No offset stored
%7|1520327546.149|COMMIT|rdkafka#consumer-1| [thrd:main]: OffsetCommit for -1 partition(s): cgrp auto commit timer: returned: Local: No offset stored
%7|1520327546.149|UNASSIGN|rdkafka#consumer-1| [thrd:main]: Group "group.test": unassign done in state up (join state init): without new assignment: OffsetCommit done (__NO_OFFSET)
%7|1520327547.149|JOIN|rdkafka#consumer-1| [thrd:main]: Group "group.test": join with 1 (1) subscribed topic(s)
%7|1520327547.149|CGRPMETADATA|rdkafka#consumer-1| [thrd:main]: consumer join: metadata for subscription is up to date (4929ms old)
%7|1520327547.149|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "group.test" changed join state init -> wait-join (v1, state up)
%7|1520327547.149|BROADCAST|rdkafka#consumer-1| [thrd:ssl://192.168.2.6:8092/bootstrap]: Broadcasting state change
%7|1520327547.149|COMMIT|rdkafka#consumer-1| [thrd:main]: OffsetCommit internal error: Local: No offset stored

How to reproduce

`

conf = { 'group.id': 'group.test',
         'bootstrap.servers': '192.168.2.6:8092,192.168.2.6:8093,192.168.2.6:8094',
         'security.protocol': 'ssl' ,
         'ssl.key.location': './kafka_test_cert/client.key.unsecure',
         'ssl.ca.location': './kafka_test_cert/ca-cert',
         'ssl.certificate.location': './kafka_test_cert/ca-cert/client.crt',
         'ssl.key.password': '123456',
         'api.version.request': True,
         'broker.version.fallback': '0.10.2.1',
         'log.connection.close': False,
         'log_level': 6,
         'message.max.bytes': 1000000,
         'statistics.interval.ms': 0,
         'session.timeout.ms': 1000000,
         'enable.auto.commit': True,
         'auto.commit.interval.ms': 1000,
         'fetch.min.bytes': 1024 * 1024,
         'fetch.wait.max.ms': 1000,
         'fetch.message.max.bytes': 1048576,
         'default.topic.config': {
             'auto.offset.reset': 'smallest',
         },
         'debug': 'generic,broker,topic,metadata,cgrp,fetch',}

consumer = Consumer(**conf)

consumer.commit(offsets=[TopicPartition('TOPIC_TEST',0,4269)], async=True)
consumer.subscribe(['TOPIC_TEST'])

while True:

    result = consumer.poll(timeout=0.00001)

    if not result:
        continue

    if result.error():
        if result.error().code() == KafkaError._PARTITION_EOF:
            logging.warning('%s [%d] reached end at offset %d\n'.format(
                result.topic(), result.partition(), result.offset()
            ))
        else:
            logging.error('encourage error:\n{}'.format(json.dumps(
                {
                    'name': result.error().name(),
                    'code': result.error().code(),
                    'description': result.error().str()
                }, indent=1)))
    else:
        logging.info(result.offset())`

Checklist

Please provide the following information:

  • [√ ] confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): ('0.11.0', 720896) and ('0.11.3', 721919)

  • [√ ] Apache Kafka broker version: 0.10.2.1

  • [√ ] Client configuration: `

       { 'group.id': 'group.test',
      'bootstrap.servers': '192.168.2.6:8092,192.168.2.6:8093,192.168.2.6:8094',
      'security.protocol': 'ssl' ,
      'ssl.key.location': './kafka_test_cert/client.key.unsecure',
      'ssl.ca.location': './kafka_test_cert/ca-cert',
      'ssl.certificate.location': './kafka_test_cert/ca-cert/client.crt',
      'ssl.key.password': '123456',
      'api.version.request': True,
      'broker.version.fallback': '0.10.2.1',
      'log.connection.close': False,
      'log_level': 6,
      'message.max.bytes': 1000000,
      'statistics.interval.ms': 0,
      'session.timeout.ms': 1000000,
      'enable.auto.commit': True,
      'auto.commit.interval.ms': 1000,
      'fetch.min.bytes': 1024 * 1024,
      'fetch.wait.max.ms': 1000,
      'fetch.message.max.bytes': 1048576,
      'default.topic.config': {
          'auto.offset.reset': 'smallest',
      },
      'debug': 'generic,broker,topic,metadata,cgrp,fetch',}`

  • [√ ] Operating system: CentOS

  • [√ ] Provide client logs (with 'debug': '..' as necessary)

  • Provide broker log excerpts

  • Critical issue
@edenhill
Copy link
Contributor

edenhill commented Mar 6, 2018

%7|1520327542.223|JOINGROUP|rdkafka#consumer-1| [thrd:main]: JoinGroup response: GenerationId 0, Protocol , LeaderId (me), my MemberId , 0 members in group: Broker: Invalid session timeout

@edenhill
Copy link
Contributor

edenhill commented Mar 6, 2018

Your session.timeout.ms is out of range

@joukosusi
Copy link
Author

@edenhill sorry for my fault, will close this issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@edenhill @joukosusi