Kafka leader -1

水深无声 2022-11-14 10:10 167阅读 0赞

问题错误日志：

21/03/28 10:17:06 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:06 INFO internals.Fetcher: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Resetti  
ng offset for partition nb-EIP-SFA-database-0 to offset 138.  
21/03/28 10:17:06 INFO internals.Fetcher: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Resetti  
ng offset for partition log\_server\_stream-0 to offset 0.  
21/03/28 10:17:06 INFO internals.Fetcher: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Resetti  
ng offset for partition nb-EIP-DD-easysales\_market-0 to offset 349915.  
21/03/28 10:17:06 INFO internals.Fetcher: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Resetti  
ng offset for partition nb-EIP-DD-peoplesoft-0 to offset 0.  
21/03/28 10:17:06 INFO internals.Fetcher: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Resetti  
ng offset for partition nb-EIP-DD-wechat-0 to offset 4588.  
21/03/28 10:17:06 INFO internals.Fetcher: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Resetti  
ng offset for partition nb-EIP-DD-easysales\_bpit-0 to offset 792559.  
21/03/28 10:17:06 INFO internals.Fetcher: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Resetti  
ng offset for partition nb-EIP-DD-office365-0 to offset 1300090.  
21/03/28 10:17:06 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:06 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:06 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:07 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:08 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:09 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:10 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:11 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:12 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:13 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:14 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:15 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:16 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:17 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:18 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:19 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.

nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:58 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:17:59 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:18:00 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:18:01 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:18:02 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:18:03 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:18:04 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:18:05 WARN clients.NetworkClient: \[Consumer clientId=consumer-1, groupId=kafka\_loader\_20210325\] Con  
nection to node 3 could not be established. Broker may not be available.  
21/03/28 10:18:06 ERROR streaming.StreamingContext: Error starting the context, marking it as stopped  
org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before the position for partition n  
b-EIP-DD-master\_shared-0 could be determined  
        at ... run in separate thread using org.apache.spark.util.ThreadUtils ... ()  
        at org.apache.spark.streaming.StreamingContext.liftedTree1$1(StreamingContext.scala:583)  
        at org.apache.spark.streaming.StreamingContext.start(StreamingContext.scala:575)  
        at com.mininglamp.eip.cona.scala.kafka.loader.spark.streaming.RunLoader$.main(RunLoader.scala:283)  
        at com.mininglamp.eip.cona.scala.kafka.loader.spark.streaming.RunLoader.main(RunLoader.scala)  
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
        at java.lang.reflect.Method.invoke(Method.java:498)  
        at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:685)  
21/03/28 10:18:06 INFO scheduler.ReceiverTracker: ReceiverTracker stopped  
21/03/28 10:18:06 INFO scheduler.JobGenerator: Stopping JobGenerator immediately  
21/03/28 10:18:06 INFO util.RecurringTimer: Stopped timer for JobGenerator after time -1  
21/03/28 10:18:06 INFO scheduler.JobGenerator: Stopped JobGenerator

bin/kafka-topics.sh --describe --zookeeper ip:30001 --topic test  
Topic: test      PartitionCount:2        ReplicationFactor:1     Configs:  
        Topic: test Partition: 0    Leader: -1      Replicas: 3     Isr: 3  
        Topic: test Partition: 1    Leader: 0       Replicas: 0     Isr: 0  
          
发现这个topic的leader是： -1

方法1：  
删除Topic，重新创建。

首先查看topic：

方法2：

直接修改：

set /brokers/topics/test/partitions/0/state \{"controller\_epoch":140,"leader":0,"version":1,"leader\_epoch":12,"isr":\[0\]\}  
查看状态：  
get /brokers/topics/test/partitions/0/state

\{"controller\_epoch":452,"leader":0,"version":1,"leader\_epoch":3577,"isr":\[0\]\}  
cZxid = 0x4200000924  
ctime = Thu Mar 25 17:14:38 CST 2021  
mZxid = 0x4800002ee7  
mtime = Sun Mar 28 13:37:08 CST 2021  
pZxid = 0x4200000924  
cversion = 0  
dataVersion = 3578  
aclVersion = 0  
ephemeralOwner = 0x0  
dataLength = 77  
numChildren = 0

get /brokers/topics/test/partitions/1/state

解决方案  
直接修改zookeeper中该分区对应的state  
PS：非常不建议只有1个副本  
          
方法3： 修改topic的副本数量

创建一个json文件，内容如下：

\{  
  "version": 1,  
  "partitions": \[  
    \{  
      "topic": "input\_stream",  
      "partition": 0,  
      "replicas": \[0\]  
    \}  
  \]  
\}

执行修改命令  
   
bin/kafka-reassign-partitions.sh --zookeeper ip:30001 --reassignment-json-file /data/kafka\_2.11-0.11.0.0/input\_stream.json --execute  
   
修改完成后稍等片刻即可生效