Kafka常见错误

Dear 丶 2023-05-28 05:18 3阅读 0赞

Kafka常见错误：

1）Malformed line in offset checkpoint file

cat logs/server.log.2020-03-27-12

\[2020-03-27 12:59:50,150\] ERROR \[KafkaApi-100\] Error when handling request \{controller\_id=100,controller\_epoch=4,partition\_states=\[\{topic=test03,partition=0,controller\_epoch=3,leader=100,leader\_epoch=2,isr=\[100\],zk\_version=2,replicas=\[100\]\},\{topic=test03,partition=3,controller\_epoch=3,leader=100,leader\_epoch=2,isr=\[100\],zk\_version=2,replicas=\[100\]\},\{topic=test03,partition=2,controller\_epoch=3,leader=100,leader\_epoch=2,isr=\[100\],zk\_version=2,replicas=\[100\]\},\{topic=test03,partition=1,controller\_epoch=3,leader=100,leader\_epoch=2,isr=\[100\],zk\_version=2,replicas=\[100\]\}\],live\_leaders=\[\{id=100,host=192.168.100.148,port=9092\}\]\} (kafka.server.KafkaApis)  
java.io.IOException: Malformed line in offset checkpoint file: '  
        at kafka.server.OffsetCheckpoint.malformedLineException$1(OffsetCheckpoint.scala:81)  
        at kafka.server.OffsetCheckpoint.liftedTree2$1(OffsetCheckpoint.scala:104)  
        at kafka.server.OffsetCheckpoint.read(OffsetCheckpoint.scala:86)  
        at kafka.cluster.Partition.getOrCreateReplica(Partition.scala:98)  
        at kafka.cluster.Partition$$anonfun$4$$anonfun$apply$2.apply(Partition.scala:176)  
        at kafka.cluster.Partition$$anonfun$4$$anonfun$apply$2.apply(Partition.scala:176)  
        at scala.collection.mutable.HashSet.foreach(HashSet.scala:78)  
        at kafka.cluster.Partition$$anonfun$4.apply(Partition.scala:176)  
        at kafka.cluster.Partition$$anonfun$4.apply(Partition.scala:170)  
        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231)  
        at kafka.utils.CoreUtils$.inWriteLock(CoreUtils.scala:239)  
        at kafka.cluster.Partition.makeLeader(Partition.scala:170)  
        at kafka.server.ReplicaManager$$anonfun$makeLeaders$4.apply(ReplicaManager.scala:699)  
        at kafka.server.ReplicaManager$$anonfun$makeLeaders$4.apply(ReplicaManager.scala:698)  
        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)  
        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)  
        at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)  
        at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)  
        at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)  
        at kafka.server.ReplicaManager.makeLeaders(ReplicaManager.scala:698)  
        at kafka.server.ReplicaManager.becomeLeaderOrFollower(ReplicaManager.scala:644)  
        at kafka.server.KafkaApis.handleLeaderAndIsrRequest(KafkaApis.scala:144)  
        at kafka.server.KafkaApis.handle(KafkaApis.scala:80)  
        at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:60)  
        at java.lang.Thread.run(Thread.java:745)  
\[2020-03-27 12:59:50,159\] INFO \[ReplicaFetcherManager on broker 100\] Removed fetcher for partitions \[test03,3\],\[test03,0\],\[test03,1\],\[test03,2\] (kafka.server.ReplicaFetcherManager)

可能原因：有可能增加了分区后，kafka就不能找到分区leader了，报该错误。kafka会在log.dirs配置项的路径下维护这两个和offset相关的值：  
文件1）replication-offset-checkpoint  
文件2）recovery-point-offset-checkpoint  
解决办法：删除所有log文件夹（查看config/server.properties中的log.dirs配置路径）下的这两个文件即可。