拉新网 - 挖掘京东今日特价爆款商品 - 线报平台

小天天

Moderator

注册时间： 2019-09-29

帖子： 886

zookeeper 迁移后导致kafka 报错问题处理

给zookeeper迁移导致问题现象如下：

1.kibana无法读取最新数据
2.kafka 机器的/var/log/message报错如下

Feb 14 17:42:09 log-kafka-37 kafka_exporter: E0214 17:42:09.368293   17973 kafka_exporter.go:438] Cannot get oldest offset of topic elk partition 0: kafka server: Tried to send a message to a replica that is not the leader for some partition. Your metadata is out of date
Feb 14 17:42:24 log-kafka-37 kafka_exporter: E0214 17:42:24.317699   17973 kafka_exporter.go:426] Cannot get current offset of topic elk partition 0: kafka server: Tried to send a message to a replica that is not the leader for some partition. Your metadata is out of date

处理办法：
查看并删除kafka旧的topic

查看topic信息
1）
bin/kafka-topics.sh --list --bootstrap-server kafka-36:9092,kafka-37:9092,kafka-38:9092
》__consumer_offsets
》elk

bin/kafka-topics.sh --describe --bootstrap-server kafka-36:9092,kafka-37:9092,kafka-38:9092 --topic elk
bin/kafka-topics.sh --describe --bootstrap-server kafka-36:9092,kafka-37:9092,kafka-38:9092 --topic __consumer_offsets

3）删除topic
bin/kafka-topics.sh --delete--bootstrap-server kafka-36:9092,kafka-37:9092,kafka-38:9092 --topic elk
bin/kafka-topics.sh --delete --bootstrap-server kafka-36:9092,kafka-37:9092,kafka-38:9092 --topic __consumer_offsets

即可自动创建并恢复