Kafka日志:
broker 133 received LeaderAndIsrRequest with correlation id 1 from controller 132 epoch 35 fro partition [__consumer_offsets,13] but cannot become follower since the new leader -1 is unavailable
this server is not the leader for that topic-partition
對於broker 133 received LeaderAndIsrRequest with correlation id 1 from controller 132 epoch 35 fro partition [__consumer_offsets,13] but cannot become follower since the new leader -1 is unavailable
錯誤是因為
因其中一個broker掛掉無法獲取leader的brokerid
但3個broker中的其中一個掛掉,是可以將數據寫入另外兩個broker的。至於沒有寫入另外兩個broker報錯org.apache.kafka.common.errors.NotLeaderForPartition可能是因為可能我們的producer端的代碼里沒加 reties 參數,默認就發送一次,遇到leader選舉時,找不到leader就會發送失敗,造成程序停止
解決辦法
producer端加上參數 reties=3, 重試發送三次(默認100ms重試一次 由 retry.backoff.ms控制);
如果還需要保證消息發送的有序性,記得加上參數 max.in.flight.requests.per.connection = 1 限制客戶端在單個連接上能夠發送的未響應請求的個數,設置此值是1表示kafka broker在響應請求之前client不能再向同一個broker發送請求。(注意:設置此參數是為了滿足必須順序消費的場景,比如binlog數據)