kafka分區遷移失敗任務的處理


前一個文章"單副本kafka遷移分區"介紹了,怎樣對分區進行遷移,然而在遷移最后一台broker的過程中,controler掛掉了,導致遷移任務一直沒有進度,這里記錄一下處理的過程

如果有一個遷移任務在執行時,重復執行,會有下面的提示,./bin/kafka-reassign-partitions.sh 是不能進行取消或者更改遷移任務狀態的。

kafka.common.AdminCommandFailedException: Partition reassignment currently in progress for Map

 

處理思路:

1.登陸zk,把當前的任務刪除

2.重新執行任務

 

具體操作:

2.1.登陸zk,刪除任務

[zk: localhost:8181(CONNECTED) 2] ls /
[cluster, controller_epoch, controller, brokers, zookeeper, kafka-manager, admin, isr_change_notification, consumers, burrow, latest_producer_id_block, config]
[zk: localhost:8181(CONNECTED) 3] ls /admin/reassign_partitions
[]

#查看當前的任務
[zk: localhost:8181(CONNECTED) 5] get /admin/reassign_partitions 
{"version":1,"partitions":[{"topic":"yuanlog_json","partition":589,"replicas":[135]},{"topic":"………………
cZxid = 0x403a9ff6a
ctime = Thu Jul 25 10:50:12 CST 2019
mZxid = 0x403c68a74
mtime = Fri Jul 26 05:47:36 CST 2019
pZxid = 0x403a9ff6a
cversion = 0
dataVersion = 136189
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 9348
numChildren = 0

#刪除任務
[zk: localhost:8181(CONNECTED) 8] rmr /admin/reassign_partitions

[work@kafka-node-134 kafka_2.11-0.10.0.0]$ ./bin/kafka-reassign-partitions.sh --zookeeper 10.69.59.186:8181 --reassignment-json-file ./tmp/136_exec0801.json --execute
Current partition replica assignment

2.2 重新執行遷移任務

[work@kafka-node-134 kafka_2.11-0.10.0.0]$ ./bin/kafka-reassign-partitions.sh --zookeeper 10.10.10.1:8181 --reassignment-json-file ./tmp/136_exec0801.json --execute
Current partition replica assignment

………………

Successfully started reassignment of partitions {"version":1,"partitions":[{"topic":"XX","partition":237,"replicas":[157]}]}

2.3 查看任務狀態

[work@kafka-node-134 kafka_2.11-0.10.0.0]$ ./bin/kafka-reassign-partitions.sh --zookeeper 10.10.10.1:8181 --reassignment-json-file ./tmp/136_exec0801.json --verify

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM