怎么徹底刪除kafka的topic,然后重建?
網上都說用kafka-run-class.shkafka.admin.DeleteTopicCommand 命令刪除topic,但是並沒有成功,用
kafka-topics.sh命令查看依然可以查看到topic,應該怎樣才能徹底刪除topic?
kafka-topics.sh命令查看依然可以查看到topic,應該怎樣才能徹底刪除topic?
2 個回復
**kafka 0.8.1.1以及之前版本**都無法使用類似一條命令就徹底刪除topic,此命令不過只是在zookeeper注銷信息而已,但是實際的日志內容還是保存在kafka log中,如果想徹底刪除topic,過程如下:
1、從zookeerer刪除信息
./bin/kafka-run-class.sh kafka.admin.DeleteTopicCommand --zookeeper 10.0.1.10:2181,10.0.1.11:2181,10.0.1.12:2181 --topic test成功后返回信息:deletion succeeded!
2、利用JPS命令查看kafka和zookeeper進程,kill掉QuorumPeerMain和Kafka進程
3、從kafka的log.dirs目錄刪除文件,可以看到多個子目錄名字如test-0,test-1…test-n(就是你topic的partition個數)
進入到kafka的log.dirs目錄,執行
rm –fr test-0……test-n(4) 修改日志目錄的recovery-point-offset-checkpoint和replication-offset-checkpoint文件(要小心刪除,否則待會kafka不能正常啟動起來)
replication-offset-checkpoint格式如下:
0
4(partition總數)
test 0 0
test 3 0
hehe 0 0
hehe 1 0
修改后如下:
0
2(partition總數)
hehe 0 0
hehe 1 0
把含有test行全部去掉,並且把partition總數修改為減去test的partition的剩余數目,同理recovery-point-offset-checkpoint也是這樣修改。
完成后就可以正常啟動zookeeper和kafka。**從kafka 0.8.2.1**開始可以直接刪除topic,步驟如下:
kafka 0.8.1.1之前版本,kafka-topics.sh命令連--delete Delete a topic參數都沒有,如下:
# ./kafka-topics.sh --help
Command must include exactly one action: --list, --describe, --create or --alter
Option Description
------ -----------
--alter Alter the configuration for the topic.
--config <name=value> A topic configuration override for the
topic being created or altered.
--create Create a new topic.
--deleteConfig <name> A topic configuration override to be
removed for an existing topic
--describe List details for the given topics.
--help Print usage information.
--list List all available topics.
--partitions <Integer: # of partitions> The number of partitions for the topic
being created or altered (WARNING:
If partitions are increased for a
topic that has a key, the partition
logic or ordering of the messages
will be affected
--replica-assignment A list of manual partition-to-broker
<broker_id_for_part1_replica1 : assignments for the topic being
broker_id_for_part1_replica2 , created or altered.
broker_id_for_part2_replica1 :
broker_id_for_part2_replica2 , ...>
--replication-factor <Integer: The replication factor for each
replication factor> partition in the topic being created.
--topic <topic> The topic to be create, alter or
describe. Can also accept a regular
expression except for --create option
--topics-with-overrides if set when describing topics, only
show topics that have overridden
configs
--unavailable-partitions if set when describing topics, only
show partitions whose leader is not
available
--under-replicated-partitions if set when describing topics, only
show under replicated partitions
--zookeeper <urls> REQUIRED: The connection string for
the zookeeper connection in the form
host:port. Multiple URLS can be
given to allow fail-over.
# **而kafka 0.8.2.1**新增加了這個參數
刪除過程:
1、在kafka配置文件中添加刪除參數
delete.topic.enable=true2、利用命令刪除需要刪除的topic
bin/kafka-topics.sh --zookeeper zk_host:port/chroot --delete --topic my_topic_namehttp://kafka.apache.org/documentation.html#gettingStarted如下:
1、從zookeerer刪除信息
./bin/kafka-run-class.sh kafka.admin.DeleteTopicCommand --zookeeper 10.0.1.10:2181,10.0.1.11:2181,10.0.1.12:2181 --topic test成功后返回信息:deletion succeeded!
2、利用JPS命令查看kafka和zookeeper進程,kill掉QuorumPeerMain和Kafka進程
3、從kafka的log.dirs目錄刪除文件,可以看到多個子目錄名字如test-0,test-1…test-n(就是你topic的partition個數)
進入到kafka的log.dirs目錄,執行
rm –fr test-0……test-n(4) 修改日志目錄的recovery-point-offset-checkpoint和replication-offset-checkpoint文件(要小心刪除,否則待會kafka不能正常啟動起來)
replication-offset-checkpoint格式如下:
0
4(partition總數)
test 0 0
test 3 0
hehe 0 0
hehe 1 0
修改后如下:
0
2(partition總數)
hehe 0 0
hehe 1 0
把含有test行全部去掉,並且把partition總數修改為減去test的partition的剩余數目,同理recovery-point-offset-checkpoint也是這樣修改。
完成后就可以正常啟動zookeeper和kafka。**從kafka 0.8.2.1**開始可以直接刪除topic,步驟如下:
kafka 0.8.1.1之前版本,kafka-topics.sh命令連--delete Delete a topic參數都沒有,如下:
# ./kafka-topics.sh --help
Command must include exactly one action: --list, --describe, --create or --alter
Option Description
------ -----------
--alter Alter the configuration for the topic.
--config <name=value> A topic configuration override for the
topic being created or altered.
--create Create a new topic.
--deleteConfig <name> A topic configuration override to be
removed for an existing topic
--describe List details for the given topics.
--help Print usage information.
--list List all available topics.
--partitions <Integer: # of partitions> The number of partitions for the topic
being created or altered (WARNING:
If partitions are increased for a
topic that has a key, the partition
logic or ordering of the messages
will be affected
--replica-assignment A list of manual partition-to-broker
<broker_id_for_part1_replica1 : assignments for the topic being
broker_id_for_part1_replica2 , created or altered.
broker_id_for_part2_replica1 :
broker_id_for_part2_replica2 , ...>
--replication-factor <Integer: The replication factor for each
replication factor> partition in the topic being created.
--topic <topic> The topic to be create, alter or
describe. Can also accept a regular
expression except for --create option
--topics-with-overrides if set when describing topics, only
show topics that have overridden
configs
--unavailable-partitions if set when describing topics, only
show partitions whose leader is not
available
--under-replicated-partitions if set when describing topics, only
show under replicated partitions
--zookeeper <urls> REQUIRED: The connection string for
the zookeeper connection in the form
host:port. Multiple URLS can be
given to allow fail-over.
# **而kafka 0.8.2.1**新增加了這個參數
刪除過程:
1、在kafka配置文件中添加刪除參數
delete.topic.enable=true2、利用命令刪除需要刪除的topic
bin/kafka-topics.sh --zookeeper zk_host:port/chroot --delete --topic my_topic_namehttp://kafka.apache.org/documentation.html#gettingStarted如下:
版本不一樣,刪除方法不一樣,最好是升級到新版較穩定的比較好。
being • 2016-01-03 18:52
好回答,總結不錯!