kafka系列十、kafka常用管理命令

一、Topic管理

1、創建topic

kafka-topics.sh --zookeeper 47.52.199.52:2181 --create --topic test-15 --replication-factor 1 --partitions 3

2、新增partition

 kafka-topics.sh --zookeeper zk.server --alter --topic test --replication-factor 1 --partitions 3

注：topic一旦創建，partition只能增加，不能減少

3、刪除topic

kafka-topics.sh --zookeeper zk.server --delete --topic test 

4、查看topic列表

kafka-topics.sh --zookeeper zk.server --list

5、查看topic詳細信息

kafka-topics.sh --zookeeper zk.server --topic test --describe

6、查看某個topic的message數量

./bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list 47.52.199.51:9092 --topic consumer-send

二、consumer管理

1、查看consumer Group列表

./bin/kafka-consumer-groups.sh  --list  --bootstrap-server 192.168.88.108:9092

2、查看指定group.id的消費情況

./bin/kafka-consumer-groups.sh --bootstrap-server 47.52.199.51:9092 --group test-1 --describe

結果：

TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                     HOST            CLIENT-ID
consumer-send   0          5697            5697            0               consumer-3-77500c76-ef09-4047-9ae0-89c20cc379fc /183.17.238.48  consumer-3
producer-syn    0          4125            4125            0               -                                               -               -

CURRENT-OFFSET:當前消費偏移量

LOG-END-OFFSET：末尾偏移量

3、刪除group

./bin/kafka-consumer-groups.sh --bootstrap-server 47.52.199.51:9092 --group test-1 --delete

4、重置offset

1、要求修改的group不能active,查看是否active

[root@izj6c46svwddzpu0evy0vbz kafka_2.11-2.0.1]# ./bin/kafka-consumer-groups.sh --bootstrap-server 47.52.199.51:9092 --group test_4 --describe
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Consumer group 'test_4' has no active members.

TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID     HOST            CLIENT-ID
consumer-send   0          5697            5697            0               -               -               -
producer-syn    0          4125            4125            0               -               -               -

2、重置命令

./bin/kafka-consumer-groups.sh --bootstrap-server 47.52.199.51:9092 --group test_4 --reset-offsets -to-offset 100 --topic consumer-send --execute

3、導出offset

./bin/kafka-consumer-groups.sh --bootstrap-server 47.52.199.51:9092 --group test_4 --reset-offsets -to-offset 100 --topic consumer-send --export > 1.txt

三、動態配置

1、再平衡

./bin/kafka-preferred-replica-election.sh --zookeeper 47.52.199.51:2181/chroot

四、生產消費者

1、啟動kafka

nohup bin/kafka-server-start.sh config/server.properties > /dev/null 2>&1 &

2、創建消費者

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic middleware --from-beginning

3、創建生產者

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test