kafka--通過python操作topic

修改 topic 的分區數

shiyanlou:bin/ $ ./kafka-topics.sh --zookeeper localhost:2181 --alter --topic mySendTopic --partitions 4
WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
Adding partitions succeeded!

如果 topic 的 partition 數量增加了，那么消息的分區邏輯或者消息的順序都會受到影響。這一點我們在學習 consumer 課程中詳解。

查看是否增加了：

shiyanlou:bin/ $ ./kafka-topics.sh --zookeeper localhost:2181 --describe --topic mySendTopic           [15:00:54]
Topic:mySendTopic PartitionCount:4 ReplicationFactor:2 Configs:
 Topic: mySendTopic Partition: 0 Leader: 2 Replicas: 2,0 Isr: 2,0
 Topic: mySendTopic Partition: 1 Leader: 0 Replicas: 0,1 Isr: 0,1
 Topic: mySendTopic Partition: 2 Leader: 1 Replicas: 1,2 Isr: 1,2
 Topic: mySendTopic Partition: 3 Leader: 2 Replicas: 2,1 Isr: 2,1

對於一個topic來說，其partition的數量，只能增加，不能減少

刪除topic：

shiyanlou:bin/ $./kafka-topics.sh --zookeeper localhost:2181 --delete --topic myDeleteTopic

myDeleteTopic 只是標記為刪除，沒有真正刪除。要想徹底刪除，需在 server.properties 中設置 delete.topic.enable=true。運行如下命令驗證這一點：./kafka-topics.sh --zookeeper localhost:2181 --list

安裝python操作kafka的包

shiyanlou:bin/ $ sudo pip install pykafka

shiyanlou:~/ $ cat pykafk.py                                         [16:10:15]
#!/usr/bin/python

from pykafka import KafkaClient
client = KafkaClient(hosts="127.0.0.1:9092,127.0.0.1:9093, 127.0.0.1:9094")
#查看主題
print client.topics
#查看brokers
print client.brokers
topic = client.topics['mySendTopic']
for n in client.brokers:
    host = client.brokers[n].host
    port = client.brokers[n].port
    id = client.brokers[n].port
    print "host=%s |port=%s|broker.id=%s" %(host,port,id)
shiyanlou:~/ $ python pykafk.py                                      [16:10:31]
{'myFirstTopic': None, 'mySendTopic': None}
{0: <pykafka.broker.Broker at 0x7ff88c363590 (host=857dafc13648, port=9092, id=0)>, 1: <pykafka.broker.Broker at 0x7ff88c363dd0 (host=857dafc13648, port=9093, id=1)>, 2: <pykafka.broker.Broker at 0x7ff88c376110 (host=857dafc13648, port=9094, id=2)>}
host=857dafc13648 |port=9092|broker.id=9092
host=857dafc13648 |port=9093|broker.id=9093
host=857dafc13648 |port=9094|broker.id=9094