環境基於docker，三台物理主機192.168.0.27、192.168.0.28、192.168.0.29，每台主機部署一個zookeeper節點，一個kafka節點，共三個zookeeper節點，三個kafka節點，容器之間的網絡采用host模式
1.拉取鏡像
2.啟動容器
step1. 創建配置文件zoo.cfg並替換容器內的該文件，不同的鏡像有可能zoo.cfg位置不同
# The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/opt/zookeeper-3.4.13/data # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1 server.27=192.168.0.27:2888:3888 server.28=192.168.0.28:2888:3888 server.29=192.168.0.29:2888:3888 myid=27
3.每台機器都要替換，注意myid需要不同，該鏡像中/opt/zookeeper-3.4.13/data下沒有myid，所以需要通過掛載的方式添加或者進容器創建
4.啟動zookeeper（每台機器執行）
docker run -p 2181:2181 -p 2888:2888 -p 3888:3888 --name zookeeper27 --network host -v /images/zoo.cfg:/opt/zookeeper-3.4.13/conf/zoo.cfg -v /images/myid:/opt/zookeeper-3.4.13/data/myid -it wurstmeister/kafka:latest
5.查看zookeeper運行情況
#進入容器 docker exec -it zookeeper27 #查看zookeeper運行情況 echo stat | nc 192.168.0.27
可以看到zookeeper集群啟動成功，並自動選舉了lader
6.啟動kafka
kafka集群同樣采用host模式
zks="192.168.0.27:2181,192.168.0.28:2181,192.168.0.29:2181"; docker run -p 9092:9092 --name kafka27 --network host -d -e KAFKA_BROKER_ID=27 -e KAFKA_ZOOKEEPER_CONNECT=${zks} -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://宿主機IP:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 wurstmeister/kafka:latest
7.分別進入容器kafka27、kafka28、kafka29創建主題test27、test28、test29。下面以kafka27為例。
#創建topic ./kafka-topics.sh --create --zookeeper 192.168.0.27:2181,192.168.0.28:2181,192.168.0.29:2181 --replication-factor 3 --partitions 3 --topic test27 #replication-factor 表示該topic需要在不同的broker中保存幾份, partitions為幾個分區 #查看已經創建的topic ./kafka-topics.sh --list --zookeeper 192.168.0.27:2181,192.168.0.28:2181,192.168.0.29:2181 #查看指定topic詳情 ./kafka-topics.sh --zookeeper 192.168.0.27:2181,192.168.0.28:2181,192.168.0.29:2181 --topic test27 --describe #創建生產者 ./kafka-console-producer.sh --broker-list 192.168.0.27:9092,192.168.0.28:9092,192.168.0.29:9092 --topic test、 #創建消費者 ./kafka-console-consumer.sh --bootstrap-server 192.168.0.27:9092,192.168.0.28:9092,192.168.0.29:9092 --topic test --from-beginning