centos安裝kafka

一、首先安裝zookeeper

1、下載

[root@orderer src]# cd /usr/local/src/
[root@orderer src]# wget http://mirrors.hust.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
--2020-06-02 16:49:53--  http://mirrors.hust.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
?????? mirrors.hust.edu.cn (mirrors.hust.edu.cn)... 202.114.18.160
???? mirrors.hust.edu.cn (mirrors.hust.edu.cn)|202.114.18.160|:80... ????
??? HTTP ??,??????... 200 OK
??:37676320 (36M) [application/octet-stream]
?????: “zookeeper-3.4.14.tar.gz.1”

100%[=================================================================================================================>] 37,676,320  1.91MB/s ?? 16s      

2020-06-02 16:50:09 (2.22 MB/s) - ??? “zookeeper-3.4.14.tar.gz.1” [37676320/37676320])

[root@orderer src]# 

2、解壓

[root@orderer src]# tar -zxvf zookeeper-3.4.14.tar.gz

3、移動到目標目錄下

[root@orderer src]# mv zookeeper-3.4.14/ ../

4、復制conf下的zoo_sample.cfg為zoo.cfg

[root@orderer conf]# cd /usr/local/zookeeper-3.4.14/conf/
[root@orderer conf]# cp zoo_sample.cfg zoo.cfg
[root@orderer conf]# 

5、在/usr/local/zookeeper-3.4.14下新建一個data目錄，作為數據目錄

root@orderer zookeeper-3.4.14]# mkdir /usr/local/zookeeper-3.4.14/data
[root@orderer zookeeper-3.4.14]# 

6、編輯zoo.cfg文件

dataDir=/usr/local/zookeeper-3.4.14/data

設置dataDir為剛才建立的目錄路徑

7、啟動zookeeper

[root@orderer zookeeper-3.4.14]# /usr/local/zookeeper-3.4.14/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@orderer zookeeper-3.4.14]#

8、查看狀態,默認監聽端口為2181

[root@orderer zookeeper-3.4.14]# /usr/local/zookeeper-3.4.14/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.14/bin/../conf/zoo.cfg
Mode: standalone
[root@orderer zookeeper-3.4.14]# 

9、停止

[root@orderer zookeeper-3.4.14]# /usr/local/zookeeper-3.4.14/bin/zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.14/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@orderer zookeeper-3.4.14]# 

 

二、安裝kafka

1、下載

[root@orderer src]# wget http://mirrors.hust.edu.cn/apache/kafka/2.2.2/kafka_2.11-2.2.2.tgz

2、解壓

[root@orderer src]# tar -zxvf kafka_2.11-2.2.2.tgz 

 3、移動到目標目錄下

[root@orderer src]# mv kafka_2.11-2.2.2/ ../

 4、編輯config/server.properties配置文件

[root@orderer kafka_2.11-2.2.2]# vim config/server.properties 

log.dirs=/usr/local/kafka_2.11-2.2.2/kafka-logs

5、首先啟動zookpeer

[root@orderer zookeeper-3.4.14]# bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@orderer zookeeper-3.4.14]#

6、再啟動kafka

[root@orderer kafka_2.11-2.2.2]# bin/kafka-server-start.sh config/server.properties

最后顯示

[2020-06-02 17:08:50,824] INFO [ProducerId Manager 0]: Acquired new producerId block (brokerId:0,blockStartProducerId:0,blockEndProducerId:999) by writing to Zk with path version 1 (kafka.coordinator.transaction.ProducerIdManager)
[2020-06-02 17:08:50,867] INFO [TransactionCoordinator id=0] Starting up. (kafka.coordinator.transaction.TransactionCoordinator)
[2020-06-02 17:08:50,870] INFO [TransactionCoordinator id=0] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)
[2020-06-02 17:08:50,871] INFO [Transaction Marker Channel Manager 0]: Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2020-06-02 17:08:50,946] INFO [/config/changes-event-process-thread]: Starting (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)
[2020-06-02 17:08:50,966] INFO [SocketServer brokerId=0] Started data-plane processors for 1 acceptors (kafka.network.SocketServer)
[2020-06-02 17:08:50,973] INFO Kafka version: 2.2.2 (org.apache.kafka.common.utils.AppInfoParser)
[2020-06-02 17:08:50,973] INFO Kafka commitId: 1d348535a0a747d1 (org.apache.kafka.common.utils.AppInfoParser)
[2020-06-02 17:08:50,976] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)

啟動成功。

7、查看

[root@orderer kafka_2.11-2.2.2]# jps
27937 QuorumPeerMain
28052 Kafka
28602 Jps
[root@orderer kafka_2.11-2.2.2]# 

默認服務端口為9092

[root@orderer kafka_2.11-2.2.2]# netstat -tunlp|grep 9092
tcp6       0      0 :::9092                 :::*                    LISTEN      28052/java          
[root@orderer kafka_2.11-2.2.2]# 

8、停止

[root@orderer kafka_2.11-2.2.2]# bin/kafka-server-stop.sh config/server.properties 

9,、創建topic

[root@orderer kafka_2.11-2.2.2]# bin/kafka-topics.sh --zookeeper localhost:2181 --create --replication-factor 1 --partitions 1 --topic demo
Created topic demo.
[root@orderer kafka_2.11-2.2.2]# 

10、查看topic列表

[root@orderer kafka_2.11-2.2.2]# bin/kafka-topics.sh --zookeeper localhost:2181 --list
demo
[root@orderer kafka_2.11-2.2.2]# 

11、查看指定topic信息

[root@orderer kafka_2.11-2.2.2]# bin/kafka-topics.sh  --zookeeper localhost:2181 --describe --topic demo
Topic:demo      PartitionCount:1        ReplicationFactor:1     Configs:
        Topic: demo     Partition: 0    Leader: 0       Replicas: 0     Isr: 0
[root@orderer kafka_2.11-2.2.2]# 

12、啟動生產者

[root@orderer kafka_2.11-2.2.2]# bin/kafka-console-producer.sh --broker-list localhost:9092 --topic demo
>

13、啟動消費者

新開一個會話窗口

[root@orderer src]# cd /usr/local/kafka_2.11-2.2.2/
[root@orderer kafka_2.11-2.2.2]# bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic demo

14、切換到生產者窗口輸入數據

[root@orderer kafka_2.11-2.2.2]# bin/kafka-console-producer.sh --broker-list localhost:9092 --topic demo
>hello!
>

切換到消費者窗口查看

[root@orderer kafka_2.11-2.2.2]# bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic demo
hello!

消費者已經消費到數據了

 15、刪除topic

[root@orderer kafka_2.11-2.2.2]# bin/kaftopics.sh --zookeeper localhost:2181 --delete --topic demo
Topic demo is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.