運維需求見表：
| 主機IP | 部署服務 | 部署目錄 | 說明 |
| 192.168.21.66 | Ansible | inventory:/etc/ansible/hosts roles:/opt/ansible |
見我的Ansible系列的其他章節，此處不贅述安裝Ansible和Ansible安裝JDK |
| 192.168.21.67 | Zookeeper,Kafka | /opt/zookeeper-1/{data,logs,inuse} /opt/kafka-1/{inuse,log} |
其中/opt/zookeeper-{1,2,3}/inuse為zookeeper-3.4.10目錄的軟鏈, /opt/kakfa-{1,2,3}/inuse為kafka_2.11-1.0.0目錄的軟鏈。 zookeeper的端口為2181,313326,313327。 kafka的端口為9092。 運行用戶為：jiangwei。
|
| 192.168.21.68 | Zookeeper,Kafka | /opt/zookeeper-2/{data,logs,inuse} /opt/kafka-1/{inuse,log} |
|
| 192.168.21.69 | Zookeeper,Kafka | /opt/zookeeper-3/{data,logs,inuse} /opt/kafka-1/{inuse,log} |
1.關聯文檔：
- 見我的Ansible學習系列之安裝 Ansible
- 見我的Ansible學習系列之Ansible 安裝 JDK
2.前提是：部署服務器的jiangwei用戶安裝了JDK軟件
Ansible 安裝 Zookeeper 集群
3.vi /etc/ansible/hosts，內容如下：
[zookeeper1]
192.168.21.67
[zookeeper2]
192.168.21.68
[zookeeper3]
192.168.21.69
[zookeeper:children]
zookeeper1
zookeeper2
zookeeper3
[zookeeper:vars]
ansible_python_interpreter=/usr/bin/python3
ansible_ssh_user=root
[kafka]
192.168.21.67
192.168.21.68
192.168.21.69
[kafka:vars]
ansible_python_interpreter=/usr/bin/python3
ansible_ssh_user=root
4./opt/ansible/zookeeper目錄結構：
zookeeper
├── files
│ ├── myid1
│ ├── myid2
│ ├── myid3
│ ├── zoo1.cfg
│ ├── zoo2.cfg
│ ├── zoo3.cfg
│ └── zookeeper-3.4.10.tar.gz
└── tasks
├── main.yml
├── zk1.yml
├── zk2.yml
└── zk3.yml
其中zookeeper-3.4.10.tar.gz為安裝包。
4.1）myid1、myid2和myid3的文件內容分別為1、2、3。
4.2）zoo1.cfg內容如下：
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/opt/zookeeper-1/data
dataLogDir=/opt/zookeeper-1/logs
clientPort=2181
server.1=192.168.21.67:31326:31327
server.2=192.168.21.68:31326:31327
server.3=192.168.21.69:31326:31327
備注：zoo2.cfg、zoo3.cfg同理，只需改變dataDir和dataLogDir即可，參考對應的運維需求表的部署目錄。
4.3）zk1.yml內容如下：
- name: Create directory /opt/zookeeper-1
file: path=/opt/zookeeper-1 state=directory owner=jiangwei group=jiangwei
when: ansible_default_ipv4.address == "192.168.21.67"
- name: Unarchive Zookeeper Software
unarchive: src=zookeeper-3.4.10.tar.gz dest=/opt/zookeeper-1 owner=jiangwei group=jiangwei
when: ansible_default_ipv4.address == "192.168.21.67"
- name: copy zoo.cfg
copy: src=zoo1.cfg dest=/opt/zookeeper-1/zookeeper-3.4.10/conf/zoo.cfg owner=jiangwei group=jiangwei
when: ansible_default_ipv4.address == "192.168.21.67"
- name: create directory /opt/zookeeper-1/data
file: path=/opt/zookeeper-1/data state=directory owner=jiangwei group=jiangwei
when: ansible_default_ipv4.address == "192.168.21.67"
- name: create directory /opt/zookeeper-1/logs
file: path=/opt/zookeeper-1/logs state=directory owner=jiangwei group=jiangwei
when: ansible_default_ipv4.address == "192.168.21.67"
- name: set myid
copy: src=myid1 dest=/opt/zookeeper-1/data/myid owner=jiangwei group=jiangwei
when: ansible_default_ipv4.address == "192.168.21.67"
- name: make link
file: path=/opt/zookeeper-1/inuse src=/opt/zookeeper-1/zookeeper-3.4.10 state=link owner=jiangwei group=jiangwei
when: ansible_default_ipv4.address == "192.168.21.67"
- name: start zookeeper
command: sudo su - jiangwei -c "/opt/zookeeper-1/inuse/bin/zkServer.sh start" /bin/bash
when: ansible_default_ipv4.address == "192.168.21.67"
備注：zk2.yml和zk3.yml同理，目錄按運維需求而定。main.yml包含zk1.yml、zk2.yml、zk3.yml
5.zkinstall.yml內容如下：
- hosts: zookeeper
roles:
- zookeeper
6.執行：ansible-playbook zkinstall.yml即可完成zookeeper集群安裝
Ansible 安裝 kafka 集群
7.vi /etc/ansible/hosts，內容如下：
[kafka]
192.168.21.67
192.168.21.68
192.168.21.69
[kafka:vars]
ansible_python_interpreter=/usr/bin/python3
ansible_ssh_user=root
8./opt/ansible/kafka目錄結構如下：
kafka
├── files
│ ├── kafka_2.11-1.0.0.tgz
│ ├── server1.properties
│ ├── server2.properties
│ └── server3.properties
└── tasks
├── kf1.yml
├── kf2.yml
├── kf3.yml
└── main.yml
其中kafka_2.11-1.0.0.tgz為安裝包。
8.1）server1.properties內容如下：
broker.id=67
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
replica.fetch.max.bytes=5048576
message.max.byte=5048576
default.replication.factor=1
host.name=192.168.21.67
port=9092
zookeeper.connect=192.168.21.67:2181,192.168.21.68:2181,192.168.21.69:2181
log.dirs=/opt/kafka-1/log
num.partitions=2
備注：server2.properties和server3.properties同理，broker.id、log.dirs按照運維需求而來。
8.2）kf1.yml內容如下：
- name: Create directory /opt/kafka-1
file: path=/opt/kafka-1 state=directory owner=jiangwei group=jiangwei
when: ansible_default_ipv4.address == "192.168.21.67"
- name: Unarchive Kafka Software
unarchive: src=kafka_2.11-1.0.0.tgz dest=/opt/kafka-1 owner=jiangwei group=jiangwei
when: ansible_default_ipv4.address == "192.168.21.67"
- name: make link
file: path=/opt/kafka-1/inuse src=/opt/kafka-1/kafka_2.11-1.0.0 state=link owner=jiangwei group=jiangwei
when: ansible_default_ipv4.address == "192.168.21.67"
- name: copy kafka
copy: src=server1.properties dest=/opt/kafka-1/inuse/config/server.properties owner=jiangwei group=jiangwei
when: ansible_default_ipv4.address == "192.168.21.67"
- name: start kafka-1
command: sudo su - jiangwei -c "/opt/kafka-1/inuse/bin/kafka-server-start.sh -daemon /opt/kafka-1/inuse/config/server.properties"
when: ansible_default_ipv4.address == "192.168.21.67"
備注：kf2.yml和kf3.yml同理，注意目錄和配置文件的對應關系。main.yml包含kf1.yml、kf2.yml、kf3.yml。
9.kafkainstall.yml內容如下：
- hosts: kafka
roles:
- kafka
10.執行：ansible-playbook kakfkainstall.yml 完成kafka集群部署
驗證
11.在192.168.21.67上發送消息，執行命令：
/opt/kafka-1/inuse/bin/kafka-console-producer.sh --broker-list 192.168.21.67:9092, 192.168.21.68:9092, 192.168.21.69:9092 --topic test
12.在192.168.21.68上接受消息，執行命令：
/opt/kafka-1/inuse/bin/kafka-console-consumer.sh --zookeeper 192.168.21.67:2181, 192.168.21.68:2181, 192.168.21.69:2181 --topic test –from-beginning