Zookeeper安裝篇

1.由於zookeeper的選舉機制是leader選舉，要求 可用節點數量 > 總節點數量/2 。注意 是 > , 不是 ≥。所以需要用三台虛擬機來 裝，當然內存足夠任性的話選五台也不是不行，我這里就用三台，主機名分別為hd1,hd2,hd3
2.准備好安裝包
我選用的版本是apache-zookeeper-3.5.9-bin.tar.gz，下載鏈接： https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.5.9/
在寫這篇文章時最新的版本是3.7.0，不過為了穩定考慮選用稍微老兩個版本的3.5.9

前言 zookeeper的數據目錄 是 data 且權限 755 具體查看官網的hdfs的權限管理

上傳並解壓

我一般把安裝包放在/opt/packages,安裝目錄統一在/opt/apps，方便管理

cd /opt/packages
tar -zxf apache-zookeeper-3.5.9-bin.tar.gz -C ../apps/
cd ../apps
# 創建軟鏈接
ln -s apache-zookeeper-3.5.9-bin zookeeper
cd zookeeper/conf
cp zoo_sample.cfg zoo.cfg

配置zookeeper

zoo.cfg

注意各項配置的末尾不要有空格，不然無法識別

# 修改配置文件
vim zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
# 設置數據目錄
dataDir=/data/zookeeper
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
# 第一個端口用於Follower和Leader之間的數據同步和其他通信，第二個端口用於Leader選舉過程中投票通信
server.1=hadoop102:2888:3888
server.2=hadoop103:2888:3888
server.3=hadoop104:2888:3888

zkEnv.sh

修改 zookeeper/bin/zkEnv.sh 腳本，在腳本中給 ZOO_LOG_DIR 設置日志所在目錄
后續安裝的所有軟件的數據目錄我都會設置在/data下，方便查看管理
修改后

ZOOBINDIR="${ZOOBINDIR:-/usr/bin}"
ZOOKEEPER_PREFIX="${ZOOBINDIR}/.."
ZOO_LOG_DIR=/data/zookeeper/logs
#check to see if the conf dir is given as an optional argument
if [ $# -gt 1 ]
then
    if [ "--config" = "$1" ]
          then
              shift
              confdir=$1
              shift
              ZOOCFGDIR=$confdir
    fi
fi
 
if [ "x$ZOOCFGDIR" = "x" ]
then
 

創建myid

上面在zoo.cfg中把dataDir設置為了/data/zookeeper
所以需要在這個目錄下新建myid文件用來區分三個zookeeper
注意三台的id必須不同

hadoop102:
echo "1" > /data/zookeeper/myid 
hadoop103:
echo "2" > /data/zookeeper/myid 
hadoop104:
echo "3" > /data/zookeeper/myid 

配置環境變量

vim /etc/profile.d/my_env.sh

export JAVA_HOME=/usr/java/jdk1.8.0_144
export JRE_HOME=$JAVA_HOME/jre
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

#zookeeper
ZOOKEEPER_HOME=/opt/apps/zookeeper
export PATH=$PATH:$ZOOKEEPER_HOME/bin

生效也可以采用或者關閉終端重新連接

source /etc/profile.d/my_env.sh

分發軟件包腳本

#!/bin/bash
#1. 判斷參數個數
if [ $# -lt 1 ]
then
  echo Not Enough Arguement!
  exit;
fi
#2. 遍歷集群所有機器
for host in hadoop102 hadoop103 hadoop104
do
  echo ====================  $host  ====================
  #3. 遍歷所有目錄，挨個發送
  for file in $@
  do
    #4 判斷文件是否存在
    if [ -e $file ]
    then
      #5. 獲取父目錄
      pdir=$(cd -P $(dirname $file); pwd)
      #6. 獲取當前文件的名稱
      fname=$(basename $file)
      ssh $host "mkdir -p $pdir"
      rsync -av $pdir/$fname $host:$pdir
    else
      echo $file does not exists!
    fi
  done
done

啟動zookeeper

為了方便，這里同樣准備多機操作腳本

cd ~
mkdir zk
cd zk
touch ips
vim ips
 
hadoop102
hadoop103
hadoop104
:wq
# 創建多機操作腳本
vim ssh_all.sh
#!/bin/bash
#獲得執行的文件所在的目錄
 
cd `dirname $0`
#查看當前所在的工作目錄
path=`pwd`
#使用cat命令查看ips文件內容，獲得5台機器的主機名稱，是字符串形式
#轉數組
ips=(`cat ${path}/ips`)
 
for i in ${ips[*]};
do
        #拼接命令 使用位置變量$*接收執行的linux命令
	#這里要寫你安裝軟件用的用戶名，我用的是hadoop
        cmd="ssh hadoop@${i} \"source /etc/profile;source ~/.bash_profile;$*\""
        echo $cmd
        if eval $cmd;then
                echo "OK"
        else
                echo "FAIL"
        fi
done;
:wq
chmod +x ssh_root.sh
# 寫好腳本后在hd1上執行
[hadoop@hadoop102 zk]$ sh ~/zk/ssh_all.sh zkServer.sh start
ssh hadoop@hadoop102 "source /etc/profile;source ~/.bash_profile;zkServer.sh start"
/usr/local/jdk/bin/java
ZooKeeper JMX enabled by default
Using config: /opt/apps/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
OK
ssh hadoop@hadoop103 "source /etc/profile;source ~/.bash_profile;zkServer.sh start"
/usr/local/jdk/bin/java
ZooKeeper JMX enabled by default
Using config: /opt/apps/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
OK
ssh hadoop@hadoop104 "source /etc/profile;source ~/.bash_profile;zkServer.sh start"
/usr/local/jdk/bin/java
ZooKeeper JMX enabled by default
Using config: /opt/apps/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
OK
# 進入客戶端
[hadoop@hadoop102 zk]$ zkCli.sh -server hadoop103:2181,hadoop103:2181,hadoop104:2181
WATCHER::
 
WatchedEvent state:SyncConnected type:None path:null
[zk: hadoop102:2181.hadoop103:2181,hadoop104:2181(CONNECTED) 0] ls /
[zookeeper]
[zk: hadoop102:2181.hadoop103:2181,hadoop104:2181(CONNECTED) 1]