碼上快樂

相關內容    簡體    繁體

zookeeper偽分布式安裝

本文轉載自   查看原文   2018-01-05 21:22   1708    分布式

本文介紹zookeeper偽分布式安裝。

所謂 “偽分布式集群” 就是在1台PC中啟動多個zookeeper的實例。“完全分布式集群” 是每1台PC啟動1個ZooKeeper實例。

由於我的測試環境PC數量有限，所以在1台PC中啟動3個zookeeper的實例。下面以3個節點為例，介紹安裝的過程

 

新建3個文件夾

[work@xxx zookeeper-3.4.6]$ mkdir zoo_1 zoo_2 zoo_3

 

復制配置文件

[work@xxx conf]$ cp zoo.cfg zoo1.cfg
[work@xxx conf]$ cp zoo.cfg zoo2.cfg  
[work@xxx conf]$ cp zoo.cfg zoo3.cfg

 

修改配置文件

3個節點的配置文件分別如下，注意修改路徑和端口

介紹一下配置文件里面的字段含義：

1、tickTime：這個時間是作為 Zookeeper 服務器之間或客戶端與服務器之間維持心跳的時間間隔，也就是每個 tickTime 時間就會發送一個心跳

2、dataDir：顧名思義就是 Zookeeper 保存數據的目錄，默認情況下，Zookeeper 將寫數據的日志文件也保存在這個目錄里

3、clientPort：這個端口就是客戶端連接 Zookeeper 服務器的端口，Zookeeper 會監聽這個端口，接受客戶端的訪問請求

4、initLimit：這個配置項是用來配置 Zookeeper 接受客戶端（這里所說的客戶端不是用戶連接 Zookeeper 服務器的客戶端，而是 Zookeeper 服務器集群中連接到 Leader 的 Follower 服務器）初始化連接時最長能忍受多少個心跳時間間隔數。當已經超過 5個心跳的時間（也就是 tickTime）長度后 Zookeeper 服務器還沒有收到客戶端的返回信息，那么表明這個客戶端連接失敗。總的時間長度就是 5 * 2000 = 10 秒

5、syncLimit：這個配置項標識 Leader 與 Follower 之間發送消息，請求和應答時間長度，最長不能超過多少個 tickTime 的時間長度，總的時間長度就是 2 * 2000 = 4 秒

6、server.A=B：C：D：其中 A 是一個數字，表示這個是第幾號服務器；B 是這個服務器的 ip 地址；C 表示的是這個服務器與集群中的 Leader 服務器交換信息的端口；D 表示的是萬一集群中的 Leader 服務器掛了，需要一個端口來重新進行選舉，選出一個新的 Leader，而這個端口就是用來執行選舉時服務器相互通信的端口。如果是偽集群的配置方式，由於 B 都是一樣，所以不同的 Zookeeper 實例通信端口號不能一樣，所以要給它們分配不同的端口號

 

寫入myid

[work@xxx zookeeper-3.4.6]$ echo "1" > zoo_1/myid
[work@xxx zookeeper-3.4.6]$ echo "2" > zoo_2/myid   
[work@xxx zookeeper-3.4.6]$ echo "3" > zoo_3/myid

 

啟動

[work@xxx zookeeper-3.4.6]$ bin/zkServer.sh start zoo1.cfg
JMX enabled by default
Using config: /home/work/soft/zookeeper-3.4.6/bin/../conf/zoo1.cfg
Starting zookeeper ... STARTED


[work@xxx zookeeper-3.4.6]$ bin/zkServer.sh start zoo2.cfg
JMX enabled by default
Using config: /home/work/soft/zookeeper-3.4.6/bin/../conf/zoo2.cfg
Starting zookeeper ... STARTED

[work@xxx zookeeper-3.4.6]$ bin/zkServer.sh start zoo3.cfg
JMX enabled by default
Using config: /home/work/soft/zookeeper-3.4.6/bin/../conf/zoo3.cfg
Starting zookeeper ... STARTED

 

查看狀態
[work@xxx zookeeper-3.4.6]$ bin/zkServer.sh status zoo1.cfg   
JMX enabled by default
Using config: /home/work/soft/zookeeper-3.4.6/bin/../conf/zoo1.cfg
Mode: follower

[work@xxx zookeeper-3.4.6]$ bin/zkServer.sh status zoo2.cfg
JMX enabled by default
Using config: /home/work/soft/zookeeper-3.4.6/bin/../conf/zoo2.cfg
Mode: leader

[work@xxx zookeeper-3.4.6]$ bin/zkServer.sh status zoo3.cfg
JMX enabled by default
Using config: /home/work/soft/zookeeper-3.4.6/bin/../conf/zoo3.cfg
Mode: follower

可以發現zoo_2被選為了leader

 

查看目錄

[work@xxx zookeeper-3.4.6]$ tree zoo_1   
zoo_1
|-- myid
|-- version-2
|   |-- acceptedEpoch
|   |-- currentEpoch
|   `-- snapshot.0
`-- zookeeper_server.pid

1 directory, 5 files

[work@xxx zookeeper-3.4.6]$ tree zoo_2
zoo_2
|-- myid
|-- version-2
|   |-- acceptedEpoch
|   `-- currentEpoch
`-- zookeeper_server.pid

1 directory, 4 files

[work@xxx zookeeper-3.4.6]$ tree zoo_3
zoo_3
|-- myid
|-- version-2
|   |-- acceptedEpoch
|   |-- currentEpoch
|   `-- snapshot.100000000
`-- zookeeper_server.pid

1 directory, 5 files
[work@xxx zookeeper-3.4.6]$

 

安裝成功后，用cli測試一下，一直報下面這個SB錯誤！

[work@xxx zookeeper-3.4.6]$ bin/zkCli.sh -server 127.0.0.1:2182
Connecting to 127.0.0.1:2182
2018-01-05 21:32:45,482 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2018-01-05 21:32:45,485 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=xxx
2018-01-05 21:32:45,486 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.7.0_111
2018-01-05 21:32:45,488 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2018-01-05 21:32:45,488 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64/jre
2018-01-05 21:32:45,488 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/home/work/soft/zookeeper-3.4.6/bin/../build/classes:/home/work/soft/zookeeper-3.4.6/bin/../build/lib/*.jar:/home/work/soft/zookeeper-3.4.6/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/work/soft/zookeeper-3.4.6/bin/../lib/slf4j-api-1.6.1.jar:/home/work/soft/zookeeper-3.4.6/bin/../lib/netty-3.7.0.Final.jar:/home/work/soft/zookeeper-3.4.6/bin/../lib/log4j-1.2.16.jar:/home/work/soft/zookeeper-3.4.6/bin/../lib/jline-0.9.94.jar:/home/work/soft/zookeeper-3.4.6/bin/../zookeeper-3.4.6.jar:/home/work/soft/zookeeper-3.4.6/bin/../src/java/lib/*.jar:/home/work/soft/zookeeper-3.4.6/bin/../conf:
2018-01-05 21:32:45,488 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2018-01-05 21:32:45,488 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2018-01-05 21:32:45,488 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2018-01-05 21:32:45,488 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2018-01-05 21:32:45,489 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2018-01-05 21:32:45,489 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=3.10.0_1-0-0-8
2018-01-05 21:32:45,489 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=work
2018-01-05 21:32:45,489 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/home/work
2018-01-05 21:32:45,489 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/home/work/soft/zookeeper-3.4.6
2018-01-05 21:32:45,490 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=127.0.0.1:2182 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@7a04c4aa
Welcome to ZooKeeper!
2018-01-05 21:32:45,519 [myid:] - INFO  [main-SendThread(127.0.0.1:2182):ClientCnxn$SendThread@975] - Opening socket connection to server 127.0.0.1/127.0.0.1:2182. Will not attempt to authenticate using SASL (unknown error)
2018-01-05 21:32:45,525 [myid:] - WARN  [main-SendThread(127.0.0.1:2182):ClientCnxn$SendThread@1102] - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
JLine support is enabled
[zk: 127.0.0.1:2182(CONNECTING) 0] 2018-01-05 21:32:46,629 [myid:] - INFO  [main-SendThread(127.0.0.1:2182):ClientCnxn$SendThread@975] - Opening socket connection to server 127.0.0.1/127.0.0.1:2182. Will not attempt to authenticate using SASL (unknown error)
2018-01-05 21:32:46,630 [myid:] - WARN  [main-SendThread(127.0.0.1:2182):ClientCnxn$SendThread@1102] - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2018-01-05 21:32:47,731 [myid:] - INFO  [main-SendThread(127.0.0.1:2182):ClientCnxn$SendThread@975] - Opening socket connection to server 127.0.0.1/127.0.0.1:2182. Will not attempt to authenticate using SASL (unknown error)
2018-01-05 21:32:47,731 [myid:] - WARN  [main-SendThread(127.0.0.1:2182):ClientCnxn$SendThread@1102] - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)

 

在網上查了很久也沒有找到解決方法，但是用cli連接單節點的zookeeper是沒有問題的，奇怪！

參考了http://blog.itpub.net/12219480/viewspace-1778740/ 里面的說明，不過沒有驗證，先貼到下面：

1、zookeeper配置完畢之候報錯，信息如下：

  1. grid@master1:~$ zkCli.sh -server 10.10.10.70:2181
  2. ..............................
  3. 2015-08-21 06:29:48,412 [myid:] - INFO [main-SendThread(10.10.10.70:2181):ClientCnxn$SendThread@975] - Opening socket connection to server 10.10.10.70/10.10.10.70:2181. Will not attempt to authenticate using SASL (unknown error)
  4. Welcome to
  5. 2015-08-21 06:29:48,420 [myid:] - WARN [main-SendThread(10.10.10.70:2181):ClientCnxn$SendThread@1102] - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
  6. java.net.ConnectException: Connection refused
  7.         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  8.         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
  9.         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
  10.         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)

這是連接zookeeper的server一直不斷的報錯
2、查看zookeeper配置信息

  1. grid@master1:/usr/local/zookeeper/conf$ cat zoo.cfg
  2. # The number of milliseconds of each tick
  3. tickTime=2000
  4. # The number of ticks that the initial
  5. # synchronization phase can take
  6. initLimit=10
  7. # The number of ticks that can pass between
  8. # sending a request and getting an acknowledgement
  9. syncLimit=5
  10. # the directory where the snapshot is stored.
  11. # do not use /tmp for storage, /tmp here is just
  12. # example sakes.
  13. dataDir=/data/zk
  14. dataLogDir=/data/zk/logs
  15. # the port at which the clients will connect
  16. clientPort=2181
  17. # the maximum number of client connections.
  18. # increase this if you need to handle more clients
  19. #maxClientCnxns=60
  20. #
  21. # Be sure to read the maintenance section of the
  22. # administrator guide before turning on autopurge.
  23. #
  24. # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
  25. #
  26. # The number of snapshots to retain in dataDir
  27. #autopurge.snapRetainCount=3
  28. # Purge task interval in hours
  29. # Set to "0" to disable auto purge feature
  30. #autopurge.purgeInterval=1

  32. server.70=10.10.10.70:2888:3888
  33. server.71=10.10.10.71:2888:3888
  34. server.72=10.10.10.72:2888:3888
  35. server.73=10.10.10.73:2888:3888

2、在每個服務器上更改myid文件，如10.10.10.70文件

70

之后啟動zookeeper，服務正常

 

 

本文參考自：

https://www.cnblogs.com/haippy/archive/2012/07/19/2599989.html

http://blog.csdn.net/chengwenyao18/article/details/23611995

https://www.cnblogs.com/driftingshine/p/6649104.html

http://blog.itpub.net/12219480/viewspace-1778740/

 


免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。



猜您在找 zookeeper 偽分布式安裝 cdh版本的zookeeper安裝以及配置（偽分布式模式） Zookeeper 初體驗之——偽分布式安裝 Zookeeper安裝（本地，偽分布式，集群） zookeeper偽分布式集群搭建 Zookeeper 偽分布式部署 Macbook Hbase(1.2.6) 偽分布式安裝，Hadoop(2.8.2) ，使用自帶zookeeper Hbase 偽分布式模式安裝 在mac上安裝hadoop偽分布式 hbase的偽分布式安裝
 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM