Zookeeper與HBase的安裝


  一、Zookeeper的安裝

  1.http://www-us.apache.org/dist/zookeeper/stable/下載Zookeeper安裝包,並將zookeeper-3.4.12.tar.gz復制到/home/jun下並解壓

[jun@master ~]$ cd zookeeper-3.4.12/
[jun@master zookeeper-3.4.12]$ ls -l
total 1604
drwxr-xr-x.  2 jun jun     149 Mar 27 12:32 bin
-rw-rw-r--.  1 jun jun   87945 Mar 27 12:32 build.xml
drwxr-xr-x.  2 jun jun      77 Mar 27 12:32 conf
drwxr-xr-x. 10 jun jun     130 Mar 27 12:32 contrib
drwxr-xr-x.  2 jun jun    4096 Mar 27 12:37 dist-maven
drwxr-xr-x.  6 jun jun    4096 Mar 27 12:32 docs
-rw-rw-r--.  1 jun jun    1709 Mar 27 12:32 ivysettings.xml
-rw-rw-r--.  1 jun jun    8197 Mar 27 12:32 ivy.xml
drwxr-xr-x.  4 jun jun    4096 Mar 27 12:32 lib
-rw-rw-r--.  1 jun jun   11938 Mar 27 12:32 LICENSE.txt
-rw-rw-r--.  1 jun jun    3132 Mar 27 12:32 NOTICE.txt
-rw-rw-r--.  1 jun jun    1585 Mar 27 12:32 README.md
-rw-rw-r--.  1 jun jun    1770 Mar 27 12:32 README_packaging.txt
drwxr-xr-x.  5 jun jun      47 Mar 27 12:32 recipes
drwxr-xr-x.  8 jun jun     211 Mar 27 12:32 src
-rw-rw-r--.  1 jun jun 1483366 Mar 27 12:32 zookeeper-3.4.12.jar
-rw-rw-r--.  1 jun jun     819 Mar 27 12:36 zookeeper-3.4.12.jar.asc
-rw-rw-r--.  1 jun jun      33 Mar 27 12:32 zookeeper-3.4.12.jar.md5
-rw-rw-r--.  1 jun jun      41 Mar 27 12:32 zookeeper-3.4.12.jar.sha1

  2.使用gedit編輯zoo.cfg配置文件

[jun@master zookeeper-3.4.12]$ cp /home/jun/zookeeper-3.4.12/conf/zoo_sample.cfg  /home/jun/zookeeper-3.4.12/conf/zoo.cfg
[jun@master zookeeper-3.4.12]$ gedit /home/jun/zookeeper-3.4.12/conf/zoo.cfg 

  將下面的配置增加到最后面,配置的意思是“服務器編號、服務器地址、LF通信端口(Leader與Follower)、選舉端口”

server.1=master:2888:3888
server.2=slave0:2888:3888
server.3=slave1:2888:3888

  3.創建節點標志文件  

  由於在zoo.cfg當中配置了三台服務器,每一台服務器本地也需要一個myid文件,里面包含服務器的ID號,需要在三台機器上都創建myid文件。以master為例:

  首先遞歸創建目錄(-p)

[jun@master zookeeper-3.4.12]$ mkdir -p /tmp/zookeeper
[jun@master zookeeper-3.4.12]$ gedit /tmp/zookeeper/myid

  然后新建myid文件並輸入對應的id號,只輸入一個1即可。

[jun@master zookeeper-3.4.12]$ cat /tmp/zookeeper/myid 
1

  另外兩台機器同理。

  4.將Zookeeper安裝文件從master復制到slave上(-r的意思是拷貝所有的文件和文件夾)

[jun@master zookeeper-3.4.12]$ scp -r /home/jun/zookeeper-3.4.12/   slave0:~
[jun@master zookeeper-3.4.12]$ scp -r /home/jun/zookeeper-3.4.12/   slave1:~

  5.Zookeeper集群的啟動

  (1)需要在三個節點上分別執行啟動操作

[jun@master zookeeper-3.4.12]$ /home/jun/zookeeper-3.4.12/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/jun/zookeeper-3.4.12/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

[jun@slave0 ~]$ /home/jun/zookeeper-3.4.12/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/jun/zookeeper-3.4.12/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

[jun@slave1 ~]$ /home/jun/zookeeper-3.4.12/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/jun/zookeeper-3.4.12/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

  (2)至少啟動兩台服務器后,集群才會開始選舉leader節點,可以在三台服務器上分別查看集群的狀態,可以看到slave0節點是leader節點。

[jun@master ~]$ /home/jun/zookeeper-3.4.12/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/jun/zookeeper-3.4.12/bin/../conf/zoo.cfg
Mode: follower

[jun@slave0 ~]$ /home/jun/zookeeper-3.4.12/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/jun/zookeeper-3.4.12/bin/../conf/zoo.cfg
Mode: leader

[jun@slave1 ~]$ /home/jun/zookeeper-3.4.12/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/jun/zookeeper-3.4.12/bin/../conf/zoo.cfg
Mode: follower

  (3)Zookeeper客戶端交互式操作

[jun@master ~]$ /home/jun/zookeeper-3.4.12/bin/zkCli.sh -server master:2181
Connecting to master:2181
2018-07-23 14:06:51,047 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.12-e5259e437540f349646870ea94dc2658c4e44b3b, built on 03/27/2018 03:55 GMT
2018-07-23 14:06:51,049 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=master
2018-07-23 14:06:51,050 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_171
2018-07-23 14:06:51,051 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2018-07-23 14:06:51,051 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/java/jdk1.8.0_171/jre
2018-07-23 14:06:51,051 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/home/jun/zookeeper-3.4.12/bin/../build/classes:/home/jun/zookeeper-3.4.12/bin/../build/lib/*.jar:/home/jun/zookeeper-3.4.12/bin/../lib/slf4j-log4j12-1.7.25.jar:/home/jun/zookeeper-3.4.12/bin/../lib/slf4j-api-1.7.25.jar:/home/jun/zookeeper-3.4.12/bin/../lib/netty-3.10.6.Final.jar:/home/jun/zookeeper-3.4.12/bin/../lib/log4j-1.2.17.jar:/home/jun/zookeeper-3.4.12/bin/../lib/jline-0.9.94.jar:/home/jun/zookeeper-3.4.12/bin/../lib/audience-annotations-0.5.0.jar:/home/jun/zookeeper-3.4.12/bin/../zookeeper-3.4.12.jar:/home/jun/zookeeper-3.4.12/bin/../src/java/lib/*.jar:/home/jun/zookeeper-3.4.12/bin/../conf:
2018-07-23 14:06:51,051 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2018-07-23 14:06:51,051 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2018-07-23 14:06:51,051 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2018-07-23 14:06:51,052 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2018-07-23 14:06:51,052 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2018-07-23 14:06:51,052 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=3.10.0-862.el7.x86_64
2018-07-23 14:06:51,052 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=jun
2018-07-23 14:06:51,052 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/home/jun
2018-07-23 14:06:51,052 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/home/jun
2018-07-23 14:06:51,053 [myid:] - INFO  [main:ZooKeeper@441] - Initiating client connection, connectString=master:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@799f7e29
Welcome to ZooKeeper!
2018-07-23 14:06:51,111 [myid:] - INFO  [main-SendThread(master:2181):ClientCnxn$SendThread@1028] - Opening socket connection to server master/192.168.1.100:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2018-07-23 14:06:51,191 [myid:] - INFO  [main-SendThread(master:2181):ClientCnxn$SendThread@878] - Socket connection established to master/192.168.1.100:2181, initiating session
[zk: master:2181(CONNECTING) 0] 2018-07-23 14:06:51,269 [myid:] - INFO  [main-SendThread(master:2181):ClientCnxn$SendThread@1302] - Session establishment complete on server master/192.168.1.100:2181, sessionid = 0x1000005d8900000, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null

[zk: master:2181(CONNECTED) 0] help
ZooKeeper -server host:port cmd args
    stat path [watch]
    set path data [version]
    ls path [watch]
    delquota [-n|-b] path
    ls2 path [watch]
    setAcl path acl
    setquota -n|-b val path
    history 
    redo cmdno
    printwatches on|off
    delete path [version]
    sync path
    listquota path
    rmr path
    get path [watch]
    create [-s] [-e] path data acl
    addauth scheme auth
    quit 
    getAcl path
    close 
    connect host:port
[zk: master:2181(CONNECTED) 1] quit
Quitting...
2018-07-23 14:07:36,575 [myid:] - INFO  [main:ZooKeeper@687] - Session: 0x1000005d8900000 closed
2018-07-23 14:07:36,577 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@521] - EventThread shut down for session: 0x1000005d8900000

  

  二、HBase的安裝

  1.從http://apache.fayea.com/hbase/stable/下載hbase-1.2.6.1-bin.tar.gz並賦值到/home/jun下並解壓。、

[jun@master ~]$ cd hbase-1.2.6.1/
[jun@master hbase-1.2.6.1]$ ls -l
total 340
drwxr-xr-x.  4 jun jun   4096 Jun  4 09:43 bin
-rw-r--r--.  1 jun jun 129753 Jun  4 09:43 CHANGES.txt
drwxr-xr-x.  2 jun jun    178 Jun  4 09:43 conf
drwxr-xr-x. 12 jun jun   4096 Jun  4 12:34 docs
drwxr-xr-x.  7 jun jun     80 Jun  4 12:19 hbase-webapps
-rw-rw-r--.  1 jun jun    261 Jun  4 12:36 LEGAL
drwxrwxr-x.  3 jun jun   8192 Jul 23 14:11 lib
-rw-rw-r--.  1 jun jun 143082 Jun  4 12:36 LICENSE.txt
-rw-rw-r--.  1 jun jun  42141 Jun  4 12:36 NOTICE.txt
-rw-r--r--.  1 jun jun   1477 Jun  4 09:43 README.txt

  2.修改conf文件夾下的環境變量hbase-env.sh

# The java implementation to use.  Java 1.7+ required.
export JAVA_HOME=/usr/java/jdk1.8.0_171/

  由於使用的jdk1.8,因此還需要將下面兩行注釋掉:

# Configure PermSize. Only needed in JDK7. You can safely remove it for JDK8+
#export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -XX:PermSize=128m -XX:MaxPermSize=128m"
#export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -XX:PermSize=128m -XX:MaxPermSize=128m"

  3.修改conf文件夾下的配置文件hbase-site.xml

<configuration>

    <property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>
    </property>

    <property>
        <name>hbase.rootdir</name>
        <value>hdfs://master:9000/hbase</value>
    </property>

    <property>
        <name>hbase.zookeeper.quorum</name>
        <value>master</value>
    </property>

    <property>
        <name>hbase.master.info.port</name>
        <value>60010</value>
    </property>

</configuration>

  4.設置conf文件夾下的regionservers

  regionservers文件類似Hadoop的slaves文件,其中保存了rigionserver的列表,啟動HBase的時候,系統將根據該文件建立HBase集群。需要先刪除localhost並添加兩條配置

slave0
slave1

  5.設置linux環境變量並source生效

#hbase
export HBASE_HOME=/home/jun/hbase-1.2.6.1/
export PATH=$HBASE_HOME/bin:$PATH
export HADOOP_CLASSPATH=$HBASE_HOME/lib/*

  6.將HBase安裝文件復制到slave節點

[jun@master conf]$ scp -r /home/jun/hbase-1.2.6.1/  slave0:~/
[jun@master conf]$ scp -r /home/jun/hbase-1.2.6.1/  slave1:~/

  7.啟動並驗證HBase

  (1)執行start-hbase.sh啟動HBase,可以看到,系統首先啟動HBase自帶的ZooKeeper,然后啟動HBase的HMaster,接着分別啟動slave節點上的regionservers。

[jun@master conf]$ start-hbase.sh 
master: starting zookeeper, logging to /home/jun/hbase-1.2.6.1/bin/../logs/hbase-jun-zookeeper-master.out
starting master, logging to /home/jun/hbase-1.2.6.1//logs/hbase-jun-master-master.out
slave0: starting regionserver, logging to /home/jun/hbase-1.2.6.1/bin/../logs/hbase-jun-regionserver-slave0.out
slave1: starting regionserver, logging to /home/jun/hbase-1.2.6.1/bin/../logs/hbase-jun-regionserver-slave1.out

  (2)查看各結點的jps,其中HMaster是HBase的主控節點進程,HQuorumPeer是HBase的ZooKeeper進程(即HBase內置的ZooKeeper)。

[jun@master ~]$ jps
6273 Jps
3587 SecondaryNameNode
3796 ResourceManager
5989 HMaster
5912 HQuorumPeer
3357 NameNode

[jun@slave0 ~]$ jps
3441 NodeManager
4177 HRegionServer
3621 QuorumPeerMain
3271 DataNode
4375 Jps

[jun@slave1 ~]$ jps
3232 DataNode
4370 Jps
3606 QuorumPeerMain
4169 HRegionServer
3407 NodeManager

  (3)使用Web方式查看HBase系統的運行狀態,訪問http://master:60010

 

  三、HBase的Shell命令

  執行hbase shell即可進入HBase Shell

  1.創建表,其中第一個參數是表名,f1和f12是列名

hbase(main):001:0> create "test1", {NAME => 'f1', VERSION => 5}
Unknown argument ignored for column family f1: 1.8.7
0 row(s) in 2.6300 seconds

=> Hbase::Table - test1
hbase(main):002:0> create 'test2', 'f12'
0 row(s) in 2.2360 seconds

=> Hbase::Table - test2

  2.查看列表

hbase(main):003:0> list
TABLE                                                                                                                                             
test1                                                                                                                                             
test2                                                                                                                                             
2 row(s) in 0.0270 seconds

=> ["test1", "test2"]

  3.插入數據,其中test1是表名,id001是行鍵(row key),f1是列族名,uid是屬性名,001是屬性值

hbase(main):004:0> put 'test1', 'id001', 'f1:uid', '001'
0 row(s) in 0.2300 seconds

  4.掃描查詢數據

hbase(main):005:0> scan 'test1'
ROW                                   COLUMN+CELL                                                                                                 
 id001                                column=f1:uid, timestamp=1532330789700, value=001                                                           
1 row(s) in 0.0750 seconds

  5.查看表結構

hbase(main):007:0> describe 'test1'
Table test1 is ENABLED                                                                                                                            
test1                                                                                                                                             
COLUMN FAMILIES DESCRIPTION                                                                                                                       
{NAME => 'f1', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => '
FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}                       
1 row(s) in 0.0310 seconds

hbase(main):008:0> describe 'test2'
Table test2 is ENABLED                                                                                                                            
test2                                                                                                                                             
COLUMN FAMILIES DESCRIPTION                                                                                                                       
{NAME => 'f12', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 
'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}                      
1 row(s) in 0.0140 seconds

  6.刪除表

hbase(main):009:0> list
TABLE                                                                                                                                             
test1                                                                                                                                             
test2                                                                                                                                             
2 row(s) in 0.0100 seconds

=> ["test1", "test2"]
hbase(main):010:0> disable 'test2'
0 row(s) in 2.2890 seconds

hbase(main):011:0> drop 'test2'
0 row(s) in 1.2660 seconds

hbase(main):012:0> list
TABLE                                                                                                                                             
test1                                                                                                                                             
1 row(s) in 0.0110 seconds

=> ["test1"]

  7.退出HBase Shell

hbase(main):013:0> exit

 

  四、HBase和ZooKeeper的關系

  如果用戶在使用HBase自帶的ZooKeeper的時候出現異常,導致HBase不能正常工作,這時可以停止使用自帶的Zookeeper,轉而啟動獨立安裝的ZooKeeper,並重啟HBase,可以快速解決上述問題。

  要想讓HBase使用獨立安裝的Zookeeper,需要對HBase進行一些配置上的修改。

  1.修改conf/hbase-env.sh文件,添加如下代碼(true表示使用自帶的ZooKeeper)

export HBASE_MANAGES_ZK=false

  2.將Zookeeper配置文件復制到HBase的CLASSPATH

cp /home/jun/zookeeper-3.4.12/conf/zoo.cfg  /home/jun/hbase-1.2.6.1/conf

  3.重啟HBase即可

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM