Hbase啟動時報錯ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts

剛安裝時嘗試啟動了HBASE是沒問題的，相關配置詳見：https://blog.csdn.net/ls_6468/article/details/79637680?tdsourcetag=s_pcqq_aiomsg

第二次啟動出現問題。

2019-04-08 11:31:38,638 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts
2019-04-08 11:31:38,639 WARN  [main] zookeeper.ZKUtil: hconnection-0x71e35c40x0, quorum=slave:2181,master:2181, baseZNode=/hbase Unable to set watcher on znode (/hbase/hbaseid)
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
	at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:222)
	at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481)
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86)
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:849)
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:670)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:450)
	at org.jruby.javasupport.JavaMethod.invokeStaticDirect(JavaMethod.java:362)
	at org.jruby.java.invokers.StaticMethodInvoker.call(StaticMethodInvoker.java:58)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
	at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
	at org.jruby.ast.InstAsgnNode.interpret(InstAsgnNode.java:95)
	at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
	at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
	at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
	at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169)
	at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302)
	at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:148)
	at org.jruby.RubyClass.newInstance(RubyClass.java:822)
	at org.jruby.RubyClass$i$newInstance.call(RubyClass$i$newInstance.gen:65535)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrNBlock.call(JavaMethod.java:249)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135)
	at opt.hbase_minus_1_dot_0_dot_0_minus_cdh5_dot_6_dot_0.bin.hirb.__file__(/opt/hbase-1.0.0-cdh5.6.0/bin/hirb.rb:131)
	at opt.hbase_minus_1_dot_0_dot_0_minus_cdh5_dot_6_dot_0.bin.hirb.load(/opt/hbase-1.0.0-cdh5.6.0/bin/hirb.rb)
	at org.jruby.Ruby.runScript(Ruby.java:697)
	at org.jruby.Ruby.runScript(Ruby.java:690)
	at org.jruby.Ruby.runNormally(Ruby.java:597)
	at org.jruby.Ruby.runFromMain(Ruby.java:446)
	at org.jruby.Main.doRunFromMain(Main.java:369)
	at org.jruby.Main.internalRun(Main.java:258)
	at org.jruby.Main.run(Main.java:224)
	at org.jruby.Main.run(Main.java:208)
	at org.jruby.Main.main(Main.java:188)
2019-04-08 11:31:38,649 ERROR [main] zookeeper.ZooKeeperWatcher: hconnection-0x71e35c40x0, quorum=slave:2181,master:2181, baseZNode=/hbase Received unexpected KeeperException, re-throwing exception
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid

都知道HBASE是基於zookeeper的，而zookeeper是基於Hadoop的，啟動了Hadoop后我們總會啟動zookeeper再啟動HBASE，其中注意到配置時在hbase-env.sh文件中有如下設置

cd /opt/hbase-1.0.0-cdh5.6.0/conf
vi hbase-env.sh

添加如下：

export JAVA_HOME=/opt/jdk1.7.0_71    //Java安裝路徑
export HBASE_LOG_DIR=/opt/hbase-1.0.0-cdh5.6.0/logs    //hbase日志存放路徑，沒有的話會自動創建
export HADOOP_HOME=/opt/hadoop-2.6.0-cdh5.6.0         //Hadoop安裝路徑
export HBASE_MANAGES_ZK=false       //true為使用自帶的zookeeper

那么說明HBASE內是自帶zookeeper的，這就可能會出現情況 ，在你配置是是否使用了hbase自帶的zookeeper或者是自己安裝的zookeeper。端口2181是否被搶占。

解決方法：
1).hbase-env.sh 將 export HBASE_MANAGES_ZK 屬性設置為false

export HBASE_MANAGES_ZK=false
以下也可以導致這個問題：

1、netstat -antp | fgrep 2181，就知道端口被什么進程占了

2、已經啟動ZK，但是HBase並沒有使用外部的ZK，導致沖突

那么根據報錯，確實是2181端口出了問題，至於是什么導致的不知道，於是使用命令

netstat -antp | fgrep 2181

或者用lsof命令查詢{可能有的機器沒有裝netstat}

找出到底是誰占用了我的端口

結果

[master@master bin]$ netstat -antp | fgrep 2181
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 :::2181                     :::*                        LISTEN      3829/java       

確實占用了這個端口
然后執行：kill -9 3829
殺死該進程。
重新啟動HBASE，問題解決！

總結一下，得到的教訓是，hbase自帶zookeeper服務，因此配置時注意選擇用自己安裝的還是hbase自帶的，若用自帶的，可以不用啟動zookeeper，否則每次啟動hbase前要自行啟動zookeeper，而hbase自帶的zookeeper會在啟動hbase時自動啟動所有節點的zookeeper服務

之后成功登入HBASE，但是查表時出現錯誤，並且顯示znode==null的內容，檢查jps發現HMaster消失，檢查后發現啟動hbase服務后能成功啟動HMaster進程，但是登入hbase后HMaster自動消失，於是判定為使Master上的服務有問題。

解決方案
1、進入zookeeper的bin目錄；

2、執行客戶端腳本：$sh zkCli.sh

3、查看Zookeeper節點信息： ls /

4、遞歸刪除hbase節點：rmr /hbase

5、退出：quit

6、重啟Zookeeper

或者

#1、直接啟動客戶端腳本

[root@master ~]# zkCli.sh

#2、查看Zookeeper節點信息

[zk: localhost:2181(CONNECTED) 0] ls /

#3、遞歸刪除hbase節點

[zk: localhost:2181(CONNECTED) 1] rmr /hbase

#4、退出客戶端

[zk: localhost:2181(CONNECTED) 2] quit

#5、重啟Zookeeper服務

[root@master ~]# zkServer.sh stop
[root@master ~]# zkServer.sh start