cdh版hbase構建Phoenix 遇到的坑

 

Phoenix 構建cdh版hbase遇到的坑

 

1. 安裝phoenix

下載：在github上下載對應版本https://github.com/apache/phoenix

解壓：略

編譯：

修改根目錄及其子目錄下的pom.xml文件，使cdh版本對應自己集群版本。如圖所示

注意：編譯中修改版本號遵循原始的寫法。4.14.0-cdh5.12.1 版本不可以寫成4.14-cdh5.12.1，因為在編譯時會有正則校驗，書寫不規范會導致編譯不通過。

 命令：

 vim pom.xml
/cdh5.12  #搜索

 

修改所有的cdh版本號

 

編譯：mvn clean package -DskipTests

拷貝jar :將下面的jar包拷貝至每一台RegionServer的lib下面

/opt/phoenix/APACHE_PHOENIX-4.14.0-cdh5.12.1.p0.0/lib/phoenix/phoenix-4.14.0-cdh5.12.1-server.jar

 

 

2. 配置hbase-site.xml

注意前方高能：phoenix4.8版本是個分水嶺，配置有所不同

• phoenix4.8以下版本的配置如下：

 配置HMaster的hbase-site.xml

<!-- Phoenix訂制的索引負載均衡器 --> 
<property>
 <name>hbase.master.loadbalancer.class</name> <value>org.apache.phoenix.hbase.index.balancer.IndexLoadBalancer</value> 
</property> 
<!-- Phoenix訂制的索引觀察者 --> 
<property>
 <name>hbase.coprocessor.master.classes</name> <value>org.apache.phoenix.hbase.index.master.IndexMasterObserver</value> 
</property> 

配置RegionServer的hbase-site.xml

<!-- Enables custom WAL edits to be written, ensuring proper writing/replay of the index updates. This codec supports the usual host of WALEdit options, most notably WALEdit compression. --> 
<property>
  <name>hbase.regionserver.wal.codec</name>
  <value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value> 
</property> 
<!-- Prevent deadlocks from occurring during index maintenance for global indexes (HBase 0.98.4+ and Phoenix 4.3.1+ only) by ensuring index updates are processed with a higher priority than data updates. It also prevents deadlocks by ensuring metadata rpc calls are processed with a higher priority than data rpc calls --> 
<property>
  <name>hbase.region.server.rpc.scheduler.factory.class</name>  
  <value>org.apache.hadoop.hbase.ipc.PhoenixRpcSchedulerFactory</value> 
  <description>Factory to create the Phoenix RPC Scheduler that uses separate queues for index and metadata updates</description>
</property> 
<property>
  <name>hbase.rpc.controllerfactory.class</name>  
  <value>org.apache.hadoop.hbase.ipc.controller.ServerRpcControllerFactory</value> 
  <description>Factory to create the Phoenix RPC Scheduler that uses separate queues for index and metadata updates</description> 
</property>
 <!-- To support local index regions merge on data regions merge you will need to add the following parameter to hbase-site.xml in all the region servers and restart. (It’s applicable for Phoenix 4.3+ versions) --> 
 <property>
   <name>hbase.coprocessor.regionserver.classes</name>
     <value>org.apache.hadoop.hbase.regionserver.LocalIndexMerger</value> 
   </property> 

 

• phoenix4.8+版本只需添加以下配置

 

<!-- Enables custom WAL edits to be written, ensuring proper writing/replay of the index updates. This codec supports the usual host of WALEdit options, most notably WALEdit compression. --> 
<property>
  <name>hbase.regionserver.wal.codec</name>
  <value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value> 
</property> 
<!-- Prevent deadlocks from occurring during index maintenance for global indexes (HBase 0.98.4+ and Phoenix 4.3.1+ only) by ensuring index updates are processed with a higher priority than data updates. It also prevents deadlocks by ensuring metadata rpc calls are processed with a higher priority than data rpc calls --> 
<property>
  <name>hbase.region.server.rpc.scheduler.factory.class</name>
  <value>org.apache.hadoop.hbase.ipc.PhoenixRpcSchedulerFactory</value> 
  <description>Factory to create the Phoenix RPC Scheduler that uses separate queues for index and metadata updates</description> 
</property> 
<property>
  <name>hbase.rpc.controllerfactory.class</name>
  <value>org.apache.hadoop.hbase.ipc.controller.ServerRpcControllerFactory</value>
   <description>Factory to create the Phoenix RPC Scheduler that uses separate queues for index and metadata updates</description> 
</property>

• 解釋

在4.8版本之后，安裝Phoenix需要把hbase-site.xml文件中的這三個配置項給刪掉，因為這三個配置項在4.7及之前的版本會導致在”security enabled”的情況下，每次建立和phoenix server的會話時都會建立一個新的連接實例，短時間內創建大量連接會觸發Zookeeper內置的保護機制，拒絕連接。

簡單來說，就是HBase在重啟的時候，之前賦值到hbase lib目錄下的phoenix-4.X.X-HBase-1.X-server.jar包會讀取hbase-site.xml文件，結果發現了這三個過期的配置項，然后引起了異常，把這三個配置項刪掉就好。。。。。。

(MMP，就這個搞了半天)

山重水復已無路，柳暗花明又一村。感謝這位熊蝶

https://blog.csdn.net/silentasriddle/article/details/78758950

 

 

如果是cdh版本的hbase，則在管理界面可直接修改，如下圖（phoenxi 4.14的）

 

 

 

 

3. 修改和zookeeper相關的配置保證啟動時通信不超時

修改zookeeper的配置maxSessionTimeout

默認是6000，改為1200000，單位是ms。這個值要大於hbase中的值。

 

 

修改hbase的配置

 

 

QQ:1710816711(歡迎交流，廣告勿擾)