ambari hdp 集成 impala

1.下載ambari-impala-service

VERSION=`hdp-select status hadoop-client | sed 's/hadoop-client - \([0-9]\.[0-9]\).*/\1/'`
sudo git clone https://github.com/cas-bigdatalab/ambari-impala-service.git /var/lib/ambari-server/resources/stacks/HDP/$VERSION/services/IMPALA

2./etc/yum.repos.d新建impala.repo

[cloudera-cdh5]
# Packages for Cloudera's Distribution for Hadoop, Version 5, on RedHat	or CentOS 6 x86_64
name=Cloudera's Distribution for Hadoop, Version 5
baseurl=https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5/
gpgkey =https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera    
gpgcheck = 1

3.重啟ambari-server

sudo service ambari-server restart

4.安裝impala

Actions-->Add service -->impala，然后一步一步安裝

因為要去cloudera官網下載impala的rpm文件，安裝過程比較慢，可能會失敗，多試幾次就好了

5.hdfs增加配置

#HDFS config we need add below config to /etc/hadoop/conf/core-site.xml
<property>
    <name>dfs.client.read.shortcircuit</name> 
   <value>true</value>
</property>

<property>
    <name>dfs.client.read.shortcircuit.skip.checksum</name>
        <value>false</value>
</property>

<property> 
    <name>dfs.datanode.hdfs-blocks-metadata.enabled</name> 
    <value>true</value>
</property>

we need add below config to /etc/hadoop/conf/hdfs-site.xml
<property>
    <name>dfs.datanode.hdfs-blocks-metadata.enabled</name> 
    <value>true</value>
</property>
<property> 
    <name>dfs.block.local-path-access.user</name> 
    <value>impala</value>
</property>
<property>
    <name>dfs.client.file-block-storage-locations.timeout.millis</name>
    <value>60000</value>
</property>

 6.重啟hdfs

7.啟動impala

8.把hbase的jar包拷貝到 /usr/lib/impala/lib 並創建軟連接

這里注意根據安裝的impala的版本對應的cdh的版本選擇對應的hbase的版本，否則可能出現版本不兼容的問題

ln -s hbase-annotations-1.2.0-cdh5.11.0.jar hbase-annotations.jar
ln -s hbase-client-1.2.0-cdh5.11.0.jar hbase-client.jar
ln -s hbase-common-1.2.0-cdh5.11.0.jar hbase-common.jar
ln -s hbase-examples-1.2.0-cdh5.11.0.jar hbase-examples.jar
ln -s hbase-external-blockcache-1.2.0-cdh5.11.0.jar hbase-external-blockcache.jar
ln -s hbase-hadoop2-compat-1.2.0-cdh5.11.0.jar hbase-hadoop2-compat.jar
ln -s hbase-hadoop-compat-1.2.0-cdh5.11.0.jar hbase-hadoop-compat.jar
ln -s hbase-it-1.2.0-cdh5.11.0.jar hbase-it.jar
ln -s hbase-prefix-tree-1.2.0-cdh5.11.0.jar hbase-prefix-tree.jar
ln -s hbase-procedure-1.2.0-cdh5.11.0.jar hbase-procedure.jar
ln -s hbase-protocol-1.2.0-cdh5.11.0.jar hbase-protocol.jar
ln -s hbase-resource-bundle-1.2.0-cdh5.11.0.jar hbase-resource-bundle.jar
ln -s hbase-rest-1.2.0-cdh5.11.0.jar hbase-rest.jar
ln -s hbase-rsgroup-1.2.0-cdh5.11.0.jar hbase-rsgroup.jar
ln -s hbase-server-1.2.0-cdh5.11.0.jar hbase-server.jar
ln -s hbase-shell-1.2.0-cdh5.11.0.jar hbase-shell.jar
ln -s hbase-spark-1.2.0-cdh5.11.0.jar hbase-spark.jar
ln -s hbase-thrift-1.2.0-cdh5.11.0.jar hbase-thrift.jar

如果沒有拷貝hbase的jar到/usr/lib/impala/lib 下則會報錯：

光拷貝hbase jar包到/usr/lib/impala/lib 下如果不修改名字的話會識別不到，impala-server啟動時會報類似的錯：F0510 04:53:11.157616 25119 impalad-main.cc:64] NoClassDefFoundError: org/apache/hadoop/hbase/client/Scan

CAUSED BY: ClassNotFoundException: org.apache.hadoop.hbase.client.Scan