hive-3.1.2 整合進 hadoop-3.3.0 + hbase-2.2.4

一、下載匹配hadoop-3.x.y 版本的hive 3.1.2

下載地址 ： http://mirror.bit.edu.cn/apache/hive/

二、上傳至安裝目錄 /home/apache-hive-3.1.2-bin.tar.gz

 解壓：tar -zxvf apache-hive-3.1.2-bin.tar.gz  后重命名目錄：/home/hive-3.1.2

三、編輯 /etc/profile 文件

......
if [ -n "${BASH_VERSION-}" ] ; then
        if [ -f /etc/bashrc ] ; then
                # Bash login shells run only /etc/profile
                # Bash non-login shells run only /etc/bashrc
                # Check for double sourcing is done in /etc/bashrc.
                . /etc/bashrc
       fi
fi

export JAVA_HOME=/usr/java/jdk1.8.0_131
export JRE_HOME=${JAVA_HOME}/jre
export HADOOP_HOME=/home/hadoop-3.3.0
export HIVE_HOME=/home/hive-3.1.2
export CLASSPATH=$($HADOOP_HOME/bin/hadoop classpath):$CLASSPATH
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH:$HIVE_HOME/bin

執行生效： source /etc/profile

四、將 /home/hadoop-3.3.0/share/hadoop/common/lib/guava-27.0-jre.jar   拷貝並替換掉 /home/hive-3.1.2/lib/guava-19.0.jar 

五、上傳mysql-connector-java-5.1.47.jar 至/home/hive-3.1.2/lib 目錄下；

創建mysql數據庫：db_hive

創建 hive-site.xml並上傳至/home/hive-3.1.2/conf 目錄下：

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
--><configuration>
  <!-- WARNING!!! This file is auto generated for documentation purposes ONLY! -->
  <!-- WARNING!!! Any changes you make to this file will be ignored by Hive.   -->
  <!-- WARNING!!! You must make your changes in hive-site.xml instead.         -->
  <!-- Hive Execution Parameters -->
   <!-- 插入一下代碼 -->
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>數據庫用戶</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>數據庫密碼</value>
    </property>
   <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://****.mysql.rds.aliyuncs.com/db_hive?useSSL=false&amp;rewriteBatchedStatements=true&amp;useServerPrepStmts=true&amp;cachePrepStmts=true&amp;autoReconnect=true&amp;failOverReadOnly=false</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
        <name>hive.metastore.schema.verification</name>
        <value>false</value>
    </property>
    
    <property>
         <name>hbase.zookeeper.quorum</name>
         <value>39.108.***.***</value>
    </property>
    <property>
         <name>hbase.zookeeper.property.clientPort</name>
         <value>2181</value>
    </property>

    <!-- 到此結束代碼 -->
</configuration>

編輯hive-env.sh並覆蓋至/home/hive-3.1.2/conf 目錄下：

# Set HADOOP_HOME to point to a specific hadoop install directory
# HADOOP_HOME=${bin}/../../hadoop

# Hive Configuration Directory can be controlled by:
# export HIVE_CONF_DIR=

# Folder containing extra libraries required for hive compilation/execution can be controlled by:
# export HIVE_AUX_JARS_PATH=

export HADOOP_HOME=/home/hadoop-3.3.0 export HBASE_HOME=/home/hbase-2.2.4

六、執行# hive --version 確認是否安裝成功

七、執行hive命令：# hive

       執行db_hive數據庫表初始化命令：schematool -dbType mysql -initSchema

       執行成功后即可看到庫表信息：

     執行與hbase數據庫表關聯的建表語句：CREATE EXTERNAL TABLE test(key string,id int,name string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,user:id,user:name") TBLPROPERTIES("hbase.table.name" = "test");

    執行SQL查詢：select * from test