CentOS配置Hive

hive搭建共分為三種模式：1、embedded，2、local，3、remote server

在這里，主要是配置第3種模式：remote server模式，如下圖所示：

 

我的環境共三台虛擬機：Host0,Host2，Host3

在remote server模式中，Host0：Hive-server2

　　　　　　　　　　　　Host2：Hive-metastore

　　　　　　　　　　　　Host3：MySQL server

1、分別在Host0，Host2和Host3中安裝hive-server2，hive-metastore，mysql-server

yum install hive-server2 -y

yum install hive-metastore -y

yum install mysql-server -y 

2、在MySQL server中創建hive數據庫以及hive用戶，並設置hive用戶可以遠程登錄

create database hive;
create user hive identified by '123456';
grant all PRIVILEGES on *.* to hive@’%’ identified by ‘123456’;
flush privileges;

3、在hive-metastore中設置hive配置文件，使其能連接到MySQL server

　　hive配置文件位置：/etc/hive/conf/hive-site.xml

　　向hive-site.xml中添加下列內容

<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://Host3:3306/hive?characterEncoding=UTF-8</value>
  <description>JDBC connect string for a JDBC metastore</description>
</property>

<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
  <description>Driver class name for a JDBC metastore</description>
</property>
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>123456</value>
</property>
<property>
  <name>hive.metastore.warehouse.dir</name>
  <!-- base hdfs path -->
  <value>/user/hive/warehouse</value>
  <description>location of default database for the warehouse</description>
</property>

4、在hdfs上創建hive.metastore.warehouse.dir目錄，並修改權限

sudo -u hdfs hadoop fs -mkdir -p /user/hive/warehouse
sudo -u hdfs hadoop fs -chown -R hive:hive /user/hive

5、在hive-server2中設置hive配置文件，使其能連接到hive-metastore

　　hive配置文件位置：/etc/hive/conf/hive-site.xml

　　向hive-site.xml中添加下列內容

<property>
  <name>hive.metastore.uris</name>
  <value>thrift://Host2:9083</value>
</property>

6、分別在Host0，Host2中啟動hive-server2，hive-metastore

service hive-server2 start
service hive-metastore start

7、在另一台虛擬機中進行測試（需要安裝hive）

[root@client ~]# beeline 
which: no hbase in (/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/java/jdk1.8.0_60/bin:/root/bin)
Beeline version 1.1.0-cdh5.8.0 by Apache Hive
beeline> !connect jdbc:hive2://Host0:10000
scan complete in 11ms
Connecting to jdbc:hive2://Host0:10000
Enter username for jdbc:hive2://Host0:10000: 1
Enter password for jdbc:hive2://Host0:10000: *
Connected to: Apache Hive (version 1.1.0-cdh5.8.0)
Driver: Hive JDBC (version 1.1.0-cdh5.8.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://Host0:10000> show tables;
INFO  : Compiling command(queryId=hive_20160912015858_bd9495d2-191f-429c-bf01-a165821b6d9a): show tables
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)
INFO  : Completed compiling command(queryId=hive_20160912015858_bd9495d2-191f-429c-bf01-a165821b6d9a); Time taken: 0.027 seconds
INFO  : Concurrency mode is disabled, not creating a lock manager
INFO  : Executing command(queryId=hive_20160912015858_bd9495d2-191f-429c-bf01-a165821b6d9a): show tables
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hive_20160912015858_bd9495d2-191f-429c-bf01-a165821b6d9a); Time taken: 0.061 seconds
INFO  : OK
+-----------+--+
| tab_name  |
+-----------+--+
+-----------+--+
No rows selected (0.389 seconds)

需要注意的是：如果在hiveserver2方式中沒有使用Kerberos，則上述密碼可以隨意寫

8、在Hive中創建表，並將數據導入表中

CREATE TABLE test(id int, A int, B int, C int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';
load data inpath '/user/sqoop/sample' into table test;

/user/sqoop/sample表中的數據格式為：11419,9,160,48格式
當執行load data inpath '/user/sqoop/sample' into table test;后，/user/sqoop/sample里的數據就mv到了/user/hive/warehouse/test中去了，/user/sqoop/sample目錄為空。