hive2.1.0安裝


這里是使用mysql作為hive的元數據存儲所以要先啟動mysql的服務,MySQL的安裝可以參考下面連接

MySQL安裝:http://www.cnblogs.com/zhangXingSheng/p/6941422.html

 

下載hive(http://mirrors.cnnic.cn/apache/hive/)

或者 http://archive.apache.org/dist/hive/(hive歷史版本)
在本地進行解壓
 
下載mysql的jdbc,將下載后的jdbc放到hive安裝包的lib目錄下
http://dev.mysql.com/downloads/connector/j/
 
配置環境變量
#########hive####################
export HIVE_HOME=/usr/local/development/hive-2.1.0
export PATH=$HIVE_HOME/bin:$PATH
在hdfs上創建hive存儲數據的相關目錄
[root@node4 bin]# hdfs dfs -mkdir -p /opt/hive/warehouse
[root@node4 bin]# hdfs dfs -mkdir -p /opt/hive/log
修改hive-site.xml文件
cp hive-default.xml.template hive-site.xml
<name>hive.metastore.warehouse.dir</name>
<value>/opt/hive/warehouse</value>//對應以上在hdfs創建的文件目錄
<description>location of default database for the warehouse</description>
 
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://192.168.177.124:3306/hivedb</value>//mysql的鏈接地址,這里的hivedb是我已經在mysql中創建的數據庫,如果不寫就是default
<description>
JDBC connect string for a JDBC metastore.
To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
For example, jdbc:postgresql://myhost/dbName?ssl=true for postgres database.
</description>
 
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>//mysql的驅動
<description>Driver class name for a JDBC metastore</description>
 
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>//鏈接mysql的用戶名
<description>Username to use against metastore database</description>
 
 
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>////鏈接mysql的密碼
<description>password to use against metastore database</description>
 
<name>hive.metastore.uris</name>
<value>thrift://192.168.177.124:9083</value>
 <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description>
 
 
修改hive-log4j2.properties和hive-exec-log4j2.properties //hive的日志文件
在 新建日志存儲目錄/home/zhangxs/hive/log  和  /home/zhangxs/hive/execlog
 
(1)hive-log4j2.properties
property.hive.log.dir = ${mine:/home/zhangxs/hive/log}/${user.name}
(2)hive-exec-log4j2.properties
property.hive.log.dir = ${mine:/home/zhangxs/hive/execlog}/${user.name}
 
啟動步驟
1:首次啟動hive需要初始化
使用schematool初始(要保證hadoop集群已經啟動)
schematool -dbType mysql -initSchema
 
2:啟動hive服務
[ hive --service metastore]
 
3:新打開一個終端 輸入
hive
 
 
 
遇到問題
1:這個錯誤是因為,我在hive-site.xml文件里配置的是以root用戶登錄的,mysql不予許hive遠程root登錄,你可以配置一下權限,或者換一個普通的用戶登錄
Caused by: java.sql.SQLException: Access denied for user 'root'@'hadoop-node4.com' (using password: YES)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:964)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3970)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3906)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:873)
at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1710)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1226)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2253)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2284)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2083)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:806)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:410)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:328)
at java.sql.DriverManager.getConnection(DriverManager.java:579)
at java.sql.DriverManager.getConnection(DriverManager.java:190)
at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
問題2:hive在第一次登錄的時候使用 schematool -dbType mysql -initSchema命令初始化,以后則不用。使用
hive --service metastore啟動服務即可
Logging initialized using configuration in file:/usr/local/development/hive-2.1.0/conf/hive-log4j2.properties Async: true
Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578)
at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:518)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

 

問題3:這個是因為我修改hive-site.xml文件時候,值替換了system:java.io.tmpdir一部分,因該將
這整個value里面的值都替換掉
Logging initialized using configuration in file:/usr/local/development/hive-2.1.0/conf/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. tez, spark) or using Hive 1.X releases.
hive> show tables;
FAILED: IllegalArgumentException java.net.URISyntaxException: Relative path in absolute URI: file:./$%7B/home/zhangxs/data/hive/tmp%7D/$%7Bsystem:user.name%7D/dfb16330-2bbc-481a-bbdc-e4503c4d1a5f/hive_2016-12-12_05-12-52_963_7246570582065346726-1
hive> show tables;
FAILED: IllegalArgumentException java.net.URISyntaxException: Relative path in absolute URI: file:./$%7B/home/zhangxs/data/hive/tmp%7D/$%7Bsystem:user.name%7D/dfb16330-2bbc-481a-bbdc-e4503c4d1a5f/hive_2016-12-12_05-14-22_015_4353303889983385851-1
 
 
在啟動 schematool -dbType mysql -initSchema或者 hive --service metastore的時候命令后,應該在重新打開一個窗口執行hive命令,要不然,會卡在
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
 
Logging initialized using configuration in file:/usr/local/development/hive-2.1.0/conf/hive-log4j2.properties Async: true
^Z
[2]+ Stopped hive
 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM