Hive初始化元數據常見報錯

Hive元數據相關配置

hive的配置

下面是Hive元存儲”的存儲位置為MySQL數據庫，必要的配置。包括連接URL，驅動類，數據庫賬號以及密碼

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://master1-1:3306/hive?characterEncoding=UTF-8</value>
    <description>連接的url</description>
</property>
<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>連接驅動類名</description>
</property>
<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>********</value>
    <description>連接數據庫用戶名</description>
</property>
<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>********</value>
    <description>連接數據庫密碼</description>
</property>

mysql數據庫配置

1. 首先我們要創建一個用於保存hive元數據的數據庫

create database hive charset=utf8

2.賦予hive連接mysql用戶遠程的權限(這里我使用的是root賬戶來進行連接

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'PASSWORD' WITH GRANT OPTION

3.將連接mysql的驅動包放到hive的lib目錄下

cp mysql-connector-java.jar ${HIVE_HOME}/lib/

初始化hive的元數據

使用schematool 初始化hive的元數據

schematool -initSchema -dbType mysql

檢查數據庫

use hive

show tables;

我們會看到存放hive元數據的數據庫已經生成了很多表，到這里，就表示你hive元數據配置和初始化成功了！

hive初始化元數據常見的報錯

錯誤1

org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.

這里的排查方法是:

1. 檢查hive-site.xml的數據庫用戶名密碼有沒有寫錯

2. 檢查用於連接的用戶是否有遠程登陸的權限

錯誤2

[ERROR] Terminal initialization failed; falling back to unsupported
java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

對於這個錯誤，我們把${HADOOP_HOME}/share/hadoop/yarn/lib/jline-0.9.94.jar這個文件刪除就可以了

rm ${HADOOP_HOME}/share/hadoop/yarn/lib/jline*.jar

錯誤3

有時候，由於你不是用的root用戶進行安裝會導致一些臨時文件或者目錄沒有權限的問題

Logging initialized using configuration in file:/opt/apache-hive-2.3.5-bin/conf/hive-log4j2.properties Async: true
Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
        at org.apache.hadoop.fs.Path.initialize(Path.java:254)
        at org.apache.hadoop.fs.Path.<init>(Path.java:212)
        at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:659)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:582)
        at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:549)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:750)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
        at java.net.URI.checkPath(URI.java:1823)
        at java.net.URI.<init>(URI.java:745)
        at org.apache.hadoop.fs.Path.initialize(Path.java:251)

我們需要配置一下臨時文件目錄，而且hive的管理用戶擁有訪問這些目錄的權限，最好的創建在安裝hive的用戶的用戶目錄下，我這里是hive用戶，就創建在/home/hive這個目錄下

<property>
    <name>Hive.exec.local.scratchdir</name>
    <value>/home/hive/tmp/</value>
    <description>資源文件目錄</description>
  </property>
 
  <property>
    <name>hive.downloaded.resources.dir</name>
    <value>/home/hive/tmp/resources</value>
    <description>下載資源目錄</description>
  </property>
 
<property>
    <name>hive.server2.logging.operation.log.location</name>
    <value>/home/hive/tmp/operation_logs</value>
    <description>日志文件目錄</description>