hive常見問題解決干貨大全


 

  本人,苦心多時,歷經磨難和心血,與大家共同攻克問題難關!

 

問題一: 

  FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

 參考:http://blog.csdn.net/l1028386804/article/details/51566262

 解決辦法:

  進入所安裝的Hive的conf目錄,找到hive-site.xml,(若沒修改,則是hive-default.xml.template)。

<property>

  <name>hive.metastore.schema.verification</name>

  <value>true</value>

   <description>

   Enforce metastore schema version consistency.

   True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic

         schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures

         proper metastore schema migration. (Default)

   False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.

   </description>

</property>

 

 

改為

<property>

  <name>hive.metastore.schema.verification</name>

  <value>false</value>

   <description>

   Enforce metastore schema version consistency.

   True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic

         schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures

         proper metastore schema migration. (Default)

   False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.

   </description>

</property>

 

  然后,再重啟hive,進入即可。

   或者  在hive安裝目錄下的bin目錄下,./hive -hiveconf hive.root.logger=DEBUG,console

 

  同時,最近還有人來問我。出現如下問題SeesionHiveMetaStoreClient

  先按照我的問題一的思路去解決,若解決不了,看看你的hdfs-site.xml和core-site.xml是否正確。(因為有人問我,最后幫解決發現是這個問題)

見。如下博客

hive權威安裝出現的不解錯誤!(完美解決)兩種方法都可以

   和

http://www.cnblogs.com/braveym/p/6685045.html

 

 

 

 

 

 

 

  問題二:

    Hive報錯為:There are 2 datanode(s) running and no node(s) are excluded in this operation.

  詳細報錯為:

 org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /tmp/hive/zhou/ed69eb4e-855a-4b18-a498-345e8685533a/hive_2015-12-14_05-32-03_335_9179215496953152524-1/-mr-10004/f8d2e938-b439-495f-b886-801c98f660da/map.xml could only be replicated to 0 nodes instead of minReplication (=1).  There are 2 datanode(s) running and no node(s) are excluded in this operation.

  參考:http://jingyan.baidu.com/article/7082dc1c65a76be40a89bd09.html

  解決方法:

      出現此類報錯主要原因是datanode存在問題,要么硬盤容量不夠,要么datanode服務器down了。檢查datanode,重啟Hadoop即可解決。

 

 

  問題三:

   hive保持,  Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.CopyTask

  參考:http://blog.sina.com.cn/s/blog_75353ff40102v0d3.html

  解決:

      ./hive -hiveconf hive.root.logger=DEBUG,console
    准備通過控制台輸出來定位出錯位置,沒想到這樣設置后,運行程序就成功了。自己比較莫名其妙。
    總之還是要養成常看錯誤日志解決問題。

 

   問題四:

 

Caused by: MetaException(message:Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema.  If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql))  

這是由於沒有初始化Hive元數據的數據庫,默認情況下,Hive的元數據保存在了內嵌的derby數據庫里。  

執行如下命令方可  

 在hive安裝目錄下,執行 ./bin/schematool -initSchema -dbType derby  

 

  參考: http://fengshulin.iteye.com/blog/2320359

 

  問題五:

Exception in thread "main"java.lang.RuntimeException: java.lang.IllegalArgumentException:java.net.URISyntaxException: Relative path in absolute URI:${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

 

解決方案:

在hive 安裝目錄下,創建一個臨時的IO文件iotmp。比如hive的安裝目錄是/home/hadoop/app/hive

然后將路徑配置到hive-site.xml文件的以下參數中:

< property> < name>hive.querylog.location< /name> < value>/home/hadoop/app/hive/iotmp< /value> < description>Location of Hive run time structured log file< /description> < /property> < property> < name>hive.exec.local.scratchdir< /name> < value>/home/hadoop/app/hive/iotmp< /value> < description>Local scratch space for Hive jobs< /description> < /property> < property> < name>hive.downloaded.resources.dir< /name> < value>/home/hadoop/app/hive/iotmp< /value> < description>Temporary local directory for added resources in the remote file system.< /description> < /property>


問題六:
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

  解決辦法:

[hadoop@HadoopSlave1 hive-1.2.1]$ hive --service metastore &

   如果實在不行,就,再參考這篇。https://my.oschina.net/AlbertHa/blog/339022

   再實在不行,見http://www.aboutyun.com/thread-11131-1-1.html




問題七:

[hadoop@HadoopSlave1 hive-1.2.1]$ bin/hive

Logging initialized using configuration in jar:file:/home/hadoop/app/hive-1.2.1/lib/hive-common-1.2.1.jar!/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
... 8 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
... 14 more
Caused by: MetaException(message:Got exception: java.io.IOException No FileSystem for scheme: hfds)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:1213)
at org.apache.hadoop.hive.metastore.Warehouse.getFs(Warehouse.java:106)
at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:140)
at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:146)
at org.apache.hadoop.hive.metastore.Warehouse.getWhRoot(Warehouse.java:159)
at org.apache.hadoop.hive.metastore.Warehouse.getDefaultDatabasePath(Warehouse.java:177)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:600)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
... 19 more
[hadoop@HadoopSlave1 hive-1.2.1]$

 




將hadoop中share/hadoop/yarn/lib路徑下jline包換成hive中匹配的jar包

 

 

 














歡迎大家,加入我的微信公眾號:大數據躺過的坑     免費給分享
 
 
 

同時,大家可以關注我的個人博客

   http://www.cnblogs.com/zlslch/   和     http://www.cnblogs.com/lchzls/ 

   詳情請見:http://www.cnblogs.com/zlslch/p/7473861.html

 

 

  人生苦短,我願分享。本公眾號將秉持活到老學到老學習無休止的交流分享開源精神,匯聚於互聯網和個人學習工作的精華干貨知識,一切來於互聯網,反饋回互聯網。
  目前研究領域:大數據、機器學習、深度學習、人工智能、數據挖掘、數據分析。 語言涉及:Java、Scala、Python、Shell、Linux等 。同時還涉及平常所使用的手機、電腦和互聯網上的使用技巧、問題和實用軟件。 只要你一直關注和呆在群里,每天必須有收獲

 

       以及對應本平台的QQ群:161156071(大數據躺過的坑)

 

 

 

 

 

 

 

 

 

 

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM