使用Hive-JDBC遇到的一些問題解決


使用Hive-JDBC遇到的一些問題解決(轉)

標簽(空格分隔): Hive


1,java.lang.NoClassDefFoundError: org/apache/hive/service/cli/thrift/TCLIService$Iface

java.lang.NoClassDefFoundError: org/apache/hive/service/cli/thrift/TCLIService$Iface
	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
	at java.sql.DriverManager.getConnection(DriverManager.java:664)
	at java.sql.DriverManager.getConnection(DriverManager.java:208)
	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:116)
	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:107)
	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:191)
	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:161)
	at com.bonc.vbap.data.query.jdbc.ConnectionManager.getConnection(ConnectionManager.java:63)
	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext$SchemaHelper.getConnection(JdbcSchemaContext.java:308)
	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext$SchemaHelper.<init>(JdbcSchemaContext.java:112)
	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext.getSchemaHelper(JdbcSchemaContext.java:68)
	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext.getSchemaNames(JdbcSchemaContext.java:43)
	at com.bonc.vbap.data.service.impl.SchemaServiceImpl.getSchemaNames(SchemaServiceImpl.java:34)
	at com.bonc.vbap.data.schema.jdbc.TestJdbcColumn.testHiveDataQuery(TestJdbcColumn.java:252)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
	at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:252)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:94)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: java.lang.ClassNotFoundException: org.apache.hive.service.cli.thrift.TCLIService$Iface
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 44 more

解決:
所以去看下引入的依賴包,發現有hive-service和hive-service-rpc兩個包中有相同的源碼,所以在這里會拋異常,我就先把這其中一個依賴從hive-jdbc中去除掉,但是發現還是拋異常,我這里用的hive-jdbc1.1版本,發現hive-service是2.1版本,所以立馬把兩個都去除掉,重新引入一個hive-service進來就好拉

<dependency>
  <groupId>org.apache.hive</groupId>
  <artifactId>hive-jdbc</artifactId>
  <exclusions>
    <exclusion>
      <groupId>org.apache.hive</groupId>
      <artifactId>hive-service-rpc</artifactId>
    </exclusion>
    <exclusion>
      <groupId>org.apache.hive</groupId>
      <artifactId>hive-service</artifactId>
    </exclusion>
  </exclusions>
  <version>1.1.0</version>
</dependency>
<dependency>
  <groupId>org.apache.hive</groupId>
  <artifactId>hive-service</artifactId>
  <version>1.1.0</version>
</dependency>

2,上述的問題解決之后,又出現了一個新的問題:java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration

異常詳情:


java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
	at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:393)
	at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:187)
	at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
	at java.sql.DriverManager.getConnection(DriverManager.java:664)
	at java.sql.DriverManager.getConnection(DriverManager.java:208)
	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:116)
	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:107)
	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:191)
	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:161)
	at com.bonc.vbap.data.query.jdbc.ConnectionManager.getConnection(ConnectionManager.java:63)
	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext$SchemaHelper.getConnection(JdbcSchemaContext.java:308)
	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext$SchemaHelper.<init>(JdbcSchemaContext.java:112)
	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext.getSchemaHelper(JdbcSchemaContext.java:68)
	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext.getSchemaNames(JdbcSchemaContext.java:43)
	at com.bonc.vbap.data.service.impl.SchemaServiceImpl.getSchemaNames(SchemaServiceImpl.java:34)
	at com.bonc.vbap.data.schema.jdbc.TestJdbcColumn.testHiveDataQuery(TestJdbcColumn.java:252)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
	at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:252)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:94)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

引入Hadoop-commom 包


<dependency>
  <groupId>org.apache.hadoop</groupId>
  <artifactId>hadoop-common</artifactId>
  <version>2.6.5</version>
</dependency>

3. 在使用DatabaseMetaData獲取元數據信息時,拋出異常:

"Method not supported"

這是因為HiveDatabaseMetaData在實現DatabaseMetaData接口時,沒有做相應方法的實現,這里我使用到的是一個getUserName接口:

  ...
  public String getUserName() throws SQLException {
    throw new SQLException("Method not supported");
  }


可以看到在HiveDatabaseMetaData中沒有對getUserName支持,所以會拋異常,getUserName方法是獲取到當前HiveConnection連接的用戶名(Schema名稱),但是這里不支持,那就得另外想辦法,這里可以調用HiveConnection中的一個getConnParams方法得到連接參數,再調用getSchema方法獲取到Schema名稱。代碼:


HiveDatabaseMetaData hiveMetaData = (HiveDatabaseMetaData) metadata;
HiveConnection hiveConn = (HiveConnection) hiveMetaData.getConnection();
String hiveSchema = hiveConn.getSchema();


原文:https://blog.csdn.net/u010429286/article/details/78780145


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM