java 鏈接hive（kerberos認證）實例 以及 錯誤點講解

沒有kerberos認證的hive鏈接比較方便，但是有kerberos認證就需要多做一些事情。

1、pom.xml 依賴：

<dependency>
   <groupId>org.apache.hive</groupId>
   <artifactId>hive-jdbc</artifactId>
   <version>0.12.0</version>
</dependency>
<dependency>
   <groupId>org.apache.hadoop</groupId>
   <artifactId>hadoop-client</artifactId>
   <version>2.7.2</version>
</dependency>

2、示例代碼：（標黃的需要認真修改）

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.security.UserGroupInformation;
 
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
 
public class App {
    private static String JDBC_DRIVER = "org.apache.hive.jdbc.HiveDriver";
    private static String CONNECTION_URL ="jdbc:hive2://10.23.13.196:10000/ods;principal=hive/tw-manager@TDH";
 
    static {
        try {
            Class.forName(JDBC_DRIVER);
 
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
        }
    }
 
    public static void main(String[] args) throws Exception  {
        Class.forName(JDBC_DRIVER);
 
        //登錄Kerberos賬號
        System.setProperty("java.security.krb5.conf", "/root/tdh-cdh/ticket/krb5.conf");
 
        Configuration configuration = new Configuration();
        configuration.set("hadoop.security.authentication" , "Kerberos" );
        UserGroupInformation. setConfiguration(configuration);
        UserGroupInformation.loginUserFromKeytab("hive@TDH" , "/root/tdh-cdh/ticket/hive.keytab");
 
        Connection connection = null;
        ResultSet rs = null;
        PreparedStatement ps = null;
        try {
            connection = DriverManager.getConnection(CONNECTION_URL);
            ps = connection.prepareStatement("show tables");
            rs = ps.executeQuery();
            while (rs.next()) {
                System.out.println(rs.getString(1));
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        
    }
}

　　

3、注意事項：

（1）krb5.conf、hive.keytab2個文件，路徑要改對

（2）jar依賴包版本和服務器hadoop、hive版本要匹配，否則會報錯：

　　　　Caused by: org.apache.thrift.TApplicationException: Required field 'client_protocol' is unset! Struct:TOpenSessionReq

　　　　原因：This indicates a version mismatch between client and server, namely that the client is newer than the server, which is your case.

　　　　客戶端和服務端包的版本不一致；

（3）錯誤:Error: Could not open client transport with JDBC Uri: jdbc:hive2://master:10000/default: Peer indicated failure:

　　　　Unsupported mechanism type PLAIN (state=08S01,code=0)

　　　　解決: jdbc連接url時,缺少principal,整個url連接如下:jdbc:hive2://master:10000/default;principal=hive/[e-mail]

（4）錯誤: Could not open client transport with JDBC Uri: jdbc:hive2://master:10000/default;principal=hdfs/[e-mail]: Peer indicated failure:

　　　　GSS initiate failed (state=08S01,code=0)
　　解決: jdbc連接url時,principal的用戶有問題,整個url連接如下:
　　　　principal=hdfs/[e-mail] 替換為:hive/[e-mail]
　　　　jdbc:hive2://master:10000/default;principal=hive/[e-mail]

 

 

 

參考：

　　1、https://blog.csdn.net/DH2442897094/article/details/108007806?utm_medium=distribute.pc_relevant.none-task-blog-blogcommendfrommachinelearnpai2-2.channel_param&depth_1-utm_source=distribute.pc_relevant.none-task-blog-blogcommendfrommachinelearnpai2-2.channel_param

　　2、https://blog.csdn.net/weixin_44865574/article/details/106420734?utm_medium=distribute.pc_relevant.none-task-blog-title-3&spm=1001.2101.3001.4242