CDH5.16.X啟用Kerberos

 

1

• 內容概述：

1.如何安裝及配置KDC服務

2.如何通過CDH啟用Kerberos

3.如何登錄Kerberos並訪問Hadoop相關服務

4.總結

• 測試環境：

1.操作系統：Redhat7.4

2.CDH5.16.1

3.采用root用戶進行操作

2

KDC服務安裝及配置

本文檔中將KDC服務安裝在Cloudera Manager Server所在服務器上（KDC服務可根據自己需要安裝在其他服務器）

1.在Cloudera Manager服務器上安裝KDC服務

[root@ip-172-31-13-38 ~]$ yum -y install krb5-server krb5-libs krb5-auth-dialog krb5-workstation

2.修改/etc/krb5.conf配置

[root@ip-172-31-13-38 ~]$ vim /etc/krb5.conf # Configuration snippets may be placed in this directory as well includedir /etc/krb5.conf.d/ [logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] dns_lookup_realm = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true rdns = false default_realm = FAYSON.COM #default_ccache_name = KEYRING:persistent:%{uid} [realms] FAYSON.COM = { kdc = ip-172-31-13-38.ap-southeast-1.compute.internal admin_server = ip-172-31-13-38.ap-southeast-1.compute.internal } [domain_realm] .ap-southeast-1.compute.internal = FAYSON.COM ap-southeast-1.compute.internal = FAYSON.COM

標紅部分為需要修改的信息。

3.修改/var/kerberos/krb5kdc/kadm5.acl配置

[root@ip-172-31-13-38 ~]$ vim /var/kerberos/krb5kdc/kadm5.acl */[e-mail] *

4.修改/var/kerberos/krb5kdc/kdc.conf配置

[root@ip-172-31-13-38 ~]$ vim /var/kerberos/krb5kdc/kdc.conf [kdcdefaults] kdc_ports = 88 kdc_tcp_ports = 88 [realms] FAYSON.COM = { #master_key_type = aes256-cts acl_file = /var/kerberos/krb5kdc/kadm5.acl dict_file = /usr/share/dict/words admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hma c:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:norm al des-cbc-crc:normal }

 

5.創建Kerberos數據庫

[root@ip-172-31-13-38 ~]$ kdb5_util create –r FAYSON.COM -s Loading random data Initializing database '/var/kerberos/krb5kdc/principal' for realm 'FAYSON.COM', master key name 'K/[e-mail]' You will be prompted for the database Master Password. It is important that you NOT FORGET this password. Enter KDC database master key: Re-enter KDC database master key to verify: [root@ip-172-31-13-38 ~]$ 

此處需要輸入Kerberos數據庫的密碼。

6.創建Kerberos的管理賬號

[root@ip-172-31-13-38 ~]$ kadmin.local Authenticating as principal root/[e-mail] with password. kadmin.local: addprinc admin/[e-mail] WARNING: no policy specified for admin/[e-mail]; defaulting to no policy Enter password for principal "admin/[e-mail]": Re-enter password for principal "admin/[e-mail]": Principal "admin/[e-mail]" created. kadmin.local: exit [root@ip-172-31-13-38 ~]$ 

標紅部分為Kerberos管理員賬號，需要輸入管理員密碼。

7.將Kerberos服務添加到自啟動服務，並啟動krb5kdc和kadmin服務

[root@ip-172-31-13-38 ~]$ systemctl enable krb5kdc [root@ip-172-31-13-38 ~]$ systemctl enable kadmin [root@ip-172-31-13-38 ~]$ systemctl start krb5kdc [root@ip-172-31-13-38 ~]$ systemctl start kadmin

8.測試Kerberos的管理員賬號

[root@ip-172-31-13-38 ~]# kinit admin/[e-mail] Password for admin/[e-mail]: [root@ip-172-31-13-38 ~]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: admin/[e-mail] Valid starting Expires Service principal 12/09/2018 02:52:21 12/10/2018 02:52:21 krbtgt/[e-mail] renew until 12/16/2018 02:52:21 

9.為集群安裝所有Kerberos客戶端，包括Cloudera Manager

使用批處理腳本為集群所有節點安裝Kerberos客戶端

[root@ip-172-31-16-68 shell]# sh ssh_do_all.sh node.list 'yum -y install krb5-libs krb5-workstation'

10.在Cloudera Manager Server服務器上安裝額外的包

[root@ip-172-31-13-38 ~]$ yum -y install openldap-clients

11.將KDC Server上的krb5.conf文件拷貝到所有Kerberos客戶端

使用批處理腳本將Kerberos服務端的krb5.conf配置文件拷貝至集群所有節點的/etc目錄下：

[root@ip-172-31-16-68 shell]# sh bk_cp.sh node.list /etc/krb5.conf /etc/

3

CDH集群啟用Kerberos

1.在KDC中給Cloudera Manager添加管理員賬號

[root@ip-172-31-13-38 ~]$ kadmin.local Authenticating as principal root/[e-mail] with password. kadmin.local: addprinc cloudera-scm/[e-mail] WARNING: no policy specified for cloudera-scm/[e-mail]; defaulting to no policy Enter password for principal "cloudera-scm/[e-mail]": Re-enter password for principal "cloudera-scm/[e-mail]": Principal "cloudera-scm/[e-mail]" created. kadmin.local: exit [root@ip-172-31-13-38 ~]$ 

2.進入Cloudera Manager的“管理”à“安全”界面

3.選擇“啟用Kerberos”，進入如下界面

4.確保如下列出的所有檢查項都已完成

5.點擊“繼續”，配置相關的KDC信息，包括類型、KDC服務器、KDC Realm、加密類型以及待創建的Service Principal（hdfs，yarn,，hbase，hive等）的更新生命期等

6.不建議讓Cloudera Manager來管理krb5.conf, 點擊“繼續”

7.輸入Cloudera Manager的Kerbers管理員賬號，一定得和之前創建的賬號一致，點擊“繼續”

8.點擊“繼續”啟用Kerberos

9.Kerberos啟用完成，點擊“繼續”

10.勾選重啟集群，點擊“繼續”

11.集群重啟完成，點擊“繼續”

12.點擊“繼續”

點擊“完成”，至此已成功啟用Kerberos。

4

Kerberos使用

使用fayson用戶運行MapReduce任務及操作Hive，需要在集群所有節點創建fayson用戶。

1.使用kadmin創建一個fayson的principal

[root@ip-172-31-13-38 ~]$ kadmin.local Authenticating as principal root/[e-mail] with password. kadmin.local: addprinc [e-mail] WARNING: no policy specified for [e-mail]; defaulting to no policy Enter password for principal "[e-mail]": Re-enter password for principal "[e-mail]": Principal "[e-mail]" created. kadmin.local: 

2.使用fayson用戶登錄Kerberos

[root@ip-172-31-13-38 ~]$ kdestroy [root@ip-172-31-13-38 ~]$ kinit fayson [root@ip-172-31-13-38 ~]$ klist

3.在集群所有節點添加fayson用戶

使用批量腳本在所有節點添加fayson用戶

[root@ip-172-31-16-68 shell]# sh ssh_do_all.sh node.list "useradd fayson" [root@ip-172-31-16-68 shell]# sh ssh_do_all.sh node.list "id fayson"

4.運行MapReduce作業

[root@ip-172-31-13-38 ~]$ hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-0.20-mapreduce/hadoop-examples.jar pi 10 1

5.使用beeline連接hive進行測試

[root@ip-172-31-13-38 shell]# beeline Beeline version 1.1.0-cdh5.16.1 by Apache Hive beeline> !connect jdbc:hive2://localhost:10000/;principal=hive/[e-mail] scan complete in 1ms Connecting to jdbc:hive2://localhost:10000/;principal=hive/[e-mail] Connected to: Apache Hive (version 1.1.0-cdh5.16.1) Driver: Hive JDBC (version 1.1.0-cdh5.16.1) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://localhost:10000/> show tables; INFO : Compiling command(queryId=hive_20181209031313_dacae423-c8b3-4c81-bf62-d7b0903ec147): show tables INFO : Semantic Analysis Completed INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null) INFO : Completed compiling command(queryId=hive_20181209031313_dacae423-c8b3-4c81-bf62-d7b0903ec147); Time taken: 0.518 seconds INFO : Executing command(queryId=hive_20181209031313_dacae423-c8b3-4c81-bf62-d7b0903ec147): show tables INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20181209031313_dacae423-c8b3-4c81-bf62-d7b0903ec147); Time taken: 0.053 seconds INFO : OK +---------------------+--+ | tab_name | +---------------------+--+ | kudu_decimal_table | +---------------------+--+ 1 row selected (1.933 seconds)

創建一個test表

0: jdbc:hive2://localhost:10000/> create table test(id int, name string);

向test表中插入數據

0: jdbc:hive2://localhost:10000/> insert into test values(1, 'fayson');

執行一個Count語句

0: jdbc:hive2://localhost:10000/> select count(*) from test;

5

常見問題

1.使用Kerberos用戶身份運行MapReduce作業報錯

main : run as user is fayson main : requested yarn user is fayson Requested user fayson is not whitelisted and has id 501,which is below the minimum allowed 1000 Failing this attempt. Failing the application. 17/09/02 20:05:04 INFO mapreduce.Job: Counters: 0 Job Finished in 6.184 seconds java.io.FileNotFoundException: File does not exist: hdfs://ip-172-31-6-148:8020/user/fayson/QuasiMonteCarlo_1504382696029_1308422444/out/reduce-out at org.apache.hadoop.hdfs.DistributedFileSystem$20.doCall(DistributedFileSystem.java:1266) at org.apache.hadoop.hdfs.DistributedFileSystem$20.doCall(DistributedFileSystem.java:1258) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1258) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1820) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1844) at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:314) at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71) at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144) at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

問題原因：是由於Yarn限制了用戶id小於1000的用戶提交作業；

解決方法：修改Yarn的min.user.id來解決

2.進行kinit操作后，執行MR作業報“User fayson not found”

問題原因：在集群的節點上沒有fayson這個用戶

解決方法：需要在集群所有節點添加fayson用戶

6

總結

• 在CDH集群中啟用Kerberos需要先安裝Kerberos服務（krb5kdc和kadmin服務）
• 在集群所有節點需要安裝Kerberos客戶端，用於和kdc服務通信
• 在Cloudera Manager Server節點需要額外安裝openldap-clients包
• CDH集群啟用Kerberos后，使用自己定義的fayson用戶向集群提交作業需確保集群所有節點的操作系統中存在fayson用戶，否則作業會執行失敗