一、Kerberos
二、安裝
node01服務器安裝Kerberos的核心服務master KDC，node02和node03安裝Kerberos client
cm也安裝在node01上了
1.master節點配置
在node01上
yum install krb5-server krb5-libs krb5-auth-dialog krb5-workstation
修改配置文件，/etc/krb5.conf
[logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] default_realm = LOCAL.DOMAIN dns_lookup_realm = false dns_lookup_kdc = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true [realms] LOCAL.DOMAIN = { kdc = node01 admin_server = node01 } [domain_realm] .local.domain = LOCAL.DOMAIN local.domain = LOCAL.DOMAIN
修改配置文件，/var/kerberos/krb5kdc/kadm5.acl
*/admin@LOCAL.DOMAIN *
修改配置文件， /var/kerberos/krb5kdc/kdc.conf
把aes256-cts去掉,不去掉則要增加jar包
[kdcdefaults] kdc_ports = 88 kdc_tcp_ports = 88 [realms] EXAMPLE.COM = { #master_key_type = aes256-cts acl_file = /var/kerberos/krb5kdc/kadm5.acl dict_file = /usr/share/dict/words admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab supported_enctypes = aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal }
2.創建/初始化Kerberos
1)創建/初始化Kerberos數據庫，kdb5_util create -s –r LOCAL.DOMAIN ，並設置密碼
[-s]表示生成stash file，並在其中存儲master server key（krb5kdc）；
[-r]來指定一個realm name，當krb5.conf中定義了多個realm時才是必要的。
保存路徑為/var/Kerberos/krb5kdc 如果需要重建數據庫，將該目錄下的含有principal的文件全都刪除即可
[root@node01 ~]# kdb5_util create –r LOCAL.DOMAIN -s Loading random data Initializing database '/var/Kerberos/krb5kdc/principal' for realm 'LOCAL.DOMAIN', master key name 'K/M@LOCAL.DOMAIN' You will be prompted for the database Master Password. It is important that you NOT FORGET this password. Enter KDC database master key: Re-enter KDC database master key to verify:
兩次輸入密碼
2)創建Kerberos的管理賬號，兩次輸入密碼
[root@node01 ~]# kadmin.local Authenticating as principal root/admin@LOCAL.DOMAIN with password. kadmin.local: addprinc admin/admin@LOCAL.DOMAIN WARNING: no policy specified for admin/admin@LOCAL.DOMAIN; defaulting to no policy Enter password for principal "admin/admin@LOCAL.DOMAIN": Re-enter password for principal "admin/admin@LOCAL.DOMAIN": Principal "admin/admin@LOCAL.DOMAIN" created. kadmin.local: kadmin.local: exit
3.安裝Kerberos客戶端
1)給集群所有節點安裝Kerberos客戶端
node02和node03
[root@node02 ~]# yum -y install krb5-workstation krb5-libs krb5-auth-dialog Installed: krb5-workstation.x86_64 0:1.10.3-65.el6 Dependency Installed: libkadm5.x86_64 0:1.10.3-65.el6 Updated: krb5-libs.x86_64 0:1.10.3-65.el6 Dependency Updated: krb5-devel.x86_64 0:1.10.3-65.el6 Complete!
2)CM節點安裝額外組件
root@node01 ~]# yum -y install openldap-clients Running Transaction Test Transaction Test Succeeded Running Transaction Updating : openldap-2.4.40-16.el6.x86_64 1/3 Installing : openldap-clients-2.4.40-16.el6.x86_64 2/3 Cleanup : openldap-2.4.23-31.el6.x86_64 3/3 Verifying : openldap-clients-2.4.40-16.el6.x86_64 1/3 Verifying : openldap-2.4.40-16.el6.x86_64 2/3 Verifying : openldap-2.4.23-31.el6.x86_64 3/3 Installed: openldap-clients.x86_64 0:2.4.40-16.el6 Dependency Updated: openldap.x86_64 0:2.4.40-16.el6 Complete!
3)拷貝配置文件，將KDC Server上的krb5.conf文件拷貝到所有Kerberos客戶端(集群所有節點)
將node01上的/etc/krb5.conf，利用scp等命令分發到node02和node03
4.CDH集群啟用Kerberos
1)在KDC中給Cloudera Manager添加管理員賬號,並設置密碼
root@node01 ~]# kadmin.local Authenticating as principal admin/admin@LOCAL.DOMAIN with password. kadmin.local: addprinc cloudera-scm/admin@LOCAL.DOMAIN WARNING: no policy specified for cloudera-scm/admin@LOCAL.DOMAIN; defaulting to no policy Enter password for principal "cloudera-scm/admin@LOCAL.DOMAIN": Re-enter password for principal "cloudera-scm/admin@LOCAL.DOMAIN": Principal "cloudera-scm/admin@LOCAL.DOMAIN" created. kadmin.local: exit
CDH啟用Kerberos
2)進入Cloudera Manager，集群，操作，啟用kerberos
3)檢查信息，勾選
4)KDC信息
5)不建議讓Cloudera Manager來管理krb5.conf，點擊“繼續”
6) 輸入CM的Kerbers管理員賬號
7)Kerberos主體
8) 重啟集群
使用HDFS時，由於票據過期出錯，使用kinit重新登錄Cloudera Manager管理員賬號即可
[root@node01 ~]# hadoop fs -ls / 19/11/08 07:37:12 WARN security.UserGroupInformation: Exception encountered while running the renewal command for cloudera-scm/admin@LOCAL.DOMAIN. (TGT end time:1572934862000, renewalFailures: org.apache.hadoop.metrics2.lib.MutableGaugeInt@66f06ac9,renewalFailuresTotal: org.apache.hadoop.metrics2.lib.MutableGaugeLong@23f2e873) ExitCodeException exitCode=1: kinit: Ticket expired while renewing credentials at org.apache.hadoop.util.Shell.runCommand(Shell.java:601) at org.apache.hadoop.util.Shell.run(Shell.java:504) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:786) at org.apache.hadoop.util.Shell.execCommand(Shell.java:879) at org.apache.hadoop.util.Shell.execCommand(Shell.java:862) at org.apache.hadoop.security.UserGroupInformation$1.run(UserGroupInformation.java:1020) at java.lang.Thread.run(Thread.java:748) 19/11/08 07:37:12 ERROR security.UserGroupInformation: TGT is expired. Aborting renew thread for cloudera-scm/admin@LOCAL.DOMAIN. 19/11/08 07:37:12 WARN security.UserGroupInformation: PriviledgedActionException as:cloudera-scm/admin@LOCAL.DOMAIN (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 19/11/08 07:37:14 WARN security.UserGroupInformation: PriviledgedActionException as:cloudera-scm/admin@LOCAL.DOMAIN (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 19/11/08 07:37:14 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 60 seconds before. Last Login=1573169832802 ^Z [1]+ Stopped hadoop fs -ls / [root@node01 ~]# kinit cloudera-scm/admin@LOCAL.DOMAIN Password for cloudera-scm/admin@LOCAL.DOMAIN: [root@node01 ~]# hadoop fs -ls / Found 2 items drwxrwxrwt - hdfs supergroup 0 2019-10-29 19:11 /tmp drwxr-xr-x - hdfs supergroup 0 2019-11-07 22:13 /user
kafka
安裝
配置
[root@node01 zookeeper]# /opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/kafka-topics --zookeeper node02:2181 --list
19/11/08 10:50:46 INFO utils.Log4jControllerRegistration$: Registered kafka:type=kafka.Log4jController MBean 19/11/08 10:50:46 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Initializing a new session to node02:2181. 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-cdh5.14.2--1, built on 03/27/2018 20:39 GMT 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:host.name=node01 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_231 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.home=/bigdata/jdk1.8.0_231/jre 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/activation-1.1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/activation-1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/aopalliance-1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/aopalliance-repackaged-2.5.0-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/apacheds-i18n-2.0.0-M15.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/apacheds-jdbm1-2.0.0-M2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/api-asn1-api-1.0.0-M20.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/api-util-1.0.0-M20.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/argparse4j-0.7.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/asm-3.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/avro-1.7.6-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/caffeine-2.7.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/cglib-2.2.1-v20090111.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/checker-qual-2.6.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-beanutils-1.8.3.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-beanutils-core-1.8.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-cli-1.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-codec-1.9.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-collections-3.2.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-compress-1.4.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-configuration-1.6.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-digester-1.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-el-1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-httpclient-3.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-io-2.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-lang-2.6.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-lang3-3.5.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-logging-1.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-math3-3.1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-net-3.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-pool2-2.4.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/compileScala.mapping:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/connect-api-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/connect-basic-auth-extension-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/connect-file-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/connect-json-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/connect-runtime-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/connect-transforms-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/error_prone_annotations-2.3.3.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/gson-2.2.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/guava-20.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/guice-3.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/guice-servlet-3.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-annotations-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-archives-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-auth-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-common-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-common-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-core-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-shuffle-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-yarn-api-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-yarn-client-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-yarn-common-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-yarn-server-common-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-yarn-server-nodemanager-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hive-hcatalog-core-1.1.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hive-hcatalog-server-extensions-1.1.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hk2-api-2.5.0-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hk2-locator-2.5.0-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hk2-utils-2.5.0-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/htrace-core4-4.0.1-incubating.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/httpclient-4.4.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/httpcore-4.4.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-annotations-2.9.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-annotations-2.9.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-core-2.9.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-core-asl-1.9.13-cloudera.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-databind-2.9.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-jaxrs-1.8.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-jaxrs-base-2.9.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-jaxrs-json-provider-2.9.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-mapper-asl-1.9.13-cloudera.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-module-jaxb-annotations-2.9.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-xc-1.8.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javassist-3.22.0-CR2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javax.annotation-api-1.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javax.inject-1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javax.inject-2.5.0-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/java-xmlbuilder-0.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javax.ws.rs-api-2.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jaxb-api-2.2.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jaxb-api-2.3.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-client-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-common-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-container-servlet-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-container-servlet-core-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-guice-1.9.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-hk2-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-media-jaxb-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-server-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jets3t-0.9.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jettison-1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-client-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-continuation-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-http-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-io-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-security-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-server-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-servlet-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-servlets-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-util-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-util-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jopt-simple-5.0.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jsch-0.1.42.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jsp-api-2.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jsr305-3.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka_2.11-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka_2.11-2.1.0-kafka-4.0.0-sources.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-clients-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-log4j-appender-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-streams-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-streams-examples-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-streams-scala_2.11-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-streams-test-utils-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-tools-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/leveldbjni-all-1.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/libthrift-0.9.3.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/log4j-1.2.17.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/lz4-java-1.5.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/maven-artifact-3.5.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/metrics-core-2.2.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/metrics-servlet-2.2.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/netty-3.10.5.Final.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/osgi-resource-locator-1.0.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/paranamer-2.3.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/plexus-utils-3.1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/reflections-0.9.11.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/rocksdbjni-5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/scala-library-2.11.12.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/scala-logging_2.11-3.9.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/scala-reflect-2.11.12.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-binding-hive-conf-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-binding-hive-follower-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-binding-kafka-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-core-common-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-core-model-db-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-core-model-indexer-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-core-model-kafka-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-hdfs-common-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-policy-common-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-policy-indexer-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-policy-kafka-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-provider-common-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-provider-db-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-provider-file-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/servlet-api-2.5.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/shiro-core-1.2.3.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/slf4j-api-1.7.25.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/slf4j-api-1.7.5.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/slf4j-log4j12-1.7.5.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/snappy-java-1.1.7.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/stax-api-1.0-2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/validation-api-1.1.0.Final.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/xmlenc-0.52.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/xz-1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/zkclient-0.10.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/zookeeper-3.4.5-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/zstd-jni-1.3.5-4.jar:/etc/kafka/conf/sentry-conf 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-696.16.1.el6.x86_64 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:user.name=root 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:user.home=/root 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:user.dir=/etc/zookeeper 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=node02:2181 sessionTimeout=30000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@67c27493 19/11/08 10:50:46 INFO zookeeper.ClientCnxn: Opening socket connection to server node02/xxxxxx:2181. Will not attempt to authenticate using SASL (unknown error) 19/11/08 10:50:46 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Waiting until connected. 19/11/08 10:50:46 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /172.16.221.xx:35396, server: node02/172.16.237.xx:2181 19/11/08 10:50:46 INFO zookeeper.ClientCnxn: Session establishment complete on server node02/172.16.237.xx:2181, sessionid = 0x16e46647cc30394, negotiated timeout = 30000 19/11/08 10:50:46 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Connected. topic_start 19/11/08 10:50:46 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Closing. 19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Session: 0x16e46647cc30394 closed 19/11/08 10:50:46 INFO zookeeper.ClientCnxn: EventThread shut down 19/11/08 10:50:46 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Closed.
啟用Kerberos
修改security.inter.broker.protocol
重啟kafka服務完成以上配置，Kafka集群已啟用Kerberos認證
在各個節點上：
配置jaas.conf文件
[root@node01 kafka_client]# pwd /usr/local/kafka_client #創建文件 [root@node01 kafka_client]# vi jaas.conf
內容如下
KafkaClient{ com.sun.security.auth.module.Krb5LoginModule required useTicketCache=true; };
配置client.properties文件
[root@node01 kafka_client]# vi client.properties
內容如下
security.protocol=SASL_PLAINTEXT sasl.kerberos.service.name=kafka
初始化kerberos賬號
[root@node01 kafka_client]# kinit cloudera-scm/admin@LOCAL.DOMAIN Password for cloudera-scm/admin@LOCAL.DOMAIN:
不要忘了導入變量，否則會報錯
Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set
找不到jaas配置文件
在KAFKA_OPTS變量里加上" -Djava.security.auth.login.config=/path/to/kafka_server_jaas.conf"
export KAFKA_OPTS="-Djava.security.auth.login.config=/usr/local/kafka_client/jaas.conf"
根據所配置的配置文件啟動client
啟動生產端
[root@node02 kafka_client]# kafka-console-producer --broker-list node01:9092,node02:9092,node03:9092 --topic kerbero --producer.config client.properties 19/11/08 14:01:19 INFO utils.Log4jControllerRegistration$: Registered kafka:type=kafka.Log4jController MBean 19/11/08 14:01:19 INFO producer.ProducerConfig: ProducerConfig values: acks = 1 batch.size = 16384 bootstrap.servers = [node01:9092, node02:9092, node03:9092] buffer.memory = 33554432 client.dns.lookup = default client.id = console-producer compression.type = none connections.max.idle.ms = 540000 delivery.timeout.ms = 120000 enable.idempotence = false interceptor.classes = [] key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer linger.ms = 1000 max.block.ms = 60000 max.in.flight.requests.per.connection = 5 max.request.size = 1048576 metadata.max.age.ms = 300000 metric.reporters = [] metrics.num.samples = 2 metrics.recording.level = INFO metrics.sample.window.ms = 30000 partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner receive.buffer.bytes = 32768 reconnect.backoff.max.ms = 1000 reconnect.backoff.ms = 50 request.timeout.ms = 1500 retries = 3 retry.backoff.ms = 100 sasl.client.callback.handler.class = null sasl.jaas.config = null sasl.kerberos.kinit.cmd = /usr/bin/kinit sasl.kerberos.min.time.before.relogin = 60000 sasl.kerberos.service.name = kafka sasl.kerberos.ticket.renew.jitter = 0.05 sasl.kerberos.ticket.renew.window.factor = 0.8 sasl.login.callback.handler.class = null sasl.login.class = null sasl.login.refresh.buffer.seconds = 300 sasl.login.refresh.min.period.seconds = 60 sasl.login.refresh.window.factor = 0.8 sasl.login.refresh.window.jitter = 0.05 sasl.mechanism = GSSAPI security.protocol = SASL_PLAINTEXT send.buffer.bytes = 102400 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = null ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS transaction.timeout.ms = 60000 transactional.id = null value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer 19/11/08 14:01:19 INFO authenticator.AbstractLogin: Successfully logged in. 19/11/08 14:01:19 INFO kerberos.KerberosLogin: [Principal=null]: TGT refresh thread started. 19/11/08 14:01:19 INFO kerberos.KerberosLogin: [Principal=null]: TGT valid starting at: Fri Nov 08 14:00:29 CST 2019 19/11/08 14:01:19 INFO kerberos.KerberosLogin: [Principal=null]: TGT expires: Sat Nov 09 14:00:29 CST 2019 19/11/08 14:01:19 WARN kerberos.KerberosLogin: The TGT cannot be renewed beyond the next expiry date: Sat Nov 09 14:00:29 CST 2019.This process will not be able to authenticate new SASL connections after that time (for example, it will not be able to authenticate a new connection with a Kafka Broker). Ask your system administrator to either increase the 'renew until' time by doing : 'modprinc -maxrenewlife null ' within kadmin, or instead, to generate a keytab for null. Because the TGT's expiry cannot be further extended by refreshing, exiting refresh thread now. 19/11/08 14:01:20 INFO utils.AppInfoParser: Kafka version : 2.1.0-kafka-4.0.0 19/11/08 14:01:20 INFO utils.AppInfoParser: Kafka commitId : unknown >19/11/08 14:06:20 INFO clients.Metadata: Cluster ID: 9EbFfkQERomQdy0wrndVjQ >hello >python >hello
啟動消費端
[root@node03 kafka_client]# kafka-console-consumer --topic kerbero --from-beginning --bootstrap-server node01:9092,node01:9092,node03:9092 --consumer.config client.properties 19/11/08 14:01:56 INFO utils.Log4jControllerRegistration$: Registered kafka:type=kafka.Log4jController MBean 19/11/08 14:01:57 INFO consumer.ConsumerConfig: ConsumerConfig values: auto.commit.interval.ms = 5000 auto.offset.reset = earliest bootstrap.servers = [node01:9092, node01:9092, node03:9092] check.crcs = true client.dns.lookup = default client.id = connections.max.idle.ms = 540000 default.api.timeout.ms = 60000 enable.auto.commit = false exclude.internal.topics = true fetch.max.bytes = 52428800 fetch.max.wait.ms = 500 fetch.min.bytes = 1 group.id = console-consumer-35064 heartbeat.interval.ms = 3000 interceptor.classes = [] internal.leave.group.on.close = true isolation.level = read_uncommitted key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer max.partition.fetch.bytes = 1048576 max.poll.interval.ms = 300000 max.poll.records = 500 metadata.max.age.ms = 300000 metric.reporters = [] metrics.num.samples = 2 metrics.recording.level = INFO metrics.sample.window.ms = 30000 partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor] receive.buffer.bytes = 65536 reconnect.backoff.max.ms = 1000 reconnect.backoff.ms = 50 request.timeout.ms = 30000 retry.backoff.ms = 100 sasl.client.callback.handler.class = null sasl.jaas.config = null sasl.kerberos.kinit.cmd = /usr/bin/kinit sasl.kerberos.min.time.before.relogin = 60000 sasl.kerberos.service.name = kafka sasl.kerberos.ticket.renew.jitter = 0.05 sasl.kerberos.ticket.renew.window.factor = 0.8 sasl.login.callback.handler.class = null sasl.login.class = null sasl.login.refresh.buffer.seconds = 300 sasl.login.refresh.min.period.seconds = 60 sasl.login.refresh.window.factor = 0.8 sasl.login.refresh.window.jitter = 0.05 sasl.mechanism = GSSAPI security.protocol = SASL_PLAINTEXT send.buffer.bytes = 131072 session.timeout.ms = 10000 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = null ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer 19/11/08 14:01:57 INFO authenticator.AbstractLogin: Successfully logged in. 19/11/08 14:01:57 INFO kerberos.KerberosLogin: [Principal=null]: TGT refresh thread started. 19/11/08 14:01:57 INFO kerberos.KerberosLogin: [Principal=null]: TGT valid starting at: Fri Nov 08 14:00:49 CST 2019 19/11/08 14:01:57 INFO kerberos.KerberosLogin: [Principal=null]: TGT expires: Sat Nov 09 14:00:49 CST 2019 19/11/08 14:01:57 WARN kerberos.KerberosLogin: The TGT cannot be renewed beyond the next expiry date: Sat Nov 09 14:00:49 CST 2019.This process will not be able to authenticate new SASL connections after that time (for example, it will not be able to authenticate a new connection with a Kafka Broker). Ask your system administrator to either increase the 'renew until' time by doing : 'modprinc -maxrenewlife null ' within kadmin, or instead, to generate a keytab for null. Because the TGT's expiry cannot be further extended by refreshing, exiting refresh thread now. 19/11/08 14:01:57 INFO utils.AppInfoParser: Kafka version : 2.1.0-kafka-4.0.0 19/11/08 14:01:57 INFO utils.AppInfoParser: Kafka commitId : unknown 19/11/08 14:01:57 INFO clients.Metadata: Cluster ID: 9EbFfkQERomQdy0wrndVjQ 19/11/08 14:01:58 INFO internals.AbstractCoordinator: [Consumer clientId=consumer-1, groupId=console-consumer-35064] Discovered group coordinator node02:9092 (id: 2147483592 rack: null) 19/11/08 14:01:58 INFO internals.ConsumerCoordinator: [Consumer clientId=consumer-1, groupId=console-consumer-35064] Revoking previously assigned partitions [] 19/11/08 14:01:58 INFO internals.AbstractCoordinator: [Consumer clientId=consumer-1, groupId=console-consumer-35064] (Re-)joining group 19/11/08 14:02:01 INFO internals.AbstractCoordinator: [Consumer clientId=consumer-1, groupId=console-consumer-35064] Successfully joined group with generation 1 19/11/08 14:02:01 INFO internals.ConsumerCoordinator: [Consumer clientId=consumer-1, groupId=console-consumer-35064] Setting newly assigned partitions [kerbero-0] 19/11/08 14:02:01 INFO internals.Fetcher: [Consumer clientId=consumer-1, groupId=console-consumer-35064] Resetting offset for partition kerbero-0 to offset 0. hello python hello
JAAS 是個什么梗
hue啟動報Kerberos Ticket Renewer已停止
解決:
原因:kerberos憑證過期;
先進入kerberos模式:
kadmin.local命令然后,然后操作下面的
kadmin.local: modprinc -maxrenewlife 90day krbtgt/YOUR_REALM.COM
kadmin.local: modprinc -maxrenewlife 90day +allow_renewable hue/<hostname>@YOUR-REALM.COM
命令出處:http://t.cn/R8ttGKM
http://web.mit.edu/kerberos/krb5-1.12/doc/admin/admin_commands/kadmin_local.html
kadmin [-O|-N] [-r realm] [-p principal] [-q query] [[-c cache_name]|[-k [-t keytab]]|-n] [-w password] [-s admin_server[:port]]
kadmin.local [-r realm] [-p principal] [-q query] [-d dbname] [-e enc:salt ...] [-m] [-x db_args]
DESCRIPTION
kadmin and kadmin.local are command-line interfaces to the Kerberos V5 administration system. They provide nearly identical functionalities; the difference is that kadmin.local directly accesses the KDC database, while kadmin performs operations using kadmind. Except as explicitly noted otherwise, this man page will use “kadmin” to refer to both versions. kadmin provides for the maintenance of Kerberos principals, password policies, and service key tables (keytabs).
The remote kadmin client uses Kerberos to authenticate to kadmind using the service principal kadmin/ADMINHOST (where ADMINHOST is the fully-qualified hostname of the admin server) or kadmin/admin. If the credentials cache contains a ticket for one of these principals, and the -c credentials_cache option is specified, that ticket is used to authenticate to kadmind. Otherwise, the -p and -k options are used to specify the client Kerberos principal name used to authenticate. Once kadmin has determined the principal name, it requests a service ticket from the KDC, and uses that service ticket to authenticate to kadmind.
Since kadmin.local directly accesses the KDC database, it usually must be run directly on the master KDC with sufficient permissions to read the KDC database. If the KDC database uses the LDAP database module, kadmin.local can be run on any host which can access the LDAP server.
kadmin.local //以超管身份進入kadmin kadmin //進入kadmin模式，需輸入密碼 kdb5_util create -r JENKIN.COM -s //創建數據庫 service krb5kdc start //啟動kdc服務 service kadmin start //啟動kadmin服務 service kprop start //啟動kprop服務 kdb5_util dump /var/kerberos/krb5kdc/slave_data //生成dump文件 kprop -f /var/kerberos/krb5kdc/slave_data master2.com //將master數據庫同步是slave kadmin模式下： addprinc -randkey root/master1@JENKIN.COM //生成隨機key的principal addprinc admin/admin //生成指定key的principal listprincs //查看principal change_password -pw xxxx admin/admin //修改admin/admin的密碼 delete_principal admin/admin //刪除principal kinit admin/admin //驗證principal是否可用 xst -norandkey -k /var/kerberos/krb5kdc/keytab/root.keytab root/master1@JENKIN.COM host/master1@JENKIN.COM //為principal生成keytab，可同時添加多個 ktadd -k /etc/krb5.keytab host/master1@JENKIN.COM //ktadd也可生成keytab kinit -k -t /var/kerberos/krb5kdc/keytab/root.keytab root/master1@JENKIN.COM //測試keytab是否可用 klist -e -k -t /var/kerberos/krb5kdc/keytab/root.keytab //查看keytab