0x01 啟動報錯
1.1 第一種錯誤
2017-02-17 17:25:29,224] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) kafka.common.KafkaException: Failed to acquire lock on file .lock in /var/log/kafka-logs. A Kafka instance in another process or thread is using this directory. at kafka.log.LogManager$$anonfun$lockLogDirs$1.apply(LogManager.scala:100) at kafka.log.LogManager$$anonfun$lockLogDirs$1.apply(LogManager.scala:97) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:35) at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at scala.collection.AbstractTraversable.map(Traversable.scala:104) at kafka.log.LogManager.lockLogDirs(LogManager.scala:97) at kafka.log.LogManager.<init>(LogManager.scala:59) at kafka.server.KafkaServer.createLogManager(KafkaServer.scala:609) at kafka.server.KafkaServer.startup(KafkaServer.scala:183) at io.confluent.support.metrics.SupportedServerStartable.startup(SupportedServerStartable.java:100) at io.confluent.support.metrics.SupportedKafka.main(SupportedKafka.java:49)
解決方法：Failed to acquire lock on file .lock in /var/log/kafka-logs.--問題原因是有其他的進程在使用kafka，ps -ef|grep kafka，殺掉使用該目錄的進程即可；
1.2 第二種錯誤：對index文件無權限
把文件的權限更改為正確的用戶名和用戶組即可；
目錄/var/log/kafka-logs/，其中__consumer_offsets-29是偏移量；
1.3 第三種生產消費報錯:jaas連接有問題
kafka_client_jaas.conf文件配置有問題
16環境上
/opt/dataload/filesource_wangjuan/conf下kafka_client_jaas.conf
KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab="/home/client/keytabs/client.keytab" serviceName="kafka" principal="client/dcp@DCP.COM"; };
0x02 生產者報錯
2.1 第一種：生產者向topic發送消息失敗
[2017-03-09 09:16:00,982] [ERROR] [startJob_Worker-10] [DCPKafkaProducer.java line:62] produceR向topicdf02211發送信息出現異常
org.apache.kafka.common.KafkaException: Failed to construct kafka producer
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:335)
原因是配置文件：kafka_client_jaas.conf中配置有問題，keyTab的路徑不對，導致的；
2.2 第二種：生產消費報錯： Failed to construct kafka producer
報錯關鍵信息：Failed to construct kafka producer
解決方法：配置文件問題：KafkaClient中serviceName應該是kafka，之前配置成了zookeeper；重啟后，就好了；
配置文件如下：
KafkaServer { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true useTicketCache=false serviceName=kafka keyTab="/etc/security/keytabs/kafka.service.keytab" principal="kafka/dcp16@DCP.COM"; }; KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true serviceName=kafka keyTab="/etc/security/keytabs/kafka.service.keytab" principal="kafka/dcp16@DCP.COM"; }; Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true useTicketCache=false serviceName=zookeeper keyTab="/etc/security/keytabs/kafka.service.keytab" principal="kafka/dcp16@DCP.COM"; };
問題描述：
[kafka@DCP16 bin]$ ./kafka-console-producer --broker-list DCP16:9092 --topic topicin050511 --producer.config ../etc/kafka/producer.properties org.apache.kafka.common.KafkaException: Failed to construct kafka producer at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:335) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:188) at kafka.producer.NewShinyProducer.<init>(BaseProducer.scala:40) at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:45) at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala) Caused by: org.apache.kafka.common.KafkaException: java.lang.IllegalArgumentException: Conflicting serviceName values found in JAAS and Kafka configs value in JAAS file zookeeper, value in Kafka config kafka at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:86) at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:70) at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:83) at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:277) ... 4 more Caused by: java.lang.IllegalArgumentException: Conflicting serviceName values found in JAAS and Kafka configs value in JAAS file zookeeper, value in Kafka config kafka at org.apache.kafka.common.security.kerberos.KerberosLogin.getServiceName(KerberosLogin.java:305) at org.apache.kafka.common.security.kerberos.KerberosLogin.configure(KerberosLogin.java:103) at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:45) at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:68) at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:78) ... 7 more [kafka@DCP16 bin]$ ./kafka-console-producer --broker-list DCP16:9092 --topic topicin050511 --producer.config ../etc/kafka/producer.properties
2.3 消費時報錯: ERROR Unknown error when running consumer: (kafka.tools.ConsoleConsumer$)
[root@DCP16 bin]# ./kafka-console-consumer --zookeeper dcp18:2181,dcp16:2181,dcp19:2181/kafkakerberos --from-beginning --topic topicout050511 --new-consumer --consumer.config ../etc/kafka/consumer.properties --bootstrap-server DCP16:9092 [2017-05-07 22:24:37,479] ERROR Unknown error when running consumer: (kafka.tools.ConsoleConsumer$) org.apache.kafka.common.KafkaException: Failed to construct kafka consumer at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:702) at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:587) at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:569) at kafka.consumer.NewShinyConsumer.<init>(BaseConsumer.scala:53) at kafka.tools.ConsoleConsumer$.run(ConsoleConsumer.scala:64) at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:51) at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala) Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:86) at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:70) at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:83) at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:623) ... 6 more Caused by: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:899) at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:719) at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:584) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762) at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687) at javax.security.auth.login.LoginContext.login(LoginContext.java:595) at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:69) at org.apache.kafka.common.security.kerberos.KerberosLogin.login(KerberosLogin.java:110) at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:46) at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:68) at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:78)
衍生問題:
kafka生產消息就會報錯：
[2017-05-07 23:17:16,240] ERROR Error when sending message to topic topicin050511 with key: null, value: 0 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
把KafkaClient更改為如下的配置，就可以 了：
KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useTicketCache=true; };
0x03消費者報錯
3.1 第一種錯誤：replication factor: 1 larger than available brokers: 0
消費時報錯：Error while executing topic command : replication factor: 1 larger than available brokers: 0
解決辦法：/confluent-3.0.0/bin 下重啟daemon
./kafka-server-stop -daemon ../etc/kafka/server.properties
./kafka-server-start -daemon ../etc/kafka/server.properties
然后zk重啟;sh zkCli.sh -server ai186;
/usr/hdp/2.4.2.0-258/zookeeper/bin/zkCli.sh --腳本的目錄
如果還報錯，可以查看配置文件中下面的配置：
zookeeper.connect=dcp18:2181/kafkakerberos； --是group名稱
3.2 第二種錯誤：TOPIC_AUTHORIZATION_FAILED
./bin/kafka-console-consumer --zookeeper DCP185:2181,DCP186:2181,DCP187:2181/kafka --from-beginning --topic wangjuan_topic1 --new-consumer --consumer.config ./etc/kafka/consumer.properties --bootstrap-server DCP187:9092 [2017-03-02 13:44:38,398] WARN The configuration zookeeper.connection.timeout.ms = 6000 was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig) [2017-03-02 13:44:38,575] WARN Error while fetching metadata with correlation id 1 : {wangjuan_topic1=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient) [2017-03-02 13:44:38,677] WARN Error while fetching metadata with correlation id 2 : {wangjuan_topic1=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient) [2017-03-02 13:44:38,780] WARN Error while fetching metadata with correlation id 3 : {wangjuan_topic1=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)
解決方法：配置文件中下面的參數中的User的U必須是大寫；
super.users=User:kafka
或者有可能是server.properties中的adver.listen的IP是不對的，有可能是代碼中寫死的IP；
3.3 第三種錯誤的可能的解決方法：
無法消費，則查看kafka的啟動日志中的報錯信息：日志文件的所屬組不對，應該是hadoop；
或者，查看kafka對應的zookeeper的配置后綴，是否已經更改，如果更改了，則topic需要重新生成才行；
3.4 第四種錯誤：消費的tomcat報錯
[2017-04-01 06:37:21,823] [INFO] [Thread-5] [AbstractCoordinator.java line:542] Marking the coordinator DCP187:9092 (id: 2147483647 rack: null) dead for group test-consumer-group [2017-04-01 06:37:21,825] [WARN] [Thread-5] [ConsumerCoordinator.java line:476] Auto offset commit failed for group test-consumer-group: Commit offsets failed with retriable exception. You should retry committing offsets.
更改代碼中，tomcat的心跳超時時間如下：
沒有改之前的:;
./webapps/web/WEB-INF/classes/com/ai/bdx/dcp/hadoop/service/impl/DCPKafkaConsumer.class;
重啟后，日志中顯示：
[2017-04-01 10:14:56,167] [INFO] [Thread-5] [AbstractCoordinator.java line:542] Marking the coordinator DCP187:9092 (id: 2147483647 rack: null) dead for group test-consumer-group [2017-04-01 10:14:56,286] [INFO] [Thread-5] [AbstractCoordinator.java line:505] Discovered coordinator DCP187:9092 (id: 2147483647 rack: null) for group test-consumer-group.
0x04 創建topic時錯誤
創建topic時報錯：
[2017-04-10 10:32:23,776] WARN SASL configuration failed: javax.security.auth.login.LoginException: Checksum failed Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. (org.apache.zookeeper.ClientCnxn) Exception in thread "main" org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:946) at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:923) at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1230) at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:156) at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:130) at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:75) at kafka.utils.ZkUtils$.apply(ZkUtils.scala:57) at kafka.admin.TopicCommand$.main(TopicCommand.scala:54) at kafka.admin.TopicCommand.main(TopicCommand.scala)
問題定位：是jaas文件有問題：
解決方法：server.properties文件中的super.user要和jaas文件中的keytab的principle一致；
server.properties:super.users=User:client
kafka_server_jaas.conf文件改為：
KafkaServer { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true useTicketCache=false serviceName=kafka keyTab="/data/data1/confluent-3.0.0/kafka.keytab" principal="kafka@DCP.COM"; }; KafkaClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab="/home/client/client.keytab" principal="client/DCP187@DCP.COM"; }; Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true useTicketCache=false serviceName=zookeeper keyTab="/home/client/client.keytab" principal="client/DCP187@DCP.COM"; };