namenode和datanode啟動失敗


1.namenode啟動失敗,查看錯誤原因,是無法格式化,再看日志,根據日志提示,清空對應的目錄,即可解決這個問題。

 

2.datanode啟動失敗:

Can't open /var/run/cloudera-scm-agent/process/261-hdfs-DATANODE/supervisor.conf: Permission denied.

 

cd到/var/run/cloudera-scm-agent/process/261-hdfs-DATANODE目錄下,看到:

-rw------- 1 root root 3146  2月  5 18:03 2018 supervisor.conf

懷疑是權限問題,上網查詢了之后,才知道看起來是權限問題,其實不是,需要看看具體的datanode啟動日志,於是到/var/log/hadoop-hdfs/目錄下,清空目錄,重新啟動一下datanode,查看新生成的log日志,如下:

2018-02-05 18:29:10,370 WARN org.apache.hadoop.hdfs.server.common.Storage: java.io.IOException: Incompatible clusterIDs in /dfs/dn: namenode clusterID = cluster10; datanode
 clusterID = cluster20
2018-02-05 18:29:10,371 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid 5c80b393-7d62-4016-bafa-c44
63af7a45f) service to iot1/172.30.1.245:8022. Exiting. 
java.io.IOException: All specified directories are failed to load.
        at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:478)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1394)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1355)
        at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:317)
        at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:228)
        at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:829)
        at java.lang.Thread.run(Thread.java:745)
2018-02-05 18:29:10,378 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid 5c80b393-7d62-4016-bafa
-c4463af7a45f) service to iot1/172.30.1.245:8022

  

日志里有這樣一行:

java.io.IOException: Incompatible clusterIDs in /dfs/dn: namenode clusterID = cluster10; datanode clusterID = cluster20

 

這就是namenode的clientID與datanode的clientID不一致的錯誤。

 

解決辦法就是把datanode的clientID改的和namenode一致

 

datanode的clientID記錄在/dfs/dn/current/VERSION文件中(即配置文件中dfs.data.dir在本地系統的路徑的current/VERSION文件,dfs.data.dir這個目錄可以在ClouderaManager的管理頁面查找到)

 

最后修改的一致就解決問題了

# 打可執行jar包mvn package assembly:single
Can't open /var/run/cloudera-scm-agent/process/261-hdfs-DATANODE/supervisor.conf: Permission denied.
cd到/var/run/cloudera-scm-agent/process/261-hdfs-DATANODE目錄下,看到:


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM