環境:
mac系統 + hadoop2.6.0-cdh5.7.0偽分布式
問題一:
在啟動hdfs的secondarynamenode時,報錯。
正常情況:
sumengdeMacBook-Pro:sbin sumeng$ ./start-dfs.sh
18/06/11 21:35:00 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Starting namenodes on [localhost] localhost: starting namenode, logging to /Users/sumeng/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-sumeng-namenode-sumengdeMacBook-Pro.local.out localhost: starting datanode, logging to /Users/sumeng/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-sumeng-datanode-sumengdeMacBook-Pro.local.out Starting secondary namenodes [0.0.0.0] 0.0.0.0: starting secondarynamenode, logging to /Users/sumeng/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-sumeng-secondarynamenode-sumengdeMacBook-Pro.local.out 18/06/11 21:35:15 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
報錯:
sumengdeMacBook-Pro:sbin sumeng$ ./start-dfs.sh
18/06/11 21:42:19 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Starting namenodes on [localhost] localhost: starting namenode, logging to /Users/sumeng/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-sumeng-namenode-sumengdeMacBook-Pro.local.out localhost: starting datanode, logging to /Users/sumeng/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-sumeng-datanode-sumengdeMacBook-Pro.local.out Starting secondary namenodes [account.jetbrains.com]
(卡在了這里,一直啟動不了) account.jetbrains.com: ssh_exchange_identification: Connection closed by remote host
分析:
/etc/hosts/中,我在破解idea中,加了一行
0.0.0.0 account.jetbrains.com
我猜是由他引起的
方法一:
注釋掉這句話之后,就不再報這個錯誤,secondarynamenode順利啟動
方法二:
在hdfs-site.xml添加
<property> <name>dfs.secondary.http.address</name> <value>127.0.0.1:50090</value> </property>
因為不寫的話默認是0.0.0.0:50090,改為127.0.0.1就可以
問題二:
在啟動hadoop時,提示警告
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable、
方法:
https://blog.csdn.net/u014039577/article/details/49816907(里面有免費的包)
https://blog.csdn.net/u013310025/article/details/52853941
只是查了這些資料,沒有實踐