hadoop 2.x安裝:不能加載本地庫 - java.library.path錯誤


我們在啟動hadoop2.x是可能會有下面這個警告:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

這個警告實際上就是不能加載本地庫。因為這個錯誤的原因有很多,因此這里給出一種方案

1. 在hadoop-env.sh中加入DEBUG
現在我們無從判定問題在哪里,因此我們在tiny1(master)的hadoop-env.sh中加入DEBUG,即在該文件下加入這一行:

export HADOOP_ROOT_LOGGER=DEBUG,console

2.重啟hadoop查看日志輸出

17/03/09 18:49:14 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
17/03/09 18:49:14 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
17/03/09 18:49:14 DEBUG util.NativeCodeLoader: java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
17/03/09 18:49:14 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

日志中顯示他的 java.library.path是hadoop-2.7.2/lib,但是他的實際位置是hadoop-2.7.2/lib/native

3.修改hadoop-env.sh解決當前問題
添加:

export HADOOP_COMMON_LIB_NATIVE_DIR=/home/hadoop/hadoop/hadoop-2.7.2/lib/native

修改

export HADOOP_OPTS="-Djava.library.path=/home/hadoop/hadoop/hadoop-2.7.2/lib/native"

至此我們在hadoop-env.sh中的三個添加或修改的配置文件如下:

export HADOOP_COMMON_LIB_NATIVE_DIR=/home/hadoop/hadoop/hadoop-2.7.2/lib/native
export HADOOP_OPTS="-Djava.library.path=/home/hadoop/hadoop/hadoop-2.7.2/lib/native"
export HADOOP_ROOT_LOGGER=DEBUG,console

4.重啟hadoop
我們關閉hadoop並重新開啟hadoop

hadoop/hadoop-2.7.2/sbin/stop-all.sh
hadoop/hadoop-2.7.2/sbin/start-all.sh

我們查看日志:

17/03/09 19:11:22 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
17/03/09 19:11:22 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /home/hadoop/hadoop/hadoop-2.7.2/lib/native/libhadoop.so.1.0.0: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/hadoop/hadoop/hadoop-2.7.2/lib/native/libhadoop.so.1.0.0)
17/03/09 19:11:22 DEBUG util.NativeCodeLoader: java.library.path=/home/hadoop/hadoop/hadoop-2.7.2/lib/native
17/03/09 19:11:22 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

還是有錯...錯誤信息為version `GLIBC_2.14' not found,我們在另外一篇中解決


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM