今天為Hadoop配置zeromq、jzmq遇到各種問題,先是編譯出錯,到編譯成功后測試出錯等等,下面將我遇到的問題與大家分享一下。
第一個注意點是:必須先編譯安裝zeromq,然后在編譯jzmq,否則會報zmq.h文件找不到。
官方安裝配置鏈接:http://zeromq.org/bindings:java
Step 1:安裝必須的依賴包:
[mdrill01@mdrill01 ~]$ sudo yum install autoconf automake libtool gcc gcc-g++ make libuuid-dev git java-1.6.0-openjdk-devel
備注:其中一些包依據操作系統不同所以並不是上面的名稱,請使用yum list avaiable *package name*查詢可獲取的包,然后安裝。例如:查找openjdk,藍色背景的就是我們要找的。
ot@mdrill01 jzmq]# sudo yum list avaiable *openjdk* Loaded plugins: fastestmirror, langpacks ....... Available Packages java-1.6.0-openjdk.x86_64 1:1.6.0.33-1.13.5.0.el7_0 updates java-1.6.0-openjdk-demo.x86_64 1:1.6.0.33-1.13.5.0.el7_0 updates java-1.6.0-openjdk-devel.x86_64 1:1.6.0.33-1.13.5.0.el7_0 updates java-1.6.0-openjdk-javadoc.x86_64 1:1.6.0.33-1.13.5.0.el7_0 updates java-1.6.0-openjdk-src.x86_64 1:1.6.0.33-1.13.5.0.el7_0 updates java-1.7.0-openjdk.x86_64 1:1.7.0.71-2.5.3.1.el7_0 updates java-1.7.0-openjdk-accessibility.x86_64 1:1.7.0.71-2.5.3.1.el7_0 updates java-1.7.0-openjdk-demo.x86_64 1:1.7.0.71-2.5.3.1.el7_0 updates java-1.7.0-openjdk-devel.x86_64 1:1.7.0.71-2.5.3.1.el7_0 updates java-1.7.0-openjdk-headless.x86_64 1:1.7.0.71-2.5.3.1.el7_0 updates java-1.7.0-openjdk-javadoc.noarch 1:1.7.0.71-2.5.3.1.el7_0 updates java-1.7.0-openjdk-src.x86_64 1:1.7.0.71-2.5.3.1.el7_0 updates [root@mdrill01 jzmq]# sudo yum install java-1.6.0-openjdk-devel.x86_64 #安裝 Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile ............ Dependency Updated: python-javapackages.noarch 0:3.4.1-6.el7_0 Complete! [root@mdrill01 jzmq]#
Step 2:編譯安裝zeromq-2.1.7
2.1 解壓縮包
[mdrill01@mdrill01 ~]$ tar -zxvf zeromq-2.1.7.tar.gz
2.2 Build&&Install
$ cd $ ./autogen.sh $ ./configure $ make $ sudo make install
Step 3:編譯安裝jzmq
3.1從github上clone jzmq
$ git clone https://github.com/zeromq/jzmq.git
3.2 Bulid&&Install(官方提供步驟,可惜不行)
$ cd jzmq
$ ./autogen.sh
$ ./configure
$ make #在執行make的時候報make[1]: *** No rule to make target `classdist_noinst.stamp', needed by `org/zeromq/ZMQ.class'. Stop.
$ sudo make install
3.3錯誤排除
1)確保Step 1的依賴包安裝完全
2)檢查JDK是不是用的openjdk6,如果不是請將JAVA_HOME臨時指向/usr/lib/jvm/java-1.6.0-openjdk.x86_64/
[mdrill01@mdrill01 ~]$ echo $JAVA_HOME /usr/java/jdk1.6.0_45 [mdrill01@mdrill01 ~]$ JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk.x86_64/
3.4解決錯誤
1)將目錄切換至jzmq/src下
2)touch classdist_noinst.stamp
3)執行:
CLASSPATH=.:./.:$CLASSPATH javac -d . org/zeromq/ZMQ.java org/zeromq/ZMQException.java org/zeromq/ZMQQueue.java org/zeromq/ZMQForwarder.java org/zeromq/ZMQStreamer.java
4)
cd .. ./autogen.sh./configure make sudo make install
5)測試是否安裝成功,切換至jzmq/perf/目錄下,執行sh local_lat.sh tcp://127.0.0.1:5000 1 100進行測試:
[mdrill01@mdrill01 jzmq]$ cd perf/ [mdrill01@mdrill01 perf]$ sh local_lat.sh tcp://127.0.0.1:5000 1 100 Exception in thread "main" java.lang.UnsatisfiedLinkError: no jzmq in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738) at java.lang.Runtime.loadLibrary0(Runtime.java:823) at java.lang.System.loadLibrary(System.java:1028) at org.zeromq.ZMQ.<clinit>(ZMQ.java:34) at local_lat.main(local_lat.java:36)
no jzmq in java.library.path???
解決辦法:
export JAVA_LIBRARY_PATH=/usr/local/lib 或者export LD_LIBRARY_PATH=/usr/local/lib 最好將其加入~/.bashrc中 [mdrill01@mdrill01 perf]$ vi ~/.bashrc export LD_LIBRARY_PATH=/usr/local/lib
然后在進行測試一切OK,首先在一個終端下執行:sh local_lat.sh tcp://127.0.0.1:5000 1 100
[mdrill01@mdrill01 perf]$ source ~/.bashrc [mdrill01@mdrill01 perf]$ sh local_lat.sh tcp://127.0.0.1:5000 1 100
此時,打開另外一個終端執行:sh remote_lat.sh tcp://127.0.0.1:5000 1 100
[mdrill01@mdrill01 jzmq]$ cd perf/ [mdrill01@mdrill01 perf]$ pwd /home/mdrill01/component/jzmq/perf [mdrill01@mdrill01 perf]$ sh remote_lat.sh tcp://127.0.0.1:5000 1 100 message size: 1 [B] #####說明測試成功 roundtrip count: 100 mean latency: 140.0 [us] [mdrill01@mdrill01 perf]$
Hadoop and JZMQ - no jzmq in java.library.path