storm安裝筆記以及提交拓撲任務

Storm -- Distributed and fault-tolerant realtime computation

這是一個分布式的、容錯的實時計算系統

把Storm依賴組件的版本貼出來供各位參考：

java(1.7.0_45)

python(2.6.6)

zookeeper (zookeeper-3.4.5)

zeromq(zeromq-2.1.7)

jzmq(github trunk)

storm(0.8.1)

 

java和Python的安裝過程不贅述了，說下其他組件的安裝過程

安裝zookeeper

wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.5/zookeeper-3.4.5.tar.gz
tar xvf zookeeper-3.4.5.tar.gz
cd zookeeper-3.4.5
cp conf/zoo_sample.cfg conf/zoo.cfg

設置下環境變量

export ZOOKEEPER_HOME=/root/zookeeper/zookeeper-3.4.5
export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64
export PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$PATH

然后就可以啟動zookeeper了

zkServer.sh start

可以運行下zkCli.sh，測試下zookeeper是否可用

 

安裝zeromq(安裝好后/usr/local/lib下會有相應的庫文件)

wget http://download.zeromq.org/zeromq-2.1.7.tar.gz
tar xvf zeromq-2.1.7.tar.gz
cd zeromq-2.1.7
yum install -y gcc gcc-c++.x86_64 uuid-devel.x86_64 libuuid-devel.x86_64
./configure
make
make install

 

安裝jzmq

git clone https://github.com/nathanmarz/jzmq.git
./autogen.sh
./configure 
make
make install

 

開始安裝storm

wget https://github.com/downloads/nathanmarz/storm/storm-0.8.1.zip
unzip storm-0.8.1.zip
cd storm-0.8.1

修改conf/storm.yaml如下

 storm.zookeeper.servers:
     - "127.0.0.1"
 nimbus.host: "192.168.24.57"
 storm.local.dir: "/root/storm/workdir"
 java.library.path: "/usr/local/lib:/opt/local/lib:/usr/lib"
 topology.debug: "true"

然后將storm/bin加入環境變量

export STORM_HOME=/root/storm/storm-0.8.1
export PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$STORM_HOME/bin:$PATH

 啟動storm

storm nimbus &
storm supervisor &
storm ui &

jps可以看到如下進程，而且日志也沒有報錯。基本上就ok了，可以提交任務在storm環境中跑了

14157 core
14106 QuorumPeerMain
30978 Jps
14144 supervisor
13995 nimbus

 

提交拓撲的代碼

storm jar stormDemo.jar com.storm.example.SimpleTopology testhui3

殺死拓撲

storm kill testhui3

 

默認8080端口提供了一個ui dashboard

enjoy~