Storm安裝部署

0x01 環境說明

OS: CentOS-6.5-x86_64 
JDK: jdk-8u111-linux-x64 
Hadoop: hadoop-2.6.5 
Zookeeper: zookeeper-3.4.6
Storm: storm-0.9.7

0x02 安裝配置

2.1 下載

可以在下載列表中選擇你想要安裝的版本，這里下載的是apache-storm-0.9.7.tar.gz

2.2 安裝

$ tar -zxvf apache-storm-0.9.7.tar.gz
//重命名
$ mv apache-storm-0.9.7 storm-0.9.7
//建立軟鏈
$ ln -s /home/hadoop/cloud/storm-0.9.7/ /home/hadoop/cloud/storm

2.3 配置環境變量

# set storm environment
export STORM_HOME=/home/hadoop/cloud/storm
export PATH=$PATH:$STORM_HOME/bin

使環境變量生效

# source /etc/profile
# su hadoop
$ source /etc/profile

2.4 配置storm.yaml

需要自己在storm下建立data文件夾。

########### These MUST be filled in for a storm configuration
storm.zookeeper.servers:
      - "master"
      - "slave1"
      - "slave2"


nimbus.host: "master"
storm.local.dir: "/home/hadoop/cloud/storm/data"
ui.port: 8888
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
# 
# 
# ##### These may optionally be filled in:
#    
## List of custom serializations
# topology.kryo.register:
#     - org.mycompany.MyType
#     - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
#     - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
#     - "server1"
#     - "server2"

## Metrics Consumers
# topology.metrics.consumer.register:
#   - class: "backtype.storm.metric.LoggingMetricsConsumer"
#     parallelism.hint: 1
#   - class: "org.mycompany.MyMetricsConsumer"
#     parallelism.hint: 1
#     argument:
#       - endpoint: "metrics-collector.mycompany.org"

2.5 將文件分發到其它節點

scp -r /home/hadoop/cloud/storm-0.9.7 hadoop@slave1:~/cloud/
scp -r /home/hadoop/cloud/storm-0.9.7 hadoop@slave2:~/cloud/

注意：需要在所有slave節點建立軟鏈並配置環境變量。

0x03 測試

3.1 啟動服務

master節點：啟動nimbus和ui服務。

nohup /home/hadoop/cloud/storm/bin/storm ui >/dev/null 2>&1 &
nohup /home/hadoop/cloud/storm/bin/storm nimbus >/dev/null 2>&1 &

先啟動ui服務，否則nimbus服務會退出。

slave節點啟動supervisor服務。

nohup /home/hadoop/cloud/storm/bin/storm supervisor >/dev/null 2>&1 &

3.2 jps查看進程

master節點

$ jps

2947 QuorumPeerMain
3828 Jps
3485 nimbus
3437 core

slave節點

$ jps

1896 Jps
1628 QuorumPeerMain
1726 supervisor

3.3 查看 UI 狀態

http://192.168.32.100:8888/

0x04 參考鏈接

• Storm 安裝，配置詳解
• Storm集群安裝部署步驟【詳細版】
• Storm安裝部署

2017-10-21 11:20:21 星期六