CDH通過rpm包安裝

author:wangxing

  http://note.youdao.com/md/preview/preview.html?file=%2Fyws%2Fapi%2Fpersonal%2Ffile%2F5B26A5BCF92E43519F13F322CD83F6A9%3Fmethod%3Ddownload%26read%3Dtrue%26shareKey%3Da91edbb326fa4c04f89089d3d25ff92c

1. 配置yum源

使用局域網yum源

http://192.168.9.114/cloudera-cdh5.15

2. 安裝規划

Hadoop主要包括HDFS和YARN兩部分。

通過rpm安裝時系統會創建hdfs用戶和yarn用戶來管理相應的部分，hdfs用戶和yarn用戶都屬於hadoop用戶組。

componet	role	系統用戶	用戶組
HDFS	namenode,secondarynamenode,datanode	hdfs	hadoop
YARN	resourcemanager,nodemanager	yarn	hadoop

主機信息及安裝規划如下：

ip	hostname	role
192.168.9.111	earth	namenode,resourcemanager
192.168.9.112	mercury	secondarynamenode,datanode,nodemanager
192.168.9.113	venus	datanode,nodemanager
192.168.9.114	mars	datanode,nodemanager
192.168.9.115	saturn	datanode,nodemanager

3. 安裝各個組件

• 192.168.9.111 earth

yum install zookeeper hadoop hadoop-hdfs hadoop-yarn
yum install hadoop-hdfs-namenode
yum install hadoop-yarn-resourcemanager

• 192.168.9.112 mercury

yum install zookeeper hadoop hadoop-hdfs hadoop-yarn
yum install hadoop-hdfs-secondarynamenode
yum install hadoop-hdfs-datanode
yum install hadoop-yarn-nodemanager

• 192.168.9.112 venus
• 192.168.9.112 mars
• 192.168.9.112 saturn

yum install zookeeper hadoop hadoop-hdfs hadoop-yarn
yum install hadoop-hdfs-datanode
yum install hadoop-yarn-nodemanager

4. 配置

Hadoop的配置文件目錄為/etc/hadoop/conf

hdfs-site.xml

name	value
dfs.namenode.name.dir	file:///xwtec/dfs/nn
dfs.namenode.checkpoint.dir	file:///xwtec/dfs/snn
dfs.datanode.data.dir	file:///xwtec/dfs/dn

4.1 啟動HDFS

在earth上初始化hdfs並啟動namenode

# 初始化HDFS
sudo -u hdfs hadoop namenode -format 
# 啟動namenode
service hadoop-hdfs-namenode start

在mercury上啟動secondarynamenode和datanode

service hadoop-hdfs-secondarynamenode start
service hadoop-hdfs-datanode start

在venus, mars, saturn上啟動datanode

service hadoop-hdfs-datanode start

查看HDFS狀態 http://192.168.9.111:50070

4.2 啟動YARN

在earth上啟動resourcemanager

service hadoop-yarn-resourcemanager start

在mercury, venus, mars, saturn上啟動nodemanager

service hadoop-yarn-nodemanager start

查看yarn狀態 http://192.168.9.111:8088

來自為知筆記(Wiz)