角色 |
安装 |
hdfs配置 |
yarn配置 |
hdfs 格式化 |
启动yarn服务 |
启动hdfs服务 |
master |
yum install hadoop-hdfs-namenode yum install hadoop-yarn-resourcemanager |
core-site.xml hdfs-site.xml |
mapred-site.xml yarn-site.xml |
hdfs>hdfs namenode -format |
root>service hadoop-yarn-resourcemanager start |
root>service hadoop-hdfs-namenode start |
slave |
yum install hadoop-hdfs-datanode yum install hadoop-yarn-nodemanager |
core-site.xml hdfs-site.xml |
mapred-site.xml yarn-site.xml |
root>service hadoop-yarn-nodemanager start |
root>service hadoop-hdfs-namenode start |
|
proxy |
yum install hadoop-mapreduce-historyserver yum install hadoop-yarn-proxyserver |
core-site.xml hdfs-site.xml |
mapred-site.xml yarn-site.xml |
root>service hadoop-mapreduce-historyserver start root>service hadoop-yarn-proxyserver start |
||
client |
yum install hadoop-client |
core-site.xml hdfs-site.xml |
mapred-site.xml yarn-site.xml |
角色 |
hdfs本地存储目录 |
hdfs云存储目录 |
yarn本地存储目录 |
master |
mkdir -p /data/dfs/nn chown hdfs:hadoop |
||
slave |
mkdir -p /data/dfs/dn chown hdfs:hadoop |
mkdir -p /data/yarn/local mkdir -p /data/yarn/logs |
|
proxy |
|||
client |
hdfs dfs –mkdir /tmp hdfs dfs –mkdir /user/history hdfs dfs –mkdir /var/log/hadoop-yarn |
hdfs本地存储目录，必须授权给用户hdfs:hadoop;yarn本地存储目录，必须授权给用户yarn:hadoop;一切目录设置均与xml文件对应！root用户只有安装、启动服务期限，没有hadoop执行权限。
Hdfs服务web地址 |
cMaster:50070 |
Yarn服务web地址 |
cMaster:8088 |
JobHistory服务web地址 |
cProxy:19888 |
建立私有账户joe
[root@iClient~]#adduser joe |
本地新建joe用户 |
[root@iClient~]#passwd joe |
更改joe密码 |
[hdfs@iClient~]#hdfs dfs –mkdir /user/joe |
在hdfs建立joe的家目录 |
[hdfs@iClient~]#hdfs dfs –chown –R joe /user/joe |
给家目录授权 |
[hdfs@iClient~]#sudo –u joe hdfs dfs –put /……/yourdata.txt joeInput |
本地文件yourdata.txt上传至joeInput |
[hdfs@iClient~]#sudo –u joe Hadoop jar /…/hadoo-mapreduce-example.jar pi 4 10000 |
执行例程的pi，4个map，做10000次。 |