角色 |
安裝 |
hdfs配置 |
yarn配置 |
hdfs 格式化 |
啟動yarn服務 |
啟動hdfs服務 |
master |
yum install hadoop-hdfs-namenode yum install hadoop-yarn-resourcemanager |
core-site.xml hdfs-site.xml |
mapred-site.xml yarn-site.xml |
hdfs>hdfs namenode -format |
root>service hadoop-yarn-resourcemanager start |
root>service hadoop-hdfs-namenode start |
slave |
yum install hadoop-hdfs-datanode yum install hadoop-yarn-nodemanager |
core-site.xml hdfs-site.xml |
mapred-site.xml yarn-site.xml |
root>service hadoop-yarn-nodemanager start |
root>service hadoop-hdfs-namenode start |
|
proxy |
yum install hadoop-mapreduce-historyserver yum install hadoop-yarn-proxyserver |
core-site.xml hdfs-site.xml |
mapred-site.xml yarn-site.xml |
root>service hadoop-mapreduce-historyserver start root>service hadoop-yarn-proxyserver start |
||
client |
yum install hadoop-client |
core-site.xml hdfs-site.xml |
mapred-site.xml yarn-site.xml |
角色 |
hdfs本地存儲目錄 |
hdfs雲存儲目錄 |
yarn本地存儲目錄 |
master |
mkdir -p /data/dfs/nn chown hdfs:hadoop |
||
slave |
mkdir -p /data/dfs/dn chown hdfs:hadoop |
mkdir -p /data/yarn/local mkdir -p /data/yarn/logs |
|
proxy |
|||
client |
hdfs dfs –mkdir /tmp hdfs dfs –mkdir /user/history hdfs dfs –mkdir /var/log/hadoop-yarn |
hdfs本地存儲目錄，必須授權給用戶hdfs:hadoop;yarn本地存儲目錄，必須授權給用戶yarn:hadoop;一切目錄設置均與xml文件對應！root用戶只有安裝、啟動服務期限，沒有hadoop執行權限。
Hdfs服務web地址 |
cMaster:50070 |
Yarn服務web地址 |
cMaster:8088 |
JobHistory服務web地址 |
cProxy:19888 |
建立私有賬戶joe
[root@iClient~]#adduser joe |
本地新建joe用戶 |
[root@iClient~]#passwd joe |
更改joe密碼 |
[hdfs@iClient~]#hdfs dfs –mkdir /user/joe |
在hdfs建立joe的家目錄 |
[hdfs@iClient~]#hdfs dfs –chown –R joe /user/joe |
給家目錄授權 |
[hdfs@iClient~]#sudo –u joe hdfs dfs –put /……/yourdata.txt joeInput |
本地文件yourdata.txt上傳至joeInput |
[hdfs@iClient~]#sudo –u joe Hadoop jar /…/hadoo-mapreduce-example.jar pi 4 10000 |
執行例程的pi，4個map，做10000次。 |