hadoop 2.8.5安裝步驟


1、創建hadoop用戶,作為haoop的運行用戶

2、配置JAVA_HOME環境變量,修改/etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_51
export PATH=${JAVA_HOME}/bin:$PATH

3、配置hadoop用戶的PATH環境變量,修改.bash_profile
$HOME/hadoop/bin:$HOME/hadoop/bin

4、配置免密登錄,需要配置master免密登錄slave和master自身
a、namenode上執行 ssh-keygen -t rsa 生成key
b、cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
c、將生成的公鑰復制到機器namenode上的~/.ssh/authorized_keys中
ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@192.168.1.100
d、測試能登錄到自身

5、配置hadoop-env.sh及yarn-env.sh中的javahome
export JAVA_HOME=/usr/java/jdk1.8.0_51
export HADOOP_CONF_DIR=/home/hadoop/hadoop/etc/hadoop
6、修改slave 文件
7、創建目錄
mkdir tmp
mkdir hdfs
mkdir hdfs/name
mkdir hdfs/data
8、修改core-site.xml配置文件
<configuration>
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/tmp</value>
<final>true</final>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://rhel01:9000</value>
<final>true</final>
</property>
</configuration>

9、修改hdfs-site.xml文件
<configuration>
<property>
<name>dfs.http.address</name>
<value>rhel01:50070</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>rhel01:50090</value>
</property>
</configuration>

10、修改mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>rhel01:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>rhel01:19888</value>
</property>
</configuration>

11、修改yarn-site.xml文件
<configuration>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>rhel01</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>rhel01:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>rhel01:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>rhel01:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>rhel01:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>rhel01:8088</value>
</property>
</configuration>

12、格式化namenode
hdfs namenode -format

13、啟動dfs
start-dfs.sh

14、啟動yarn
start-yarn.sh


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM