RedHat 安裝Hadoop並運行wordcount例子


1.安裝 Red Hat 環境

2.安裝JDK

3.下載hadoop2.8.0 

  http://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz

 

4.在用戶目錄下新建hadoop文件夾,並解壓hadoop壓縮包

  mkdir Hadoop

  tar -zxvf hadoop-2.8.0.tar.gz

5.為hadoop配置JAVA_HOME

[xxx@localhost ~]$ cd hadoop/hadoop-2.8.0/etc/hadoop/
[xxx@localhost hadoop]$ vi hadoop-env.sh 

6.格式化(初始化)HDFS文件系統

[xxx@localhost ~]$ cd hadoop/hadoop-2.8.0/bin
[xxx@localhost bin]$ ./hadoop namenode -format

  (ps:hadoop namenode -format 已經過時 hdfs namenode -format 最新的需要等待較長時間)

7.啟動haddop

[xxx@localhost ~]$ cd hadoop/hadoop-2.8.0/sbin/
[xxx@localhost sbin]$ ./start-all.sh

  (ps:現在推薦使用 ./start-hdfs.sh ./start-yarn.sh)

8.jps查看hadoop是否成功啟動

9.在hadoop目錄里下新建文件夾並新建兩個測試文件

[xxx@localhost ~]$ cd hadoop/
[xxx@localhost hadoop]$ mkdir test
[xxx@localhost hadoop]$ cd test/
[xxx@localhost test]$ echo "hello word" > file1.txt
[xxx@localhost test]$ echo "hello hadoop" > file2.txt

10.在hdfs文件系統里新建input文件夾並將測試文件上傳

[xxx@localhost ~]$ cd hadoop/hadoop-2.8.0/bin/
[xxx@localhost bin]$ ./hadoop fs -mkdir input
[xxx@localhost bin]$ ./hadoop fs -put ~/hadoop/test/file*.txt input
17/07/06 16:33:34 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[xxx@localhost bin]$ ./hadoop fs -ls input
17/07/06 16:33:45 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 2 items
-rw-r--r--   1 chenguangjin chenguangjin         11 2017-07-06 16:33 input/file1.txt
-rw-r--r--   1 chenguangjin chenguangjin         13 2017-07-06 16:33 input/file2.txt

  (ps:注意此處由WARN,不需要管它,可以成功上傳)

11.運行wordcount小例子

[xxx@localhost bin]$ ./hadoop jar ~/hadoop/hadoop-2.8.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.0.jar  wordcount input output

12.查看結果

[xxx@localhost bin]$ ./hadoop fs -text output/part-r-00000
17/07/06 16:46:02 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
hadoop    1
hello    2
word    1
[xxx@localhost bin]$ 

 

(over怎么樣是不是還挺有趣的)

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM