HDFS 命令深入淺出~
[root@neusoft-master ~]# hadoop dfs
Usage: hadoop fs [generic options]
[-appendToFile <localsrc> ... <dst>]
[-cat [-ignoreCrc] <src> ...]
[-checksum <src> ...]
[-chgrp [-R] GROUP PATH...]
[-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
[-chown [-R] [OWNER][:[GROUP]] PATH...]
[-copyFromLocal [-f] [-p] [-l] <localsrc> ... <dst>] #常用的命令，請熟記
[-copyToLocal [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
[-count [-q] [-h] [-v] <path> ...] #常用的命令，請熟記
[-cp [-f] [-p | -p[topax]] <src> ... <dst>] #常用的命令，請熟記
[-createSnapshot <snapshotDir> [<snapshotName>]]
[-deleteSnapshot <snapshotDir> <snapshotName>]
[-df [-h] [<path> ...]]
[-du [-s] [-h] <path> ...]
[-expunge]
[-find <path> ... <expression> ...]
[-get [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
[-getfacl [-R] <path>]
[-getfattr [-R] {-n name | -d} [-e en] <path>]
[-getmerge [-nl] <src> <localdst>]
[-help [cmd ...]]
[-ls [-d] [-h] [-R] [<path> ...]] #常用的命令，請熟記
[-mkdir [-p] <path> ...] #常用的命令，請熟記
[-moveFromLocal <localsrc> ... <dst>]
[-moveToLocal <src> <localdst>]
[-mv <src> ... <dst>]
[-put [-f] [-p] [-l] <localsrc> ... <dst>] #常用的命令，請熟記
[-renameSnapshot <snapshotDir> <oldName> <newName>]
[-rm [-f] [-r|-R] [-skipTrash] <src> ...] #常用的命令，請熟記
[-rmdir [--ignore-fail-on-non-empty] <dir> ...]
[-setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>]]
[-setfattr {-n name [-v value] | -x name} <path>]
[-setrep [-R] [-w] <rep> <path> ...]
[-stat [format] <path> ...]
[-tail [-f] <file>] #常用的命令，請熟記
[-test -[defsz] <path>]
[-text [-ignoreCrc] <src> ...] #常用的命令，請熟記
[-touchz <path> ...] #常用的命令，請熟記
[-usage [cmd ...]]
練習使用上述命令
1.[root@neusoft-master ~]# hadoop dfs -ls / #列出文件或目錄內容
2. hadoop dfs -ls -R / 遞歸列出目錄內容
之前使用的命令是：也顯示了使用ls -R代替lsr
3.查看目錄使用情況
[root@neusoft-master ~]# hadoop dfs -df /hbase
[root@neusoft-master ~]# hadoop dfs -df / #查看根目錄使用情況
4.顯示目錄中所有文件及目錄大小
[root@neusoft-master ~]# hadoop dfs -du /
5.顯示 指定路徑 下的目錄數以及文件數，輸出格式：
目錄數 文件數 大小 文件名
[root@neusoft-master ~]# hadoop dfs -count / #加上-q參數可查看文件索引
6.HDFS上創建文件
[root@neusoft-master ~]# hadoop dfs -touchz /user/root/test001
[root@neusoft-master ~]# hadoop dfs -ls /user/root #hadoop dfs -ls /
7. 將HDFS上面的文件移動到目的文件夾
[root@neusoft-master ~]# hadoop dfs -mv /user/root/test001 /user/root/test
[root@neusoft-master ~]# hadoop dfs -ls /user/root/test
17/01/28 07:53:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 2 items
-rw-r--r-- 3 root supergroup 19 2017-01-07 11:31 /user/root/test/demo1
-rw-r--r-- 3 root supergroup 0 2017-01-28 07:49 /user/root/test/test001
8.將HDFS上面 相應路徑的文件移動到回收站
[root@neusoft-master ~]# hadoop dfs -rm /user/root/test/test001
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
17/01/28 08:02:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/01/28 08:02:08 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.
Deleted /user/root/test/test001
備注1：hadoop dfs -rmr /user/root/test/test001 #刪除相應目錄以及目錄下的文件，加上-skipTrash則直接刪除
備注2：清除回收站-expunge
9.將本地centos文件上傳至hdfs文件系統目錄
[root@neusoft-master ~]# hadoop dfs -put /usr/local/filecontent/wordsTest001 /user/root/
[root@neusoft-master ~]# hadoop dfs -ls /
17/01/28 09:35:06 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 5 items
-rw-r--r-- 3 root supergroup 120064 2017-01-07 09:54 /234967-13112015163685.jpg
drwxr-xr-x - root supergroup 0 2017-01-28 06:58 /hbase
-rw-r--r-- 3 root root 20 2017-01-07 10:14 /hello2
drwxrwxrwx - root supergroup 0 2017-01-06 21:37 /tmp
drwxr-xr-x - root supergroup 0 2017-01-07 15:05 /user
[root@neusoft-master ~]# hadoop dfs -ls /user/root
17/01/28 09:37:12 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 4 items
drwxr-xr-x - root supergroup 0 2017-01-05 23:58 /user/root/input
drwxr-xr-x - root supergroup 0 2017-01-06 00:02 /user/root/output
drwxr-xr-x - root supergroup 0 2017-01-28 08:02 /user/root/test
-rw-r--r-- 3 root supergroup 30 2017-01-28 09:34 /user/root/wordsTest001
10.put的另一種實現方法
[root@neusoft-master ~]# hadoop dfs-copyFromLocal /usr/local/filecontent/wordsTest001 /user/root/test
[root@neusoft-master ~]# hadoop dfs -ls /user/root/test
11.本地文件系統移動到HDFS上
[root@neusoft-master ~]# hadoop dfs -moveFromLocal /usr/local/filecontent/word /user/root/test
[root@neusoft-master ~]#hadoop dfs -ls /user/root/test
查看本地文件系統發現文件已移走
12.下載Hdfs上面的文件到本地
[root@neusoft-master ~]# hadoop dfs -get /user/root/test/word /usr/local/filecontent/
查看結果：
補充：此時文件已存在會提醒已存在
方式2：[root@neusoft-master ~]# hadoop dfs -copyToLocal /user/root/test/word /usr/local/filecontent/
方式3：[root@neusoft-master ~]# hadoop dfs -moveToLocal /user/root/test/word /usr/local/filecontent/
#可以加-crc，表示crc文件校驗
13.將HDFS源路徑下的所有文件名排序合並成一個文件輸出到本地目錄
[root@neusoft-master ~]# hadoop dfs -getmerge /user/root/test/ /usr/local/result
addnl是可供選擇的項。表示指定在每個文件結尾添加換行符
14.查看Hdfs上面文本的內容
[root@neusoft-master ~]# hadoop dfs -cat /user/root/test/word
或者：[root@neusoft-master ~]# hadoop dfs -text /user/root/test/word
15.在HDFS上創建目錄和文件操作
創建目錄 [root@neusoft-master ~]# hadoop dfs -mkdir /user/test001
創建文件操作[root@neusoft-master ~]# hadoop dfs -touchz /user/test001/wordp
16.檢查Hdfs上面文件路徑
[root@neusoft-master ~]# hadoop dfs -test -e /user/test001/wordp
注：-e檢查文件是否存在 -z檢查文件是否為0字節 -d表示是目錄，返回1，否則0
17.顯示給定路徑的文件或目錄的統計信息，%b 文件大小%n文件名 %r複製因子 %y %Y修改日期
[root@neusoft-master ~]# hadoop dfs -stat %b%n%o%r /user/test001/wordp
17/01/28 10:47:12 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
0wordp1342177283
18. 顯示Hdfs上路徑中的文件最後1字節內容
[root@neusoft-master ~]# hadoop dfs -tail -f /user/root/test/word #-f選項的內容會隨著文件內容更新而更新
19.HDFS改變文件或目錄權限chmod
[root@neusoft-master ~]# hadoop dfs -chmod -R 777 /user/root/test/word #-R表示遞歸
[root@neusoft-master ~]# hadoop dfs -ls /user/root/test/word
17/01/28 12:22:53 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
-rwxrwxrwx 3 root supergroup 30 2017-01-28 09:56 /user/root/test/word
20.改變HDFS上面為path所屬類別。-R表示遞歸操作
[root@neusoft-master ~]# hadoop dfs -chown -R root /user/test
17/01/28 12:27:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[root@neusoft-master ~]# hadoop dfs -ls /user/test
17/01/28 12:27:38 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 1 items
drwxr-xr-x - root supergroup 0 2017-01-07 15:02 /user/test/demo2
備注：所有命令幫助文檔，可幫助查詢
Hdfs操作命令~End！