HDFS權限管理篇
作者：尹正傑
版權聲明：原創作品，謝絕轉載！否則將追究法律責任。
一.HDFS權限概述
HDFS文件權限用作每次訪問HDFS文件或目錄時嘗試的授權檢查。HDFS文件權限與通常在Linux或Unix文件系統中使用的權限非常相似。但HDFS域Linux和其他采用POSIX模型的操作系統之間也存在顯著差異。 在Linux中，每個文件和目錄都有一個用戶和組。HDFS本身沒有任何用戶或組的概念，HDFS只是從底層的操作系統實體(例如在Linux文件系統中創建的用戶和組)導出用戶和組。 在Kerberos集群中，用戶的Kerberos憑證決定客戶端進程的身份，在默認的簡單安全模式下，用戶的身份由主機操作系統確定。 與Linux文件系統一樣，可以為文件或目錄的所有者，組的成員以及其他人分配單獨的文件權限。可以像Linux中一樣使用我們熟悉的r(讀取文件並列出目錄的內容)，w(創建或刪除文件或目錄)和x(訪問目錄的子目錄)權限。 可以使用八進制(例如"755","644"等)來設置文件的模式。需要注意的是，在Linux中，x表示執行文件的權限，但在HDFS中沒有這樣的概念。 當然，您也可以為文件或目錄指定umask值。默認的umask值為"022"，但是可以通過以下方式為"${HADOOP_HOME}/etc/hadoop/core-site.xml"文件進行修改。 <property> <name>fs.permissions.umask-mode</name> <value>037</value> <description>指定HDFS集群的umask值,默認值為022(創建的文件(目錄)權限即為"755"),在Linux中，x表示執行文件權限，而在HDFS中沒有這樣的概念,盡管你可以為該文件添加x權限。</description> </property> 溫馨提示：【umask值的計算方法,公式為："實際權限 = max_permnishu & (~umask)"】 本案例中我配置的umask值為"037"，對應二進制為"000 011 111",而"(~umask)"則對應為"111 100 000"，因此創建文件的默認權限應該為"740",因此計算過程如下所示: 111 111 111 & ~(000 011 111) ---------------- 111 111 111 & 111 100 000 --------------- 111 100 000
二.HDFS權限相關的配置參數
1>.配置HDFS權限
可以通過在"${HADOOP_HOME}/etc/hadoop/hdfs-site.xml"配置文件中將參數"dfs.permissions.enabled"的值設置為ture來配置HDFS權限。 如下所示，有此參數的默認值為true，因此權限檢查功能已經打開，故HDFS不需要再執行其他任何操作來進行權限檢查。 <property> <name>dfs.permissions.enabled</name> <value>true</value> <description>如果為"true"，則在HDFS中啟用權限檢查;如果為"false"，則關閉權限檢查;默認值為"true"。</description> </property>
2>.配置HDFS超級用戶
與Linux文件系統不同，主機用戶root不是HDFS的超級用戶。超級用戶是啟動NameNode的用戶，通常是操作系統用戶(如"root","hdfs","hadoop"等)，因此用戶(如"root","hdfs","hadoop"等)通常是HDFS的超級用戶。 可以通過在"${HADOOP_HOME}/etc/hadoop/hdfs-site.xml"配置文件中設置"dfs.permissions.superusergroup"參數來配置超級用戶組(改參數僅能被設置一次)。 <property> <name>dfs.permissions.superusergroup</name> <value>admin</value> <description> 此參數指定包含HDFS超級用戶的組(可以任意指定，分配給該組的任何用戶都將是HDFS超級用戶)，默認值為"supergroup"，我這里自定義超級用戶的組名為"admin"。 需要注意的是，該組名只能被設置一次，當對NameNode節點進行格式化后在修改改參數並不生效!如果你想要的強行生效只能重新格式化NameNode節點。幸運的是，我們可以通過"hdfs dfs -chown"命令來修改某個文件或目錄的所屬者。 </description> </property>
3>.HDFS如何執行權限檢查
客戶端的身份由客戶端(用戶)的名稱和它所屬的組的列表組成。HDFS執行文件權限檢查，以確保用戶是具有適當組權限的所有者或組列表的成員。如果兩者都不屬於，HDFS檢查用戶的"其他"文件權限，如果該檢查也失敗，則拒絕客戶端訪問HDFS的請求。 如前所述，在默認的simple操作模式下，由操作系統用戶名來確定客戶端進程ID，而在Kerberos模式下，通過Kerberos憑據確定客戶端身份。 根據運行集群的模式，一旦HDFS確定了用戶的身份，它會使用"${HADOOP_HOME}/etc/hadoop/core-site.xml"文件中的"hadoop.security.group.mapping"屬性來確定用戶所屬的組列表。NameNode執行用戶到組的映射。 如果組不在Linux服務器上，僅在公司LDAP服務器上，則必須配置名為"org.apache.hadoop.security.LdapGroupsMapping"的備用組映射服務，而不是默認組映射實現。 博主推薦閱讀: https://hadoop.apache.org/docs/r2.10.0/hadoop-project-dist/hadoop-common/GroupsMapping.html
三.更改HDFS集群中文件屬性的常用命令
1>.chmod命令
chmod命令用於更改HDFS集群中文件的權限。這類似於shell的chmod命令，但有一些例外(使用"man chmod"，你會發現Linux系統支持的參數更多)。
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -help chmod -chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH... : Changes permissions of a file. This works similar to the shell's chmod command with a few exceptions. -R modifies the files recursively. This is the only option currently supported. <MODE> Mode is the same as mode used for the shell's command. The only letters recognized are 'rwxXt', e.g. +t,a+r,g-w,+rwx,o=r. <OCTALMODE> Mode specifed in 3 or 4 digits. If 4 digits, the first may be 1 or 0 to turn the sticky bit on or off, respectively. Unlike the shell command, it is not possible to specify only part of the mode, e.g. 754 is same as u=rwx,g=rx,o=r. If none of 'augo' is specified, 'a' is assumed and unlike the shell command, no umask is applied. [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwxr----- - nn admin 0 2020-10-08 11:16 /yinzhengjie/conf -rw-r--r-- 3 nn admin 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -chmod 700 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwxr----- - nn admin 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 nn admin 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls -R /yinzhengjie/ drwx-wx--x - nn admin 0 2020-10-08 11:16 /yinzhengjie/conf -rwxr----- 3 nn admin 3362 2020-10-08 11:16 /yinzhengjie/conf/hdfs.keytab -rwxr----- 3 nn admin 1346 2020-10-08 11:16 /yinzhengjie/conf/hdfs_ca_cert -rwxr----- 3 nn admin 1834 2020-10-08 11:16 /yinzhengjie/conf/hdfs_ca_key -rwxr----- 3 nn admin 115 2020-10-08 11:16 /yinzhengjie/conf/host-rack.txt -rwxr----- 3 nn admin 4181 2020-10-08 11:16 /yinzhengjie/conf/keystore -rwxr----- 3 nn admin 463 2020-10-08 11:16 /yinzhengjie/conf/toplogy.py -rwxr----- 3 nn admin 1016 2020-10-08 11:16 /yinzhengjie/conf/truststore -rwx------ 3 nn admin 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -chmod -R 731 /yinzhengjie/conf [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls -R /yinzhengjie/ drwx-wx--x - nn admin 0 2020-10-08 11:16 /yinzhengjie/conf -rwx-wx--x 3 nn admin 3362 2020-10-08 11:16 /yinzhengjie/conf/hdfs.keytab -rwx-wx--x 3 nn admin 1346 2020-10-08 11:16 /yinzhengjie/conf/hdfs_ca_cert -rwx-wx--x 3 nn admin 1834 2020-10-08 11:16 /yinzhengjie/conf/hdfs_ca_key -rwx-wx--x 3 nn admin 115 2020-10-08 11:16 /yinzhengjie/conf/host-rack.txt -rwx-wx--x 3 nn admin 4181 2020-10-08 11:16 /yinzhengjie/conf/keystore -rwx-wx--x 3 nn admin 463 2020-10-08 11:16 /yinzhengjie/conf/toplogy.py -rwx-wx--x 3 nn admin 1016 2020-10-08 11:16 /yinzhengjie/conf/truststore -rwx------ 3 nn admin 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]#
2>. chown命令
更改文件的所有者和組。這與Linux shell的chown命令類似，但有一些例外。
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -help chown -chown [-R] [OWNER][:[GROUP]] PATH... : Changes owner and group of a file. This is similar to the shell's chown command with a few exceptions. -R modifies the files recursively. This is the only option currently supported. If only the owner or group is specified, then only the owner or group is modified. The owner and group names may only consist of digits, alphabet, and any of [-_./@a-zA-Z0-9]. The names are case sensitive. WARNING: Avoid using '.' to separate user name and group though Linux allows it. If user names have dots in them and you are using local file system, you might see surprising results since the shell command 'chown' is used for local files. [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - nn admingroup 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 nn supergroup 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -chown jason /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - nn admingroup 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 jason supergroup 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - nn admingroup 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 jason supergroup 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -chown :admin /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - nn admingroup 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 jason admin 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - nn admingroup 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 jason admin 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -chown java:bigdata /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - nn admingroup 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 java bigdata 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls -R /yinzhengjie/ drwx-wx--x - nn admingroup 0 2020-10-08 11:16 /yinzhengjie/conf -rwx-wx--x 3 nn admingroup 3362 2020-10-08 11:16 /yinzhengjie/conf/hdfs.keytab -rwx-wx--x 3 nn admingroup 1346 2020-10-08 11:16 /yinzhengjie/conf/hdfs_ca_cert -rwx-wx--x 3 nn admingroup 1834 2020-10-08 11:16 /yinzhengjie/conf/hdfs_ca_key -rwx-wx--x 3 nn admingroup 115 2020-10-08 11:16 /yinzhengjie/conf/host-rack.txt -rwx-wx--x 3 nn admingroup 4181 2020-10-08 11:16 /yinzhengjie/conf/keystore -rwx-wx--x 3 nn admingroup 463 2020-10-08 11:16 /yinzhengjie/conf/toplogy.py -rwx-wx--x 3 nn admingroup 1016 2020-10-08 11:16 /yinzhengjie/conf/truststore -rwx------ 3 java bigdata 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -chown -R python:devops /yinzhengjie/conf [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls -R /yinzhengjie/ drwx-wx--x - python devops 0 2020-10-08 11:16 /yinzhengjie/conf -rwx-wx--x 3 python devops 3362 2020-10-08 11:16 /yinzhengjie/conf/hdfs.keytab -rwx-wx--x 3 python devops 1346 2020-10-08 11:16 /yinzhengjie/conf/hdfs_ca_cert -rwx-wx--x 3 python devops 1834 2020-10-08 11:16 /yinzhengjie/conf/hdfs_ca_key -rwx-wx--x 3 python devops 115 2020-10-08 11:16 /yinzhengjie/conf/host-rack.txt -rwx-wx--x 3 python devops 4181 2020-10-08 11:16 /yinzhengjie/conf/keystore -rwx-wx--x 3 python devops 463 2020-10-08 11:16 /yinzhengjie/conf/toplogy.py -rwx-wx--x 3 python devops 1016 2020-10-08 11:16 /yinzhengjie/conf/truststore -rwx------ 3 java bigdata 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]#
3>.chgrp命令
chgrp命令相當於 "-chown ...:GROUP ..."
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -help chgrp -chgrp [-R] GROUP PATH... : This is equivalent to -chown ... :GROUP ... [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - nn admin 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 nn admin 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -chgrp supergroup /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - nn admin 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 nn supergroup 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls -R /yinzhengjie/ drwx-wx--x - nn admin 0 2020-10-08 11:16 /yinzhengjie/conf -rwx-wx--x 3 nn admin 3362 2020-10-08 11:16 /yinzhengjie/conf/hdfs.keytab -rwx-wx--x 3 nn admin 1346 2020-10-08 11:16 /yinzhengjie/conf/hdfs_ca_cert -rwx-wx--x 3 nn admin 1834 2020-10-08 11:16 /yinzhengjie/conf/hdfs_ca_key -rwx-wx--x 3 nn admin 115 2020-10-08 11:16 /yinzhengjie/conf/host-rack.txt -rwx-wx--x 3 nn admin 4181 2020-10-08 11:16 /yinzhengjie/conf/keystore -rwx-wx--x 3 nn admin 463 2020-10-08 11:16 /yinzhengjie/conf/toplogy.py -rwx-wx--x 3 nn admin 1016 2020-10-08 11:16 /yinzhengjie/conf/truststore -rwx------ 3 nn supergroup 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -chgrp -R admingroup /yinzhengjie/conf [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls -R /yinzhengjie/ drwx-wx--x - nn admingroup 0 2020-10-08 11:16 /yinzhengjie/conf -rwx-wx--x 3 nn admingroup 3362 2020-10-08 11:16 /yinzhengjie/conf/hdfs.keytab -rwx-wx--x 3 nn admingroup 1346 2020-10-08 11:16 /yinzhengjie/conf/hdfs_ca_cert -rwx-wx--x 3 nn admingroup 1834 2020-10-08 11:16 /yinzhengjie/conf/hdfs_ca_key -rwx-wx--x 3 nn admingroup 115 2020-10-08 11:16 /yinzhengjie/conf/host-rack.txt -rwx-wx--x 3 nn admingroup 4181 2020-10-08 11:16 /yinzhengjie/conf/keystore -rwx-wx--x 3 nn admingroup 463 2020-10-08 11:16 /yinzhengjie/conf/toplogy.py -rwx-wx--x 3 nn admingroup 1016 2020-10-08 11:16 /yinzhengjie/conf/truststore -rwx------ 3 nn supergroup 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
四.HDFS訪問控制列表
HDFS支持使用ACL為特定用戶和組設置更細粒度的權限。當希望以細粒度的方式授予權限，並處理復雜文件權限和訪問需求時，ACL是一種很好的方式。
在默認情況下，ACL是被禁用的。為了使用HDFS ACL，可以通過將以下屬性添加到"${HADOOP_HOME}/etc/hadoop/hdfs-site.xml"配置文件中，從而在NameNode上啟用它。 <property> <name>dfs.namenode.acls.enabled</name> <value>true</value> <description>設置為true以啟用對HDFS acl（訪問控制列表）的支持。默認情況下，ACL處於禁用狀態。禁用ACL時，NameNode會拒絕與設置或獲取ACL相關的所有RPC。</description> </property>
溫馨提示:
如下圖所示，若將"dfs.namenode.acls.enabled"修改為"true"，別忘記重啟HDFS集群喲~否則NameNode並不會讓ACL生效，換句話說就是無法設置配置ACL策略。
1>.與ACL相關的命令概述
getfacl:
顯示文件和目錄的訪問控制列表（ACL）。如果目錄具有默認ACL，那么getfacl也會顯示默認ACL。
setfacl:
設置文件和目錄的訪問控制列表（ACL）。
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -help getfacl -getfacl [-R] <path> : Displays the Access Control Lists (ACLs) of files and directories. If a directory has a default ACL, then getfacl also displays the default ACL. -R List the ACLs of all files and directories recursively. <path> File or directory to list. [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -help setfacl -setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>] : Sets Access Control Lists (ACLs) of files and directories. Options: -b Remove all but the base ACL entries. The entries for user, group and others are retained for compatibility with permission bits. -k Remove the default ACL. -R Apply operations to all files and directories recursively. -m Modify ACL. New entries are added to the ACL, and existing entries are retained. -x Remove specified ACL entries. Other ACL entries are retained. --set Fully replace the ACL, discarding all existing entries. The <acl_spec> must include entries for user, group, and others for compatibility with permission bits. <acl_spec> Comma separated list of ACL entries. <path> File or directory to modify. [root@hadoop101.yinzhengjie.com ~]#
2>.使用getfacl命令檢查目錄或文件上當前的ACL信息(若之前沒有啟用ACL功能，默認是沒有ACL策略的)
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - python devops 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 java bigdata 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfacl /yinzhengjie/hosts # file: /yinzhengjie/hosts # owner: java # group: bigdata user::rwx group::--- other::--- [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfacl /yinzhengjie/conf # file: /yinzhengjie/conf # owner: python # group: devops user::rwx group::-wx other::--x [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
3>.使用setfacl命令為目錄或文件設置ACL策略
如下圖所示，在為文件設置ACL策略后，在該文件權限列表的最后添加了一個加號("+")，一次表示此文件權限的ACL條目。這一點你是否似曾相識？基本上和Linux的ACL策略有異曲同工之妙。
需要注意的是，即使可以對具有ACL的文件或目錄授予用戶特定權限，但這些權限必須在文件掩碼的范圍之內。
如下圖所示，在"-getfacl"命令的輸出中，有一個名為mask的新條目，這里，它的值為"rwx"，這意味着用戶(jason)和組(admin)在此文件上確實具有"rwx"權限。
mask屬性定義對文件的最大權限。如果掩碼是"r-x"，則雖然用戶(jason)具有"rw-"權限，但有效的權限只是"r--"。
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - python devops 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 java bigdata 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfacl /yinzhengjie/hosts # file: /yinzhengjie/hosts # owner: java # group: bigdata user::rwx group::--- other::--- [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -setfacl -m user:jason:rw- /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -setfacl -m group:admin:r-x /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - python devops 0 2020-10-08 11:16 /yinzhengjie/conf -rwxrwx---+ 3 java bigdata 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfacl /yinzhengjie/hosts # file: /yinzhengjie/hosts # owner: java # group: bigdata user::rwx user:jason:rw- group::--- group:admin:r-x mask::rwx other::--- [root@hadoop101.yinzhengjie.com ~]#
4>.使用chmod命令更改文件權限，而后在看它對新ACL掩碼的影響
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - python devops 0 2020-10-08 11:16 /yinzhengjie/conf -rwxrwx---+ 3 java bigdata 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfacl /yinzhengjie/hosts # file: /yinzhengjie/hosts # owner: java # group: bigdata user::rwx user:jason:rw- group::--- group:admin:r-x mask::rwx other::--- [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -chmod 750 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - python devops 0 2020-10-08 11:16 /yinzhengjie/conf -rwxr-x---+ 3 java bigdata 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfacl /yinzhengjie/hosts # file: /yinzhengjie/hosts # owner: java # group: bigdata user::rwx user:jason:rw- #effective:r-- group::--- group:admin:r-x mask::r-x other::--- [root@hadoop101.yinzhengjie.com ~]#
5>.刪除ACL策略
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfacl /yinzhengjie/hosts # file: /yinzhengjie/hosts # owner: java # group: bigdata user::rwx user:jason:rw- #effective:r-- group::--- group:admin:r-x mask::r-x other::--- [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -setfacl -b /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfacl /yinzhengjie/hosts # file: /yinzhengjie/hosts # owner: java # group: bigdata user::rwx group::--- other::--- [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
五.HDFS擴展屬性
HDFS允許用戶往文件或者目錄中添加一些其他的元數據，這些數據被稱為HDFS的擴展屬性，因此應用程序也可以將其他信息存儲在inode中。例如，HDFS擴展屬性可以幫助應用程序指定文檔的字符編碼。HDFS擴展屬性可以被看作是系統HDFS文件系統權限的擴展。 如下圖所示，在設置擴展屬性(XAttr)時需要制定命名空間，否則就會設置屬性失敗。雖然有5種不同訪問限制的命名空間類型(即"user","trusted","security","system","raw"): 客戶端應用程序僅使用"用戶(user)"命名空間，"用戶(user)"命名空間中的HDFS擴展屬性可通過HDFS文件權限進行管理; 還有3種其他類型的命名空間，即"系統(system)"，"安全性(security)"和"原始(raw)"，供內部HDFS和其他系統使用; "可信(trusted)"命名空間是為HDFS超級用戶保留的; 默認HDFS擴展屬性為啟用狀態，可以通過"${HADOOP_HOME}/etc/hadoop/hdfs-site.xml"配置文件中的"dfs.namenode.xattrs.enabled"的值設置為"true"或"false"來啟用或禁用該功能。 <property> <name>dfs.namenode.xattrs.enabled</name> <value>true</value> <description>是否在NameNode上啟用了對擴展屬性的支持。默認值為true</description> </property>
1>.查看文件或目錄的擴展屬性的名稱和值
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -help getfattr -getfattr [-R] {-n name | -d} [-e en] <path> : Displays the extended attribute names and values (if any) for a file or directory. -R Recursively list the attributes for all files and directories. -n name Dump the named extended attribute value. -d Dump all extended attribute values associated with pathname. -e <encoding> Encode values after retrieving them.Valid encodings are "text", "hex", and "base64". Values encoded as text strings are enclosed in double quotes ("), and values encoded as hexadecimal and base64 are prefixed with 0x and 0s, respectively. <path> The file or directory. [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - python devops 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 java bigdata 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfattr -d /yinzhengjie/hosts # file: /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfattr -R -d /yinzhengjie/conf # file: /yinzhengjie/conf # file: /yinzhengjie/conf/hdfs.keytab # file: /yinzhengjie/conf/hdfs_ca_cert # file: /yinzhengjie/conf/hdfs_ca_key # file: /yinzhengjie/conf/host-rack.txt # file: /yinzhengjie/conf/keystore # file: /yinzhengjie/conf/toplogy.py # file: /yinzhengjie/conf/truststore [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
2>.將擴展屬性的名稱和值與文件或目錄相關聯
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -help setfattr -setfattr {-n name [-v value] | -x name} <path> : Sets an extended attribute name and value for a file or directory. -n name The extended attribute name. -v value The extended attribute value. There are three different encoding methods for the value. If the argument is enclosed in double quotes, then the value is the string inside the quotes. If the argument is prefixed with 0x or 0X, then it is taken as a hexadecimal number. If the argument begins with 0s or 0S, then it is taken as a base64 encoding. -x name Remove the extended attribute. <path> The file or directory. [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - python devops 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 java bigdata 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -setfattr -n user.department -v DevOps /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -setfattr -n user.bigdata /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfattr -d /yinzhengjie/hosts # file: /yinzhengjie/hosts user.bigdata user.department="DevOps" [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
3>.刪除擴展屬性
[root@hadoop101.yinzhengjie.com ~]# hdfs dfs -ls /yinzhengjie/ Found 2 items drwx-wx--x - python devops 0 2020-10-08 11:16 /yinzhengjie/conf -rwx------ 3 java bigdata 436 2020-10-08 10:44 /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfattr -d /yinzhengjie/hosts # file: /yinzhengjie/hosts user.bigdata user.department="DevOps" [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -setfattr -x user.bigdata /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfattr -d /yinzhengjie/hosts # file: /yinzhengjie/hosts user.department="DevOps" [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -setfattr -x user.department /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# hdfs dfs -getfattr -d /yinzhengjie/hosts # file: /yinzhengjie/hosts [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
