集群
啟動/停止Hadoop集群:start-all.sh stop-all.sh
1) 啟動NameNode, DataNode
2) 啟動JournalNode, JournalNode在hdfs-site.xml中指定edits log存儲的位置,主備NameNode共享數據,方便同步
3) 啟動DFSZKFailoverController, HA會啟用ZooKeeper Failover Controller
4) 啟動YARN守護進程ResourceManager, NodeManager
啟動/停止HDFS集群:start-dfs.sh stop-dfs.sh
1) 啟動NameNode, DataNode
2) 啟動JournalNode, JournalNode在hdfs-site.xml中指定edits log存儲的位置,主備NameNode共享數據,方便同步
3) 啟動DFSZKFailoverController, HA會啟用ZooKeeper Failover Controller
啟動/停止YARN集群:start-yarn.sh stop-yarn.sh
據說如果ResourceManager和NameNode不在同一台機器上,那NameNode上無法直接使用start-yarn.sh啟動ResourceManager,但NodeManager可以啟動,ResourceManager需要在ResourceManager節點上啟動,這一點待考證
節點
啟動NameNode: hadoop-daemon.sh start namenode
啟動DataNode: hadoop-daemon.sh start datanode
啟動JournalNode節點: hadoop-daemon.sh start journalnode
啟動zkfc節點:hadoop-daemon.sh start zkfc
啟動ResourceManager: yarn-daemon.sh start resourcemanager
啟動NodeManager: yarn-daemon.sh start nodemanager
啟動ZooKeeper節點:zkServer.sh start 啟動QuorumPeerMain進程
查看ZooKeeper節點狀態:zkServer.sh status 查看leader和follower狀態
格式化HDFS: hdfs namenode -format
格式化ZKFC:hdfs zkfc -formatZK
查看NameNode和DataNode信息: hdfs dfsadmin -report
查看高可用NameNode狀態: hdfs haadmin -getServiceState nn1 (active/standby)
Web界面端口
NameNode 50070
DataNode 50075
ResourceManager 8088
NodeManager 8042
更多端口參考
安裝,查看等常用命令
查看進程:jps
kill進程:kill -9 pid
內容追加到文件:echo content > file, 如 echo 1 > myid
SSH拷貝:scp,如scp -r /usr/local/hadoop master:/usr/local
關閉防火牆:
查看狀態systemctl status firewalld
關閉防火牆systemctl stop firewalld
開機禁用防火牆systemctl disable firewalld
配置免登錄:
ssh-keygen -t rsa
ssh-copy-id -i host01
Hadoop Shell命令 http://hadoop.apache.org/docs/r1.0.4/cn/hdfs_shell.html
原文鏈接:https://blog.csdn.net/weixin_42129080/article/details/80738631