基於Ambari Server部署HDP集群實戰案例


          基於Ambari Server部署HDP集群實戰案例

                              作者:尹正傑

版權聲明:原創作品,謝絕轉載!否則將追究法律責任。

 

 

 

一.部署Ambari Server端

  博主推薦閱讀:
    https://www.cnblogs.com/yinzhengjie2020/p/12219108.html

 

二.時鍾同步

  我們在生產環境中必須同步各個服務器時間是一致的,最好是有內網的時間服務器。

  配置時間同步的好處:
    可以定位集群故障發生的具體時間;
    解決HBase,Kudu等存儲依賴於時間同步的問題(若時間不同步會報錯);

  CentOS 6.x默認使用ntpd軟件來同步時間的,但在CentOS 7.x版本之后就被更替為chrony來實現時間同步,因此咱們安裝時間同步就是用CentOS 7.x推薦的Chrony來進行時間同步。

1>.安裝chrony

[root@hdp101.yinzhengjie.org.cn ~]# yum -y install chrony
Loaded plugins: fastestmirror
Determining fastest mirrors
 * base: mirror.bit.edu.cn
 * extras: mirror.bit.edu.cn
 * updates: mirrors.huaweicloud.com
ambari-repo                                                                                                                                                                                                                                            | 2.9 kB  00:00:00     
base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
mysql-connectors-community                                                                                                                                                                                                                             | 2.5 kB  00:00:00     
mysql-tools-community                                                                                                                                                                                                                                  | 2.5 kB  00:00:00     
mysql80-community                                                                                                                                                                                                                                      | 2.5 kB  00:00:00     
updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
(1/2): extras/7/x86_64/primary_db                                                                                                                                                                                                                      | 159 kB  00:00:00     
(2/2): updates/7/x86_64/primary_db                                                                                                                                                                                                                     | 5.9 MB  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package chrony.x86_64 0:3.4-1.el7 will be installed
--> Processing Dependency: libseccomp.so.2()(64bit) for package: chrony-3.4-1.el7.x86_64
--> Running transaction check
---> Package libseccomp.x86_64 0:2.3.1-3.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                             Arch                                                            Version                                                              Repository                                                     Size
==============================================================================================================================================================================================================================================================================
Installing:
 chrony                                                              x86_64                                                          3.4-1.el7                                                            base                                                          251 k
Installing for dependencies:
 libseccomp                                                          x86_64                                                          2.3.1-3.el7                                                          base                                                           56 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  1 Package (+1 Dependent package)

Total download size: 306 k
Installed size: 788 k
Downloading packages:
(1/2): libseccomp-2.3.1-3.el7.x86_64.rpm                                                                                                                                                                                                               |  56 kB  00:00:00     
(2/2): chrony-3.4-1.el7.x86_64.rpm                                                                                                                                                                                                                     | 251 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         957 kB/s | 306 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Installing : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 
  Verifying  : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Verifying  : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 

Installed:
  chrony.x86_64 0:3.4-1.el7                                                                                                                                                                                                                                                   

Dependency Installed:
  libseccomp.x86_64 0:2.3.1-3.el7                                                                                                                                                                                                                                             

Complete!
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# yum -y install chrony

2>.禁用本地時間服務器從公網同步時間

[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf 
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# sed -r -i 's@(^server)@#\1@g' /etc/chrony.conf 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 

3>.新增時間同步服務器為本機地址

[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# vim /etc/chrony.conf 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

4>.設置允許來chronyd服務器端同步時間的客戶端地址

[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# grep ^#allow /etc/chrony.conf 
#allow 192.168.0.0/16
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# sed -r -i 's@#(allow) 192.168.0.0/16@\1 172.200.0.0/21@' /etc/chrony.conf 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# vim /etc/chrony.conf 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
allow 172.200.0.0/21          #允許客戶端IP地址同步
allow 127/8                #允許本地同步時間
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

5>.設置chronyd服務器端允許返回本地時間

[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
allow 172.200.0.0/21
allow 127/8
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# grep ^#local /etc/chrony.conf
#local stratum 10
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# sed -r -i 's@#(local)@\1@' /etc/chrony.conf 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
allow 172.200.0.0/21
allow 127/8
local stratum 10    #即使自己未能通過網絡時間服務器同步到時間,也允許將本地時間作為標准時間授時給其它客戶端,而我們上面設置了禁用默認公網的時間服務器進行同步(換句話說,這里表示不去同步任何服務器的時間)。
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

6>.啟動chronyd並設置為開機自啟動

[root@hdp101.yinzhengjie.org.cn ~]# systemctl start chronyd
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# systemctl enable chronyd
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep chronyd
chronyd.service                               enabled 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

7>.查看chronyd正在使用哪個時間服務器進行時間同步

[root@hdp101.yinzhengjie.org.cn ~]# chronyc sources
210 Number of sources = 1
MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
^* hdp101.yinzhengjie.org.cn    10   6   377    39  -1609ns[-4236ns] +/-   16us
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

8>.hdp102.yinzhengjie.org.cn客戶端同步時間

[root@hdp102.yinzhengjie.org.cn ~]# yum -y install chrony
Loaded plugins: fastestmirror
Determining fastest mirrors
 * base: mirrors.tuna.tsinghua.edu.cn
 * extras: mirrors.tuna.tsinghua.edu.cn
 * updates: mirror.bit.edu.cn
base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
(1/2): extras/7/x86_64/primary_db                                                                                                                                                                                                                      | 159 kB  00:00:00     
(2/2): updates/7/x86_64/primary_db                                                                                                                                                                                                                     | 5.9 MB  00:00:01     
Resolving Dependencies
--> Running transaction check
---> Package chrony.x86_64 0:3.4-1.el7 will be installed
--> Processing Dependency: libseccomp.so.2()(64bit) for package: chrony-3.4-1.el7.x86_64
--> Running transaction check
---> Package libseccomp.x86_64 0:2.3.1-3.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                             Arch                                                            Version                                                              Repository                                                     Size
==============================================================================================================================================================================================================================================================================
Installing:
 chrony                                                              x86_64                                                          3.4-1.el7                                                            base                                                          251 k
Installing for dependencies:
 libseccomp                                                          x86_64                                                          2.3.1-3.el7                                                          base                                                           56 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  1 Package (+1 Dependent package)

Total download size: 306 k
Installed size: 788 k
Downloading packages:
(1/2): chrony-3.4-1.el7.x86_64.rpm                                                                                                                                                                                                                     | 251 kB  00:00:00     
(2/2): libseccomp-2.3.1-3.el7.x86_64.rpm                                                                                                                                                                                                               |  56 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         1.5 MB/s | 306 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Installing : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 
  Verifying  : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Verifying  : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 

Installed:
  chrony.x86_64 0:3.4-1.el7                                                                                                                                                                                                                                                   

Dependency Installed:
  libseccomp.x86_64 0:2.3.1-3.el7                                                                                                                                                                                                                                             

Complete!
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# yum -y install chrony
[root@hdp102.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# sed -r -i 's@(^server)@#\1@g' /etc/chrony.conf
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# vim /etc/chrony.conf
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# systemctl status chronyd
● chronyd.service - NTP client/server
   Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:chronyd(8)
           man:chrony.conf(5)
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# systemctl start chronyd
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# systemctl enable chronyd
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# systemctl list-units -t service | grep chronyd
  chronyd.service                    loaded active running NTP client/server
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# systemctl status chronyd
● chronyd.service - NTP client/server
   Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2020-01-31 23:38:04 CST; 20s ago
     Docs: man:chronyd(8)
           man:chrony.conf(5)
 Main PID: 5424 (chronyd)
   CGroup: /system.slice/chronyd.service
           └─5424 /usr/sbin/chronyd

Jan 31 23:38:04 hdp102.yinzhengjie.org.cn systemd[1]: Starting NTP client/server...
Jan 31 23:38:04 hdp102.yinzhengjie.org.cn chronyd[5424]: chronyd version 3.4 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG)
Jan 31 23:38:04 hdp102.yinzhengjie.org.cn systemd[1]: Started NTP client/server.
Jan 31 23:38:09 hdp102.yinzhengjie.org.cn chronyd[5424]: Selected source 172.200.1.101
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# 

9>.hdp103.yinzhengjie.org.cn客戶端同步時間

[root@hdp103.yinzhengjie.org.cn ~]# yum -y install chrony
Loaded plugins: fastestmirror
Determining fastest mirrors
 * base: mirrors.tuna.tsinghua.edu.cn
 * extras: mirrors.tuna.tsinghua.edu.cn
 * updates: mirror.bit.edu.cn
base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
(1/2): extras/7/x86_64/primary_db                                                                                                                                                                                                                      | 159 kB  00:00:00     
(2/2): updates/7/x86_64/primary_db                                                                                                                                                                                                                     | 5.9 MB  00:00:01     
Resolving Dependencies
--> Running transaction check
---> Package chrony.x86_64 0:3.4-1.el7 will be installed
--> Processing Dependency: libseccomp.so.2()(64bit) for package: chrony-3.4-1.el7.x86_64
--> Running transaction check
---> Package libseccomp.x86_64 0:2.3.1-3.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                             Arch                                                            Version                                                              Repository                                                     Size
==============================================================================================================================================================================================================================================================================
Installing:
 chrony                                                              x86_64                                                          3.4-1.el7                                                            base                                                          251 k
Installing for dependencies:
 libseccomp                                                          x86_64                                                          2.3.1-3.el7                                                          base                                                           56 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  1 Package (+1 Dependent package)

Total download size: 306 k
Installed size: 788 k
Downloading packages:
(1/2): chrony-3.4-1.el7.x86_64.rpm                                                                                                                                                                                                                     | 251 kB  00:00:00     
(2/2): libseccomp-2.3.1-3.el7.x86_64.rpm                                                                                                                                                                                                               |  56 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         1.5 MB/s | 306 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Installing : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 
  Verifying  : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Verifying  : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 

Installed:
  chrony.x86_64 0:3.4-1.el7                                                                                                                                                                                                                                                   

Dependency Installed:
  libseccomp.x86_64 0:2.3.1-3.el7                                                                                                                                                                                                                                             

Complete!
[root@hdp103.yinzhengjie.org.cn ~]# 
 
[root@hdp103.yinzhengjie.org.cn ~]# yum -y install chrony
[root@hdp103.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# sed -r -i 's@(^server)@#\1@g' /etc/chrony.conf
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# vim /etc/chrony.conf 
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# systemctl status chronyd
● chronyd.service - NTP client/server
   Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:chronyd(8)
           man:chrony.conf(5)
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# systemctl start chronyd
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# systemctl status chronyd
● chronyd.service - NTP client/server
   Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2020-01-31 23:42:09 CST; 1s ago
     Docs: man:chronyd(8)
           man:chrony.conf(5)
  Process: 5457 ExecStartPost=/usr/libexec/chrony-helper update-daemon (code=exited, status=0/SUCCESS)
  Process: 5453 ExecStart=/usr/sbin/chronyd $OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 5455 (chronyd)
   CGroup: /system.slice/chronyd.service
           └─5455 /usr/sbin/chronyd

Jan 31 23:42:09 hdp103.yinzhengjie.org.cn systemd[1]: Starting NTP client/server...
Jan 31 23:42:09 hdp103.yinzhengjie.org.cn chronyd[5455]: chronyd version 3.4 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG)
Jan 31 23:42:09 hdp103.yinzhengjie.org.cn systemd[1]: Started NTP client/server.
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# systemctl start chronyd
[root@hdp103.yinzhengjie.org.cn ~]# systemctl enable chronyd
Created symlink from /etc/systemd/system/multi-user.target.wants/chronyd.service to /usr/lib/systemd/system/chronyd.service.
[root@hdp103.yinzhengjie.org.cn ~]# 

10>.chronyd服務端修改時間

[root@hdp101.yinzhengjie.org.cn ~]# date 
Fri Jan 31 23:44:30 CST 2020
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# date -s "2020-01-31 15:45:00"
Fri Jan 31 15:45:00 CST 2020
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# hwclock -w
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# date 
Fri Jan 31 15:45:02 CST 2020
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# chronyc sources
210 Number of sources = 1
MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
^? hdp101.yinzhengjie.org.cn     0   7   377     -     +0ns[   +0ns] +/-    0ns
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# chronyc sourcestats -v
210 Number of sources = 1
                             .- Number of sample points in measurement set.
                            /    .- Number of residual runs with same sign.
                           |    /    .- Length of measurement set (time).
                           |   |    /      .- Est. clock freq error (ppm).
                           |   |   |      /           .- Est. error in freq.
                           |   |   |     |           /         .- Est. offset.
                           |   |   |     |          |          |   On the -.
                           |   |   |     |          |          |   samples. \
                           |   |   |     |          |          |             |
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
hdp101.yinzhengjie.org.cn   0   0     0     +0.000   2000.000     +0ns  4000ms
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

11>.chronyd客戶端同步時間

[root@hdp102.yinzhengjie.org.cn ~]# date 
Fri Jan 31 23:56:05 CST 2020
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# chronyc sourcestats -v
210 Number of sources = 1
                             .- Number of sample points in measurement set.
                            /    .- Number of residual runs with same sign.
                           |    /    .- Length of measurement set (time).
                           |   |    /      .- Est. clock freq error (ppm).
                           |   |   |      /           .- Est. error in freq.
                           |   |   |     |           /         .- Est. offset.
                           |   |   |     |          |          |   On the -.
                           |   |   |     |          |          |   samples. \
                           |   |   |     |          |          |             |
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
hdp101.yinzhengjie.org.cn   6   4   347     -0.085      3.307  -4050ns    64us
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# systemctl restart chronyd
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# chronyc sourcestats -v
210 Number of sources = 1
                             .- Number of sample points in measurement set.
                            /    .- Number of residual runs with same sign.
                           |    /    .- Length of measurement set (time).
                           |   |    /      .- Est. clock freq error (ppm).
                           |   |   |      /           .- Est. error in freq.
                           |   |   |     |           /         .- Est. offset.
                           |   |   |     |          |          |   On the -.
                           |   |   |     |          |          |   samples. \
                           |   |   |     |          |          |             |
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
hdp101.yinzhengjie.org.cn   3   3     4    -20.657   1349.596   -382ns  6474ns
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# date 
Fri Jan 31 15:57:07 CST 2020
[root@hdp102.yinzhengjie.org.cn ~]# 

 

三.部署Ambari集群

1>.瀏覽器訪問"http://hdp101.yinzhengjie.org.cn:8080/",使用默認的用戶名(admin)和密碼(admin)登錄成功后啟動安裝向導(如下圖所示,點擊"LAUNCH INSTALL WIZARD") 

2>.自定義集群名稱,並點擊下一步

3>.選擇HDP版本,並選擇本地軟件源倉庫

4>.刪除不相關操作系統的倉庫信息,只保留部署Ambari的操作系統的倉庫信息

5>.注冊主機

6>.確認匹配的主機名稱

7>.等待自動注冊

8>.等待集群自

9>.集群自檢完成

  如果此步驟和上圖一樣直接跳過下面的閱讀,如果你安裝其它節點存在問題,如下圖所示,別慌,查看日志解決問題才是正道。

  查看報錯信息,如下圖所示。

  根據報錯信息將hdp101.yinzhengjie.org.cn與hdp[102-103].yinzhengjie.org.cn不同的文件同步后,之前兩個失敗的節點最終狀態變為"Success"狀態。

10>.查看集群自建結果(此步驟可省略,直接進行下一步)

11>.自定義需要安裝的服務(服務勾選少了也沒多大關系,后面可以手動添加服務)

12>.勾選Ambari Metircs(會自動安裝監控已經勾選服務的狀態)服務並點擊"NEXT",如果彈出如下圖所示的提示,點擊"PROCEED ANYWAY"

13>.根據服務器實際資源分配Master服務角色安裝到具體的主機

14>.除了NFSGateway角色(如有需要后續可以自行安裝)不勾選,其它的slave服務都勾選

15>.為Grafana Admin和Activity Explorer's Admin服務設置密碼

16>.自定義各個服務對應數據的存儲路徑(由於我們使用的是root用戶安裝,因此盡管指定的目錄不存在,Ambari會通過root用戶自動創建的喲~)

17>.確認各個服務在操作系統中以哪個用戶身份運行(保持默認即可,直接下一步)

 18>.自定義安裝服務的配置並點擊下一步

19>.在Review步驟無需做任何配置,而是對之前幾個步驟的配置有一個大概的信息概覽,如果確認配置五五后點擊"DEPLOY"就正式部署啦

20>.點擊"DEPLOY"會開始初始化任務 

21>.初始化完就開始部署了  

22>.等待服務自動安裝,啟動和測試

23>.服務安裝成功

24>.安裝完成后點擊上圖的“COMPLETE”會自動進入到"Dashboard"的監控頁面

25>.手動測試HDFS服務是否安裝成功

[root@hdp101.yinzhengjie.org.cn ~]# hdfs dfs -ls /
Found 6 items
drwxr-xr-x   - hdfs   hdfs            0 2020-01-31 23:37 /atsv2
drwxr-xr-x   - hdfs   hdfs            0 2020-01-31 23:37 /hdp
drwxr-xr-x   - mapred hdfs            0 2020-01-31 23:37 /mapred
drwxrwxrwx   - hdfs   hdfs            0 2020-01-31 23:39 /tmp
drwxr-xr-x   - hdfs   hdfs            0 2020-01-31 23:37 /user
drwxrwxrwx   - mapred hadoop          0 2020-01-31 23:40 /yinzhengjie
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# hdfs dfs -df -h
Filesystem                              Size   Used  Available  Use%
hdfs://hdp101.yinzhengjie.org.cn:8020  4.0 T  1.2 G      4.0 T    0%
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM