一.下載安裝包
使用在線安裝特別慢，所有的安裝包加起來有9個G左右，所以需要先下載相關安裝包到服務器，通過配置本地源的方式來實現的離線安裝。
在私有集群中部署時，客戶機房的網絡情況不可預測，因此也需要使用離線部署的方式。
根據官方文檔
通過ambari安裝需要下載下面的四個主要包：
1.ambari 2.7.4 （1.9G） http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2. 7.4.0
2.HDP 3.1.4 (8.9G) http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.0/HDP-3.1.4.0-centos7-rpm.tar.gz
3.HDP-UTILS-1.1.0.22 (87M) http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
4.HDP-GPL-3.1.4(158k) http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.1.4.0/HDP-GPL-3.1.4.0-centos7-gpl.tar.gz
二.修改本機名和/etc/hosts文件
2.1 修改主機名(所有機器)
#對master sudo hostnamectl set-hostname master #對slave1 sudo hostnamectl set-hostname slave1 #對slave2 sudo hostnamectl set-hostname slave2 |
2.2 修改/etc/hosts文件。(所有機器)
這里主要是為了可以實現通過名稱來查找相應的服務器。
[root@master ~]# vi /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 172.16.130.202 master 172.16.130.203 slave1 172.16.130.204 slave2 |
2.3 修改/etc/sysconfig/network（所有機器）
[root@master ~]# vi /etc/sysconfig/network NETWORKING=yes |
接下來就可以通過測試一下是否可以通過ping主機名來找到對應的服務器。
[root@master ~]# ping slave1 PING slave1 (172.16.130.203) 56(84) bytes of data. 64 bytes from slave1 (172.16.130.203): icmp_seq=1 ttl=64 time=0.300 ms 64 bytes from slave1 (172.16.130.203): icmp_seq=2 ttl=64 time=0.191 ms 64 bytes from slave1 (172.16.130.203): icmp_seq=3 ttl=64 time=0.198 ms 64 bytes from slave1 (172.16.130.203): icmp_seq=4 ttl=64 time=0.202 ms ^C --- slave1 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3000ms rtt min/avg/max/mdev = 0.191/0.222/0.300/0.048 ms |
2.3 修改文件打開限制（所有機器）
[root@master ~]# vi /etc/security/limits.conf # End of file * soft nofile 65536 * hard nofile 65536 * soft nproc 131072 * hard nproc 131072 |
2.3 關閉防火牆（所有機器）
[root@master~]#systemctl disable firewalld [root@master~]#systemctl stop firewalld |
然后
[root@master ~]# cat /etc/selinux/config # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted |
三.同步時鍾
3.1 安裝ntp服務(所有機器)
[root@master ~]#yum -y install ntp |
3.2 設置master為主服務器，開啟nptd服務(主服務器)
[root@master ~]#vi /etc/ntp.conf |
如圖 restrict 改為集群所在網關及子網掩碼。server和fudge按圖中配置。
[root@master ~]# systemctl start ntpd.service #啟動服務 [root@master ~]# systemctl enable ntpd.service #開機自啟動 |
3.3 子節點設置同步(子節點)
主服務器開啟ntp服務器以后，子節點就不需要開啟了，因為當server與client之間的時間誤差過大時（可能是1000秒），處於對修改時間可能對系統和應用帶來不可預知的問題，NTP將停止時間同步！所以如果發現NTP啟動之后時間並不進行同步時，應該考慮到可能是時間差過大引起的，此時需要先手動進行時間同步！所以直接使用定時手動同步的方式就可以了。
[root@slave1 data]# crontab -e 0-59/10 * * * * /usr/sbin/ntpdate master crontab: installing new crontab [root@slave1 data]# crontab -l 0-59/10 * * * * /usr/sbin/ntpdate master |
四. SSH無密碼登錄(主節點)
配置master節點無密碼登錄到其他節點，在master節點上操作
[root@master ~]# ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Created directory '/root/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: SHA256:NMVPXezqq+MPf95MtP5bzaghnvXSljcC4BH/QFCvm+I root@master The key's randomart image is: +---[RSA 2048]----+ | o+. . o.| | o..o . .| | o +o . . | | . + oo .| | S o.o ..| | . .o..+o| | ..o=o.o=| | ...+oB*+=| | Eo.+**BO| +----[SHA256]-----+ [root@master ~]# ssh-copy-id slave1 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub" The authenticity of host 'slave1 (192.168.12.102)' can't be established. ECDSA key fingerprint is SHA256:GG4+viH+NDpVisSLMobnjEYOs171XRVUDoG85EI+mI8. ECDSA key fingerprint is MD5:f3:bf:63:a6:89:61:c2:08:3c:7d:2a:ed:13:be:d3:ae. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@slave1's password: Permission denied, please try again. root@slave1's password: Permission denied, please try again. root@slave1's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'slave1'" and check to make sure that only the key(s) you wanted were added. [root@master ~]# ssh-copy-id slave2 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub" The authenticity of host 'slave2 (192.168.12.103)' can't be established. ECDSA key fingerprint is SHA256:Yv86oeVgTcxv8FCm5FLpiEUBTAnXgH2GOkgGboyoR+8. ECDSA key fingerprint is MD5:8c:ae:b6:c4:f1:77:1e:ab:a5:16:38:a2:d2:da:4c:af. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@slave2's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'slave2'" and check to make sure that only the key(s) you wanted were added. |
測試是否實現了無密碼登錄
[root@master hdp274]# ssh slave1 date ;ssh slave2 date;ssh master date; Thu Sep 19 19:44:37 CST 2019 Thu Sep 19 19:44:37 CST 2019 Thu Sep 19 19:44:39 CST 2019 |
將創建的秘鑰拷貝出來，因為后面ambari安裝的時候需要上傳這個秘鑰。創建秘鑰是在隱藏文件夾/root/.ssh/下面的，所以需要先把秘鑰拷貝到可見區域，然后拷貝到電腦上。下面名為id_rsa的文件就是秘鑰文件。
[root@master ~]# cd /root/.ssh/ [root@master .ssh]# ls authorized_keys id_rsa id_rsa.pub known_hosts [root@master .ssh]# cp id_rsa /root/ [root@master .ssh]# ls /root/ anaconda-ks.cfg id_rsa jdk-8u161-linux-x64.rpm |
五.修改yum源，實現離線安裝
5.1 安裝httpd服務(主服務器)
[root@master ~]# yum -y install httpd [root@master ~]# service httpd restart Redirecting to /bin/systemctl restart httpd.service [root@master ~]# chkconfig httpd on |
5.2 將上面下載的四個包放到/var/www/html目錄下（主服務器）
[root@master hdp274]# ls ambari-2.7.4.0-centos7.tar.gz HDP-GPL-3.1.4.0-centos7-gpl.tar.gz HDP-3.1.4.0-centos7-rpm.tar.gz HDP-UTILS-1.1.0.22-centos7.tar.gz [root@master hdp274]# cp * /var/www/html/ambari/ [root@master hdp274]# cd /var/www/html/ambari/ [root@master ambari]# ls ambari-2.7.4.0-centos7.tar.gz HDP-GPL-3.1.4.0-centos7-gpl.tar.gz HDP-3.1.4.0-centos7-rpm.tar.gz HDP-UTILS-1.1.0.22-centos7.tar.gz |
5.3 解壓壓縮包
[root@master ambari]# tar -zxvf ambari-2.7.4.0-centos7.tar.gz [root@master ambari]# tar -zxvf HDP-3.1.4.0-centos7-rpm.tar.gz [root@master ambari]# tar -zxvf HDP-GPL-3.1.4.0-centos7-gpl.tar.gz [root@master ambari]# mkdir HDP-UTILS-1.1.0.22 [root@master ambari]# tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C HDP-UTILS-1.1.0.22 [root@master ambari]# rm *.gz -rf |
5.4 通過瀏覽器查看是否能夠訪問這個源了。
5.5 制作本地源
（1）安裝本地源制作相關工具（主服務器）
[root@master ambari]# yum install yum-utils createrepo yum-plugin-priorities -y [root@master ambari]# createrepo ./ |
（2）修改文件里面的源地址（主服務器）
vi ambari/centos7/2.7.4.0-118/ambari.repo |
[root@master ambari]# vi HDP-UTILS-1.1.0.22/HDP-UTILS/centos7/1.1.0.22/hdp-utils.repo |
將修改好的文件拷貝到所有機器的/etc/yum.repos.d/路徑下
[root@master ambari]# cp ambari/centos7/2.7.4.0-118/ambari.repo /etc/yum.repos.d/ [root@master ambari]# scp /etc/yum.repos.d/ambari.repo slave1:/etc/yum.repos.d/ [root@master ambari]# scp /etc/yum.repos.d/ambari.repo slave2:/etc/yum.repos.d/ [root@master ambari]# cp HDP-UTILS-1.1.0.22/HDP-UTILS/centos7/1.1.0.22/hdp-utils.repo /etc/yum.repos.d/ [root@master ambari]# scp /etc/yum.repos.d/hdp-utils.repo slave1:/etc/yum.repos.d/ [root@master ambari]# scp /etc/yum.repos.d/hdp-utils.repo slave2:/etc/yum.repos.d/ |
清除yum緩存
[root@master ambari]# yum clean all [root@master ambari]# yum makecache [root@master ambari]# yum repolist |
六.安裝ambari-server
6.1 安裝和配置ambari-server
[root@master ~]# yum -y install ambari-server [root@master ~]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'enabled' SELinux mode is 'permissive' WARNING: SELinux is set to 'permissive' mode and temporarily disabled. OK to continue [y/n] (y)? y Customize user account for ambari-server daemon [y/n] (n)? y Enter user account for ambari-server daemon (root):root Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK ============================================================================== Enter choice (1): 1 JDK already exists, using /var/lib/ambari-server/resources/jdk-8u112-linux-x64.tar.gz Installing JDK to /usr/jdk64/ Successfully installed JDK to /usr/jdk64/ JCE Policy archive already exists, using /var/lib/ambari-server/resources/jce_policy-8.zip Installing JCE policy... Check JDK version for Ambari Server... JDK version found: 8 Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server. Checking GPL software agreement... GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? y Configuring database... ============================================================================== Choose one of the following options: [1] - PostgreSQL (Embedded) [2] - Oracle [3] - MySQL / MariaDB [4] - PostgreSQL [5] - Microsoft SQL Server (Tech Preview) [6] - SQL Anywhere [7] - BDB ============================================================================== Enter choice (1): 1 Database admin user (postgres): admin Database name (ambari): admi^H^H Postgres schema (ambari): ^C Aborting ... Keyboard Interrupt. [root@master ~]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'enabled' SELinux mode is 'permissive' WARNING: SELinux is set to 'permissive' mode and temporarily disabled. OK to continue [y/n] (y)? y Customize user account for ambari-server daemon [y/n] (n)? y Enter user account for ambari-server daemon (root):root Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... Do you want to change Oracle JDK [y/n] (n)? y [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK ============================================================================== Enter choice (1): 1 JDK already exists, using /var/lib/ambari-server/resources/jdk-8u112-linux-x64.tar.gz Installing JDK to /usr/jdk64/ Successfully installed JDK to /usr/jdk64/ JCE Policy archive already exists, using /var/lib/ambari-server/resources/jce_policy-8.zip Installing JCE policy... Check JDK version for Ambari Server... JDK version found: 8 Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server. Checking GPL software agreement... Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? n Configuring database... Default properties detected. Using built-in database. Configuring ambari database... Checking PostgreSQL... Running initdb: This may take up to a minute. About to start PostgreSQL Configuring local database... Configuring PostgreSQL... Backup for pg_hba found, reconfiguration not required Creating schema and user... done. Creating tables... done. Extracting system views... ambari-admin-2.7.4.0.118.jar .... Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it... Adjusting ambari-server permissions and ownership... Ambari Server 'setup' completed successfully. |
6.2 啟動ambari
[root@master ~]# ambari-server start |
成功啟動后在瀏覽器輸入Ambari地址：
七.設置Use Local Repository
配置成本地源
http://master/ambari/HDP/centos7/3.1.4.0-315
http://master/ambari/HDP-GPL/centos7/3.1.4.0-315
http://master/ambari/HDP-UTILS-1.1.0.22/HDP-UTILS/centos7/1.1.0.22
注意:需要修改本機hosts文件
172.16.130.202 master 172.16.130.203 slave1 172.16.130.204 slave2 |
八.指定要安裝集群的主機列表和之前的id_rsa文件
九.安裝hive前需要下載jdbc驅動
根據下面的問題單上的描述，得知ambari安裝hive時的說明有誤。https://issues.apache.org/jira/browse/AMBARI-23355
[root@fidcmpt01 hdp]# ls mysql-connector-java-8.0.17.jar [root@fidcmpt01 hdp]# pwd /opt/hdp [root@fidcmpt01 hdp]# ambari-server setup --jdbc-db=mysql --jdbc-driver=/opt/hdp/mysql-connector-java-8.0.17.jar |
附錄一:卸載集群
1）停止Ambari中的所有服務
2）在所有集群節點上運行python腳本
[root@master var]# python /usr/lib/python2.6/site-packages/ambari_agent/HostCleanup.py –silent –skip=users |
3）停止ambari-server和所有機器上的ambari-agent,並卸載
[root@master var]# ambari-server stop [root@master var]# ambari-agent stop [root@master opt]# yum erase ambari-server [root@master opt]# yum erase ambari-agent [root@master ~]# yum erase postgresql [root@master opt]# rm -rf /etc/ambari-server [root@master opt]# rm -rf /usr/lib/python2.6/site-packages/ambari_server [root@master opt]# rm -rf /etc/ambari-agent/ [root@master opt]# rm -rf /usr/lib/python2.6/site-packages/ambari_agent [root@master opt]# rm -rf /usr/lib/python2.6/site-packages/resource_monitoring |
3）刪除數據庫。 刪除MySQL和Postgres的實例，以便Ambari安裝和配置新的數據庫。
yum remove mysql mysql-server yum erase postgresql rm -rf /var/lib/pgsql rm -rf /var/lib/mysql |
4）刪除數據庫。 刪除MySQL和Postgres的實例，以便Ambari安裝和配置新的數據庫。
附錄二.出錯后重裝ambari-server
如果出現錯誤，請注意查看日志，根據具體的錯誤內容進行處理，默認ambari-server的日志在/var/log/ambari-server/ambari-server.log里面。如果在處理日志的過程中或者后面安裝的過程中出現一些莫名的錯誤，可以重置的安裝。如果上面進行的默認數據庫的配置，可以使用下面的代碼重置ambari-server
[root@master ~]# ambari-server stop [root@master ~]# ambari-server reset [root@master ~]# ambari-server setup |
如果在安裝的過程中出現了錯誤，又想重新安裝，可以在ambari-server開啟的情況下，執行下面的語句來移除已安裝的包，然后再通過不同的情況選擇上面兩種方式的一種對ambari-server進行重置
python /usr/lib/python2.6/site-packages/ambari_agent/HostCleanup.py --silent |
附錄三. YARN和MAPREDUCE2的服務刪除
由於YARN和MAPREDUCE2這兩個服務在ambari中是相互依賴的，無法在界面上刪除，可以使用接口方式。
curl -u admin:admin -H "X-Requested-By: ambari" -X DELETE http://localhost:8080/api/v1/clusters/hadoop2/services/YARN |