Ambari2.7.4+HDP3.1.4安裝 Centos7離線安裝

本文轉載自查看原文 2020-03-04 15:22 9556 大數據

1. 本次安裝簡單介紹

1.1Ambari

Ambari是一種基於Web的工具，支持Apache Hadoop集群的創建、管理和監控。

Ambari已支持大多數Hadoop組件，包括HDFS、MapReduce、Hive、Pig、 Hbase、Zookeeper、Sqoop和Hcatalog等。Apache Ambari 支持HDFS、MapReduce、Hive、Pig、Hbase、Zookeepr、Sqoop和Hcatalog等的集中管理。也是5個頂級hadoop管理工具之一。

Ambari 自身也是一個分布式架構的軟件，主要由兩部分組成：Ambari Server 和 Ambari Agent。簡單來說，用戶通過 Ambari Server 通知 Ambari Agent 安裝對應的軟件；Agent 會定時地發送各個機器每個軟件模塊的狀態給 Ambari Server，最終這些狀態信息會呈現在 Ambari 的 GUI，方便用戶了解到集群的各種狀態，並進行相應的維護。

1.2 HDP

HDP是hortonworks的軟件棧，里面包含了hadoop生態系統的所有軟件項目，比如HBase,Zookeeper,Hive,Pig等等。

1.3 HDP-UTILS

HDP-UTILS是工具類庫。

2. Ambari搭建前環境准備

2.1 版本介紹

截止到2020.03.03，Ambari的最新版本為2.7.5，HDP的最新版本為3.1.5

通過 https://supportmatrix.hortonworks.com/ 可以查詢Ambari和HDP各個版本支持情況

我這里選擇的是2.7.4版本，所以使用的HDP對應版本為3.1.4

2.2 搭建環境

2.2.1 所用到的環境列表

環境	版本
Linux	Centos7物理機*4（英文系統）
Ambari	2.7.4
HDP	3.1.4
HDP-UTILS	1.1.0.2
MySQL-MariaDB/ MySQL	10.4.12/5.7
OracleJDK8	1.8.0_201
Nginx	1.17.8(已有環境，選用，可不用)

2.2.2 環境下載

Ambari在線安裝特別慢，所以使用離線安裝，建議使用迅雷下載

名稱	地址
ambari	http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.4.0/ambari-2.7.4.0-centos7.tar.gz
HDP	http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.0/HDP-3.1.4.0-centos7-rpm.tar.gz
HDP-UTILS	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz

（2021.06.19更新）鑒於有很多用戶反映離線包已收費，這里提供一下白度雲地址：鏈接: https://pan.baidu.com/s/1obECabArXjHmZXVuxxpdYQ 提取碼: ma4e

JDK8環境

名稱	地址
OracleJDK	https://www.oracle.com/java/technologies/javase-jdk8-downloads.html

2.4 集群節點划分

節點名稱	局域網ip地址	服務	內存	硬盤
master（主節點）	192.168.105.137	Ambari/HDP Ambari Server MariaDB /MySql	4G	450G
slave1（子節點）	192.168.105.191	Compute node	4G	450G
slave2（子節點）	192.168.105.192	Compute node	4G	450G
slave3（子節點）	192.168.105.193	Compute node	4G	450G

2.5 修改網絡配置（所有節點）

用putty或者xshell連接執行

 # cd /etc/sysconfig/network-scripts

找到ifcfg-en開頭，后面的數字由每台機器生成各有不同，直接vi編輯即可

修改前：

修改后：

TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
IPADDR=192.168.105.137
NETMASK=255.255.255.0
GATEWAY=192.168.105.1
DNS1=114.114.114.114
DNS2=8.8.8.8
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=eno1
UUID=6c053b64-6581-4665-b3fc-c22f79c58848
DEVICE=eno1
ONBOOT=yes

重啟網絡服務：# service network start

ping一下局域網其他機器通了即可

3. 系統環境配置

3.1 安裝 JDK（所有節點）

Linux自帶的jdk或者是通過yum安裝的jdk都是openjdk
最好是使用開源的oracle jdk，缺失部分功能，如果直接安裝oracle的jdk，第三方的依賴包不會安裝，我們這里是通過yum安裝openjdk，並同時安裝了第三方依賴包，然后卸載openjdk，通過自己來安裝oracle的jdk，就能解決依賴問題。

3.1.1 安裝openjdk

[root@master ~]# yum -y install java
[root@master ~]# java -version
openjdk version "1.8.0_212"
OpenJDK Runtime Environment (build 1.8.0_212-c061)
OpenJDK 64-Bit Server VM (build 25.212-c061, mixed mode)
[root@master ~]# rpm -qa|grep java
javapackages-tools-3.4.1-11.el7.noarch
java-1.8.0-openjdk-1.8.0.212-0.c061.el7_7.x86_64
python-javapackages-3.4.1-11.el7_7.noarch
tzdata-java-2018d-1.el7_7.noarch
java-1.8.0-openjdk-headless-1.8.0.212-0.c061.el7_7.x86_64

3.1.2卸載OpenJDK

[root@master ~]# rpm -e --nodeps java-1.8.0-openjdk-headless-1.8.0.212-0.c061.el7_7.x86_64
[root@master ~]# rpm -e --nodeps java-1.8.0-openjdk-1.8.0.212-0.c061.el7_7.x86_64

安裝和卸載的時候注意一下open jdk的版本號

3.1.3安裝之前下好的 OracleJDK （注意文件位置）

[root@master ~]# rpm -ivh jdk-8u201-linux-x64.rpm 
准備中...                          ################################# [100%]
正在升級/安裝...
   1:jdk1.8-2000:1.8.0_201-fcs        ################################# [100%]
Unpacking JAR files...
    tools.jar...
    plugin.jar...
    javaws.jar...
    deploy.jar...
    rt.jar...
    jsse.jar...
    charsets.jar...
    localedata.jar...
[root@master ~]# java -version
java version "1.8.0_201"
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)
[root@master ~]# javac -version
javac 1.8.0_201

3.2 修改節點名稱（所有節點）

3.2.1 修改/etc/hosts文件（所有節點）

方便通過名稱來查找相應節點

[root@master ~]# vi /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1             localhost localhost.localdomain localhost6 localhost6.localdomain6


192.168.105.137 master
192.168.105.191 slave1
192.168.105.192 slave2
192.168.105.193 slave3

3.2.2 修改/etc/sysconfig/network（所有節點）

各節點改成相對應的節點名即可

[root@master ~]# vi /etc/sysconfig/network
# Created by anaconda
NETWORKING=yes
HOSTNAME=master

[root@master ~]# hostnamectl set-hostname master
[root@master ~]# hostname
master

接下來通過測試ping各個節點名稱是否調通

[root@master ~]# ping slave1
PING slave1 (192.168.105.191) 56(84) bytes of data.
64 bytes from slave1 (192.168.105.191): icmp_seq=1 ttl=64 time=0.320 ms
64 bytes from slave1 (192.168.105.191): icmp_seq=2 ttl=64 time=0.282 ms
64 bytes from slave1 (192.168.105.191): icmp_seq=3 ttl=64 time=0.278 ms
64 bytes from slave1 (192.168.105.191): icmp_seq=4 ttl=64 time=0.269 ms
64 bytes from slave1 (192.168.105.191): icmp_seq=5 ttl=64 time=0.279 ms
64 bytes from slave1 (192.168.105.191): icmp_seq=6 ttl=64 time=0.277 ms
64 bytes from slave1 (192.168.105.191): icmp_seq=7 ttl=64 time=0.278 ms
64 bytes from slave1 (192.168.105.191): icmp_seq=8 ttl=64 time=0.275 ms
^C
--- slave1 ping statistics ---
8 packets transmitted, 8 received, 0% packet loss, time 6999ms
rtt min/avg/max/mdev = 0.269/0.282/0.320/0.018 ms

3.3 更換所有節點為阿里巴巴yum源（所有節點）

阿里巴巴開發者社區url： https://developer.aliyun.com/mirror/
選擇Centos

可直接運行如下

[root@master ~]# mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
[root@master ~]# curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
[root@master ~]# yum makecache

3.4 同步時間ntp

3.4.1 安裝ntp服務（所有節點）

[root@master ~]# yum -y install ntp

啟動服務，查看狀態並設置開機自啟

[root@master ~]# systemctl start ntpd.service
 [root@master ~]# systemctl status ntpd.service
● ntpd.service - Network Time Service
   Loaded: loaded (/usr/lib/systemd/system/ntpd.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2020-03-02 15:41:28 CST; 23h ago
 Main PID: 3909 (ntpd)
   CGroup: /system.slice/ntpd.service
           └─3909 /usr/sbin/ntpd -u ntp:ntp -g


Mar 02 15:41:28 master ntpd[3909]: Listen normally on 4 lo ::1 UDP 123
Mar 02 15:41:28 master ntpd[3909]: Listen normally on 5 eno1 fe80::7b94:c6e6:5673:c105 UDP 123
Mar 02 15:41:28 master ntpd[3909]: Listening on routing socket on fd #22 for interface updates
Mar 02 15:41:28 master systemd[1]: Started Network Time Service.
Mar 02 15:41:28 master ntpd[3909]: 0.0.0.0 c016 06 restart
Mar 02 15:41:28 master ntpd[3909]: 0.0.0.0 c012 02 freq_set kernel 0.000 PPM
Mar 02 15:41:28 master ntpd[3909]: 0.0.0.0 c011 01 freq_not_set
Mar 02 15:41:35 master ntpd[3909]: 0.0.0.0 c614 04 freq_mode
Mar 02 15:58:21 master ntpd[3909]: 0.0.0.0 0612 02 freq_set kernel 8.697 PPM
Mar 02 15:58:21 master ntpd[3909]: 0.0.0.0 0615 05 clock_sync
[root@master ~]# systemctl enable ntpd.service

3.5 關閉防火牆（所有節點）

查看防火牆狀態

[root@master ~]# systemctl status firewalld.service

關閉防火牆

[root@master ~]# systemctl stop firewalld.service

設置開機不啟動

[root@master ~]# systemctl disable firewalld.service

查看是否成功

[root@master ~]# systemctl is-enabled firewalld.service
disabled

3.6 關閉Selinux和THP（所有節點）

3.6.1關閉Selinux

查看Selinux狀態

[root@master ~]# sestatus

關閉Selinux，提示沒有vim用yum裝一個咯或者用vi

[root@master ~]# vim /etc/sysconfig/selinux
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

3.6.2關閉THP

查看狀態

[root@yum ~]# cat /sys/kernel/mm/transparent_hugepage/defrag 
[always] madvise never
[root@yum ~]# cat /sys/kernel/mm/transparent_hugepage/enabled 
[always] madvise never

關閉THP並給予文件權限

[root@yum ~]# vim /etc/rc.d/rc.local
#!/bin/bash
# THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES
#
# It is highly advisable to create own systemd services or udev rules
# to run scripts during boot instead of using this file.
#
# In contrast to previous versions due to parallel execution during boot
# this script will NOT be run after all other services.
#
# Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure
# that this script will be executed during boot.


touch /var/lock/subsys/local

if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
        echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
        echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
[root@yum ~]# chmod +x /etc/rc.d/rc.local

3.7 修改文件打開最大限制（所有節點）

設置並查看

[root@master ~]# vim /etc/security/limits.conf 
# End of file 
* soft nofile 65536 
* hard nofile 65536 
* soft nproc 131072 
* hard nproc 131072
 
[root@master ~]# ulimit -Sn 
[root@master ~]# ulimit -Hn

3.8 SSH無密碼登陸（主節點）

回車通過，輸入密碼等確認通過即可

[root@master ~]# ssh-keygen -t rsa
[root@master ~]# ssh-copy-id slave1
root@slave1's password: 
Number of key(s) added: 1
Now try logging into the machine, with:   "ssh 'slave1'"
and check to make sure that only the key(s) you wanted were added.
[root@master ~]# ssh-copy-id slave2
Are you sure you want to continue connecting (yes/no)? yes
root@slave2's password: 
Number of key(s) added: 1
[root@master ~]# ssh-copy-id slave3
Are you sure you want to continue connecting (yes/no)? yes
root@slave3's password: 
Number of key(s) added: 1
[root@master ~]# ssh-copy-id master
Are you sure you want to continue connecting (yes/no)? yes
root@master's password: 
Number of key(s) added: 1
Now try logging into the machine, with:   "ssh 'master'"
and check to make sure that only the key(s) you wanted were added

測試是否實現無密碼登錄，無輸入密碼即可通過

[root@master ~]# ssh slave1 date ;ssh slave2 date;ssh slave3 date;ssh master date;
Tue Mar  3 16:32:07 CST 2020
Tue Mar  3 16:32:07 CST 2020
Tue Mar  3 16:32:07 CST 2020
Tue Mar  3 16:32:08 CST 2020

將剛剛創建的秘鑰拷出來，后面ambari安裝的時候需要上傳這個秘鑰。創建秘鑰是在隱藏文件夾/root/.ssh/下面的，所以需要先把秘鑰拷貝到可見區域，然后拷貝到本機上。

[root@master ~]# cd /root/.ssh/
[root@master .ssh]# ls
authorized_keys  id_rsa  id_rsa.pub  known_hosts
[root@master .ssh]# cp id_rsa /root/
[root@master .ssh]# ls /root/
ambari                               
ambari-2.7.4.0-centos7.tar.gz        id_rsa                              
anaconda-ks.cfg      install.sh  jdk-8u201-linux-x64.rpm

到這里我們先reboot重啟一下

4. 實現離線安裝，更換yum源

4.1 文件目錄展示

由於我這里主機已經安裝了Nginx，所以文件目錄的方式我這里寫了兩種展示目錄的方式，4.1.1和4.1.2選擇一種即可

4.1.1 http服務方式

[root@master ~]# yum -y install httpd
[root@master ~]# service httpd restart
Redirecting to /bin/systemctl restart httpd.service
[root@master ~]# chkconfig httpd on

安裝完成后，會生成 /var/www/html目錄（相當於Tomcat的webapps目錄）

將之前下的Ambari、HDP、HDP-UTILS三個包放到 /var/www/html 下

[root@master ~]# cd /var/www/html/
[root@master html]# mkdir ambari
拷貝文件到ambari下面
[root@master html]# cd ambari/
[root@master ambari]# ls
ambari-2.7.4.0-centos7.tar.gz   HDP-3.1.4.0-centos7-rpm.tar.gz  HDP-UTILS-1.1.0.22-centos7.tar.gz
[root@master ambari]# tar -zxvf ambari-2.7.4.0-centos7.tar.gz
[root@master ambari]# tar -zxvf HDP-3.1.4.0-centos7-rpm.tar.gz 
[root@master ambari]# mkdir HDP-UTILS
[root@master ambari]# tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C HDP-UTILS
[root@master ambari]# rm -rf ambari--2.7.4.0-centos7.tar.gz HDP-2.6.3.0-centos7-rpm.tar.gz HDP-UTILS-1.1.0.22-centos7.tar.gz 
[root@master ambari]# ls
ambari  HDP  HDP-UTILS

4.1.2 nginx服務方式

找到nginx的配置，在nginx server中的location中增加：

 server {
        listen 8001;
        location / {
            root /www//html/ambari/;
            autoindex on;
            autoindex_localtime on;
            autoindex_exact_size off;
        }
     }

參數說明：
root /data/image/
你需要開啟瀏覽的目錄，放訪問
http://IP 時候顯示的就是/data/image目錄下的內容

autoindex_localtime on;
默認為off，顯示的文件時間為GMT時間。
改為on后，顯示的文件時間為文件的服務器時間

autoindex_exact_size off;
默認為on，顯示出文件的確切大小，單位是bytes。
改為off后，顯示出文件的大概大小，單位是kB或者MB或者GB

listen 8001;
訪問端口號，

[root@master ~]# cd /var/www/
[root@master html]# mkdir ambari
拷貝文件到ambari下面
[root@master html]# cd ambari/
[root@master ambari]# ls
ambari-2.7.4.0-centos7.tar.gz   HDP-3.1.4.0-centos7-rpm.tar.gz  HDP-UTILS-1.1.0.22-centos7.tar.gz
[root@master ambari]# tar -zxvf ambari-2.7.4.0-centos7.tar.gz
[root@master ambari]# tar -zxvf HDP-3.1.4.0-centos7-rpm.tar.gz 
[root@master ambari]# mkdir HDP-UTILS
[root@master ambari]# tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C HDP-UTILS
[root@master ambari]# rm -rf ambari--2.7.4.0-centos7.tar.gz HDP-2.6.3.0-centos7-rpm.tar.gz HDP-UTILS-1.1.0.22-centos7.tar.gz 
[root@master ambari]# ls
ambari  HDP  HDP-UTILS

訪問 http://192.168.105.137/ambari/ 看是否能訪問

4.2 制作本地源

4.2.1 安裝本地源制作相關工具（主節點）

[root@master ambari]# yum install yum-utils createrepo yum-plugin-priorities -y
[root@master ambari]#  createrepo  ./

4.2.2 修改文件里面的源地址（主節點）

注意文件路徑，以自己為准

[root@master ambari]# vi ambari/centos7/2.7.4.0-118/ambari.repo
#VERSION_NUMBER=2.7.4.0-118
[ambari-2.7.4.0]
#json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
name=ambari Version - ambari-2.7.4.0
baseurl=http://192.168.105.137:8001/ambari/centos7/2.7.4.0-118
gpgcheck=1
gpgkey=http://192.168.105.137:8001/ambari/centos7/2.7.4.0-118/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[root@master ambari]# cp ambari/centos7/2.7.4.0-118/ambari.repo /etc/yum.repos.d/
[root@master ambari]# vi HDP/centos7/3.1.4.0-315/hdp.repo
#VERSION_NUMBER=3.1.4.0-315
[HDP-3.1.4.0]
name=HDP Version - HDP-3.1.4.0
baseurl=http://192.168.105.137:8001/HDP/centos7/3.1.4.0-315
gpgcheck=1
gpgkey=http://192.168.105.137:8001/HDP/centos7/3.1.4.0-315/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1



[HDP-UTILS-1.1.0.22]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
baseurl=http://192.168.105.137:8001/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22
gpgcheck=1
gpgkey=http://192.168.105.137:8001/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[root@master ambari]# cp HDP/centos7/3.1.4.0-315/hdp.repo /etc/yum.repos.d/

以上就已經創建好了

使用yum的命令清一下緩存

[root@master ambari]# yum clean all
[root@master ambari]# yum makecache
[root@master ambari]# yum repolist

4.2.3 將創建好的源文件拷貝到子節點（主節點）

[root@master ambari]#cd /etc/yum.repos.d
[root@master yum.repos.d]# scp ambari.repo slave1:/etc/yum.repos.d/
ambari.repo                                                                                        100%  274    24.8KB/s   00:00    
[root@master yum.repos.d]# scp ambari.repo slave2:/etc/yum.repos.d/
ambari.repo                                                                                        100%  274   294.9KB/s   00:00    
[root@master yum.repos.d]# scp ambari.repo slave3:/etc/yum.repos.d/
ambari.repo                                                                                        100%  274   541.7KB/s   00:00    
[root@master yum.repos.d]# scp hdp.repo slave1:/etc/yum.repos.d/
hdp.repo                                                                                           100%  482     603.6KB/s   00:00    
[root@master yum.repos.d]# scp hdp.repo slave2:/etc/yum.repos.d/
hdp.repo                                                                                           100%  482   541.9KB/s   00:00    
[root@master yum.repos.d]# scp hdp.repo slave3:/etc/yum.repos.d/
hdp.repo

5. 安裝ambari-server

這里介紹兩種模式，一種是默認postgresql數據庫的安裝方式，還有就是mysql方式，這里我們用到的是mysql，大家根據自身選一種即可

無論是用哪種，首先都要安裝ambari-server

[root@master yum.repos.d]# yum -y install ambari-server

5.1默認數據庫PostgreSQL安裝方式（主節點）

[root@master yum.repos.d]# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? n
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
Enter choice (1): 2
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/java/jdk1.8.0_202
Validating JDK on Ambari Server...done.
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? n
Configuring database...
Default properties detected. Using built-in database.
Configuring ambari database...
Checking PostgreSQL...
Running initdb: This may take up to a minute.
Initializing database ... OK



About to start PostgreSQL
Configuring local database...
Configuring PostgreSQL...
Restarting PostgreSQL
Creating schema and user...
done.
Creating tables...
done.
Extracting system views...
ambari-admin-2.7.4.0-118.jar
...........
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

啟動ambari

[root@master yum.repos.d]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start...........................................
Server started listening on 8080


DB configs consistency check: no errors and warnings were found.
Ambari Server 'start' completed successfully.

成功啟動后在瀏覽器輸入Ambari地址：
http://192.168.105.137:8080 即可看到

5.2 MySql安裝方式（主節點）

5.2.1 安裝MySql

[root@master ~]# wget http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
[root@master ~]# rpm -ivh mysql-community-release-el7-5.noarch.rpm
[root@master ~]# yum install mysql-community-serve

5.2.2 啟動mysql，設置開機啟動

[root@master ~]# service mysqld start
[root@master ~]# vi /etc/rc.local
#添加service mysqld start

5.2.3 登錄進mysql，初始化設置root密碼

[root@master ~]# mysql -uroot 


設置登錄密碼
mysql> set password for 'root'@'localhost' = password('yourPassword');

添加遠程登錄用戶
mysql> grant all privileges on *.* to 'root'@'%' identified by 'yourPassword';

遠程登錄
#mysql -uroot -h ip(遠程的ip地址) -p

5.2.4登錄mysql，執行以下的語句

CREATE DATABASE ambari;  
use ambari;  
CREATE USER 'ambari'@'%' IDENTIFIED BY 'ambari';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';  
CREATE USER 'ambari'@'localhost' IDENTIFIED BY 'ambar';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'localhost';  
CREATE USER 'ambari'@'master' IDENTIFIED BY 'ambari';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'master';  
FLUSH PRIVILEGES;  
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql  
show tables;  
use mysql;  
select Host User Password from user where user='ambari';  
CREATE DATABASE hive;  
use hive;  
CREATE USER 'hive'@'%' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';  
CREATE USER 'hive'@'localhost' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'localhost';  
CREATE USER 'hive'@'master' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'master';  
FLUSH PRIVILEGES;  
CREATE DATABASE oozie;  
use oozie;  
CREATE USER 'oozie'@'%' IDENTIFIED BY 'oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';  
CREATE USER 'oozie'@'localhost' IDENTIFIED BY 'oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'localhost';  
CREATE USER 'oozie'@'master' IDENTIFIED BY 'oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'master';  
FLUSH PRIVILEGES;

5.2.5 建立mysql與ambari-server的jdbc連接

[root@master yum.repos.d]# yum install mysql-connector-java
[root@master yum.repos.d]# ls /usr/share/java
[root@master yum.repos.d]# cp /usr/share/java/mysql-connector-java.jar /var/lib/ambari-server/resources/mysql-jdbc-driver.jar
[root@master yum.repos.d]# vi /etc/ambari-server/conf/ambari.properties
添加server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar

5.2.6設置ambari-server

[root@master yum.repos.d]# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'permissive'
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)? y
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):ambari #ambari-server 賬號。如果直接回車就是默認選擇root用戶
如果輸入已經創建的用戶就會顯示：
Adjusting ambari-server permissions and ownership...
Checking firewall status...
WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports.
OK to continue [y/n] (y)? y
Checking JDK... #設置JDK。輸入：2
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
Enter choice (1): 2 #輸入你自己的jdk位置/usr/java/jdk1.8.0_201-amd64
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/java/jdk1.8.0_201-amd64
Validating JDK on Ambari Server...done.
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? n
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y   #數據庫配置。選擇：y
Configuring database...
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
Enter choice (1): 3  #選擇mysql數據庫類型。輸入：3
# 設置數據庫的具體配置信息，根據實際情況輸入，如果和括號內相同，則可以直接回車。
Hostname (localhost): 
Port (3306): 
Database name (ambari): 
Username (ambari): ambari
Enter Database Password (bigdata): ambari
Re-enter password: ambari
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
ambari-admin-2.7.4.0.118.jar
....
Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it...
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.
[root@master yum.repos.d]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start...........................................
Server started listening on 8080


DB configs consistency check: no errors and warnings were found.
Ambari Server 'start' completed successfully.

如果啟動失敗建議查看
日志在/var/log/ambari-server/ambari-server.log里面
重置 ambari-server

[root@master ~]# ambari-server stop
[root@master ~]# ambari-server reset
[root@master ~]# ambari-server setup

如果選擇的是mysql方式，就需要先執行上面的語句，然后手動將mysql里面創建的數據庫進行刪除

[root@master ~]# mysql -uroot -p
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| ambari             |
| hive               |
| oozie              |
| performance_schema |
+--------------------+
rows in set (0.00 sec)
mysql> drop database ambari;
mysql> drop database hive;
mysql> drop database oozie;

如果在安裝的過程中出現了錯誤，又想重新安裝，可以在ambari-server開啟的情況下，執行下面的語句來移除已安裝的包，然后再通過不同的情況選擇上面兩種方式的一種對ambari-server進行重置

python /usr/lib/python2.6/site-packages/ambari_agent/HostCleanup.py --silent

6. 安裝配置部署HDP集群

6.1 登錄過程

如果你以上安裝成功
輸入主機ip：8080則會看到如下界面
賬戶：admin 密碼：admin

5.2 安裝向導

登錄后點擊進入流程

6.2.1 配置及群名字

我這用到的是hadoop

6.2.2 選擇版本並修改為本地源地址

放一張機翻的配置

ambari確認過了就next繼續了

6.2.3 安裝配置

上傳之前存好的秘鑰文件id_rsa

機翻頁面

6.2.4 確認安裝ambari的agent

確認安裝ambari的agent，並檢查出現的問題，這個地方經常出現錯誤

如圖就出現了錯誤，點擊Failed的查看錯誤日志

像我這里的錯誤是ambari的8040等端口無法訪問的問題，我放開了8000-9000的端口就可以了，我之前也遇到很多其他的問題，具體問題具體分析，多查谷歌，百度，國外的網站更容易解決問題

機翻

檢查主機可能會發現之前漏下的問題，比如說我這里防火牆沒關他就會出現提示

檢查無誤，NEXT→通過即可

如果這個步驟失敗了錯誤，記得多看日志，多找問題，如果還不行的話，回檔咯

[root@master ~]# # ambari-server stop    #停止命令
[root@master ~]# # ambari-server reset   #重置命令
[root@master ~]# # ambari-server setup   #重新設置 
[root@master ~]# # ambari-server start   #啟動命令