CDH | CDH6.3.1集群离线部署

一.准备工作

1. 离线部署:
   a. MySQL离线部署
   b. CM离线部署
   c. Parcel⽂文件离线源部署
2. 规划:

节点	MySQL组件	Parcel文件离线源	CM服务进程	大数据组件
cdh001			Activity Monitor	NN RM DN NM
cdh002			Alert Publisher Event Server	DN NM
cdh003	MySQL	Parcel	Host Monitor Service Monitor	DN NM

3.下载源:

二.集群节点初始化

1. 初始化虚拟机 设定ip 参考

   • cdh01 192.168.10.101
   • cdh02 192.168.10.102
   • cdh03 192.168.10.103

2. 关闭所有节点的防⽕火墙及清空规则
   systemctl stop firewalld
systemctl disable firewalld
iptables -F

3. 关闭所有节点的selinux
   vi /etc/selinux/config
将SELINUX=enforcing改为SELINUX=disabled
设置后需要重启才能⽣生效

4. 设置所有节点的时区⼀一致及时钟同步 [安装ntp服务]
   a. 时区设置

   [root@localhost ~]# date
   2021年 10月 22日 星期五 11:33:29 CST
   [root@localhost ~]# timedatectl 
         Local time: 五 2021-10-22 11:33:53 CST
     Universal time: 五 2021-10-22 03:33:53 UTC
           RTC time: 五 2021-10-22 03:33:54
          Time zone: Asia/Shanghai (CST, +0800)
        NTP enabled: yes
   NTP synchronized: yes
    RTC in local TZ: no
         DST active: n/a
   
   #查看命令帮助，学习⾄至关重要
   [root@localhost ~]# timedatectl --help
   timedatectl [OPTIONS...] COMMAND ...
   
   Query or change system time and date settings.
   
     -h --help                Show this help message
        --version             Show package version
        --no-pager            Do not pipe output into a pager
        --no-ask-password     Do not prompt for password
     -H --host=[USER@]HOST    Operate on remote host
     -M --machine=CONTAINER   Operate on local container
        --adjust-system-clock Adjust system clock when changing local RTC mode
   
   Commands:
     status                   Show current time settings
     set-time TIME            Set system time
     set-timezone ZONE        Set system time zone
     list-timezones           Show known time zones
     set-local-rtc BOOL       Control whether RTC is in local time
     set-ntp BOOL             Control whether NTP is enabled
   
   #查看哪些时区
   [root@localhost ~]# timedatectl list-timezones
   Africa/Abidjan
   Africa/Accra
   Africa/Addis_Ababa
   Africa/Algiers
   Africa/Asmara
   Africa/Bamako
   Africa/Bangui
   Africa/Banjul
   Africa/Bissau
   Africa/Blantyre
   Africa/Brazzaville
   
   #所有节点设置亚洲上海海时区
   [root@localhost ~]# timedatectl set-timezone Asia/Shanghai
   [root@localhost ~]# timedatectl set-timezone Asia/Shanghai
   [root@localhost ~]# timedatectl set-timezone Asia/Shanghai
   

   b. 时间同步

   #所有节点安装ntp
   [root@localhost ~]# yum install -y ntp
   [root@localhost ~]# vi /etc/ntp.conf 
   
   #time
   server 0.asia.pool.ntp.org
   server 1.asia.pool.ntp.org
   server 2.asia.pool.ntp.org
   server 3.asia.pool.ntp.org
   #当外部时间不可用时，可使用本地硬件时间
   server 127.127.1.0 iburst local clock
   #允许哪些网段的机器来同步时间
   restrict 172.21.230.0 mask 255.255.255.0 nomodify notrap
   
   #开启ntpd及查看状态
   [root@localhost ~]# systemctl start ntpd
   [root@localhost ~]# systemctl status ntpd
   ● ntpd.service - Network Time Service
      Loaded: loaded (/usr/lib/systemd/system/ntpd.service; disabled; vendor preset: disabled)
      Active: active (running) since 五 2021-10-22 15:07:19 CST; 9s ago
     Process: 3056 ExecStart=/usr/sbin/ntpd -u ntp:ntp $OPTIONS (code=exited, status=0/SUCCESS)
    Main PID: 3057 (ntpd)
      CGroup: /system.slice/ntpd.service
              └─3057 /usr/sbin/ntpd -u ntp:ntp -g
   
   10月 22 15:07:19 localhost.localdomain ntpd[3057]: Listen and drop on 1 v6wildcard :: UDP 123
   10月 22 15:07:19 localhost.localdomain ntpd[3057]: Listen normally on 2 lo 127.0.0.1 UDP 123
   10月 22 15:07:19 localhost.localdomain ntpd[3057]: Listen normally on 3 ens33 192.168.10.100 UDP 123
   10月 22 15:07:19 localhost.localdomain ntpd[3057]: Listen normally on 4 ens33 fe80::9d10:ea4d:53ef:2156 UDP 123
   10月 22 15:07:19 localhost.localdomain ntpd[3057]: Listen normally on 5 lo ::1 UDP 123
   10月 22 15:07:19 localhost.localdomain ntpd[3057]: Listening on routing socket on fd #22 for interface updates
   10月 22 15:07:24 localhost.localdomain ntpd[3057]: 0.0.0.0 c016 06 restart
   10月 22 15:07:24 localhost.localdomain ntpd[3057]: 0.0.0.0 c012 02 freq_set kernel 0.000 PPM
   10月 22 15:07:24 localhost.localdomain ntpd[3057]: 0.0.0.0 c011 01 freq_not_set
   10月 22 15:07:24 localhost.localdomain ntpd[3057]: 0.0.0.0 c514 04 freq_mode
   
   #验证
   [root@localhost ~] ntpq -p
      remote           refid      st t when poll reach   delay   offset  jitter
   ==============================================================================
   62.201.225.9    62.201.214.162   2 u   11   64    3  377.993   14.886   1.241
   139.199.214.202 100.122.36.196   2 u   22   64    1   45.016    2.133   0.000
   ntp.nic.kz      .SHM.            1 u   21   64    3  331.871    8.428   1.731
   time4.isu.net.s 216.218.192.202  2 u   21   64    3  428.601   49.938   0.811
   *LOCAL(0)        .LOCL.           5 l   24   64    3    0.000    0.000   0.000
   
   
   #其他从节点停⽌止禁⽤用ntpd服务
   [root@cdh002 ~]# systemctl stop ntpd
   [root@cdh002 ~]# systemctl disable ntpd
   Removed symlink /etc/systemd/system/multi-user.target.wants/ntpd.service.
   [root@cdh002 ~]# /usr/sbin/ntpdate cdh001 
   11 May 10:29:22 ntpdate[9370]: adjust time server 172.19.7.96 offset 0.000867 sec
   #每天凌晨同步ruozedata001节点时间
   [root@cdh003 ~]# crontab -e
   00 00 * * * /usr/sbin/ntpdate cdh001 
   
   [root@cdh003 ~]# systemctl stop ntpd
   [root@cdh003 ~]# systemctl disable ntpd
   Removed symlink /etc/systemd/system/multi-user.target.wants/ntpd.service.
   [root@cdh003 ~]# /usr/sbin/ntpdate ruozedata001
   11 May 10:29:22 ntpdate[9370]: adjust time server 172.19.7.96 offset 0.000867 sec
   #每天凌晨同步cdh001 节点时间
   [root@cdh003 ~]# crontab -e
   00 00 * * * /usr/sbin/ntpdate cdh001 
   

5. 每个节点部署JDK（记住部署在 /usr/java 下 不然cdh会报错找不到java）

   mkdir /usr/java
   tar -xzvf jdk-8u181-linux-x64.gz -C /usr/java/
   #切记必须修正所属⽤用户及⽤用户组
   chown -R root:root /usr/java/jdk1.8.0_181
   echo "export JAVA_HOME=/usr/java/jdk1.8.0_181" >> /etc/profile
   echo "export PATH=/usr/java/jdk1.8.0_181/bin:${PATH}" >> /etc/profile
   source /etc/profile
   which java
   

6. MySQL部署
   MySQL5.7部署
   登录mysql

7. 创建CDH的元数据库和⽤用户、amon服务的数据库及⽤用户

   create database cmf DEFAULT CHARACTER SET utf8;
   create database amon DEFAULT CHARACTER SET utf8;
   grant all on cmf.* TO 'cmf'@'%' IDENTIFIED BY '1234';
   grant all on amon.* TO 'amon'@'%' IDENTIFIED BY '1234';
   flush privileges;
   

8. 部署MySQL节点部署mysql jdbc jar
   mkdir -p /usr/share/java/
   重命名不不能带版本号
   cp mysql-connector-java-5.1.47.jar /usr/share/java/mysql-connector-java.jar

三.CDH部署

1. 离线部署cm server及agent

   1.1.所有节点创建⽬目录及解压
   mkdir /opt/cloudera-manager
   tar -xzvf cm6.3.1-redhat7.tar.gz -C /opt/cloudera-manager/
   1.2.选择cdh01为cm server，不下载依赖包直接部署
   cd /opt/cloudera-manager/cm6.3.1/RPMS/x86_64
   rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
   rpm -ivh cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
   1.3.所有节点(包含cdh01)的cm agent，不下载依赖包直接部署
   cd /opt/cloudera-manager/cm6.3.1/RPMS/x86_64
   rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
   rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
   1.4.所有节点修改agent的配置，指向server的节点 cdh01
   sed -i "s/server_host=localhost/server_host=ruozedata001/g" /etc/cloudera-scm-agen
   t/config.ini
   
   1.5.主节点修改server的配置:
   vi /etc/cloudera-scm-server/db.properties
   
   com.cloudera.cmf.db.type=mysql  #数据库类型
   com.cloudera.cmf.db.host=cdh01  #主机
   com.cloudera.cmf.db.name=cmf    #数据库名称
   com.cloudera.cmf.db.user=cmf    #数据库用户 不建议root但是可以
   com.cloudera.cmf.db.password=1234   #数据库密码
   com.cloudera.cmf.db.setupType=EXTERNAL
   

2. cdh01节点部署离线parcel源

   2.1.安装httpd服务
   yum install -y httpd
   2.2.部署离线parcel源
   $ mkdir -p /var/www/html/cdh6_parcel
   [root@cdh01 CDH6.3.1]# cp CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel /var/
   www/html/cdh6_parcel/
   [root@cdh01 CDH6.3.1]# mv CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1 /
   var/www/html/cdh6_parcel/CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha
   [root@cdh01 CDH6.3.1]# mv manifest.json /var/www/html/cdh6_parcel/
   $ ll
   total 3081664
   -rw-r--r-- 1 root root 2083878000 Oct 23 13:44 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7
   .parcel
   -rw-r--r-- 1 root root 40 Oct 23 13:44 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7
   .parcel.sha1
   -rw-r--r-- 1 root root 1411444147 Oct 23 13:43 cm6.3.1-redhat7.tar.gz
   -rw-r--r-- 1 root root 173271626 Oct 23 13:43 jdk-8u45-linux-x64.gz
   -rw-r--r-- 1 root root 33887 Oct 23 13:43 manifest.json
   -rw-r--r-- 1 root root 548193637 Oct 23 13:42 mysql-5.7.11-linux-glibc2.5-x86_64.
   tar.gz
   -rw-r--r-- 1 root root 1007502 Sep 1 12:31 mysql-connector-java-5.1.47.jar
   $ mv CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel /var/www/html/cdh6_parcel/
   $ mv CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1 /var/www/html/cdh6_parcel/CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha
   $ mv manifest.json /var/www/html/cdh6_parcel/
   sha1名称⽂文件时，重命名去掉1，不不然在部署过程CM认为如上⽂文件下载未完整，会持续下载
   2.3.启动httpd，window查看
   systemctl start httpd
   
   #检验
   window/mac: http://ruozedata001/cdh6_parcel/ 查看是否可以访问
   

3. cdh01 节点启动Server

   3.1.启动server
   systemctl start cloudera-scm-server
   查看⽇日志:
   cd /var/log/cloudera-scm-server/
   $ ll
   total 20
   -rw-r----- 1 cloudera-scm cloudera-scm 18265 Oct 23 15:37 cloudera-scm-server.log
   -rw-r----- 1 cloudera-scm cloudera-scm 0 Oct 23 15:37 cmf-server-nio.log
   -rw-r----- 1 cloudera-scm cloudera-scm 0 Oct 23 15:37 cmf-server-perf.log
   $ tail -F cloudera-scm-server.log
   有错误就根据错误解决，没有错误，等待1min，出现7180端⼝口，表明是成功的。
   
   3.2.等待1min，打开 http://cdh01:7180 账号密码:admin/admin
   3.4.假如打不不开，去看server的log，根据错误仔细排查错误
   

4. 所有节点启动Agent
   systemctl start cloudera-scm-agent
5. Web界⾯面操作
   http://ruozedata001:7180/
账号密码:admin/admin
6. 欢迎使⽤用Cloudera Manager--最终⽤用户许可条款与条件。勾选
7. 欢迎使⽤用Cloudera Manager--您想要部署哪个版本？选择Cloudera Express免费版本
8. 欢迎:在CM⾥里里添加⼀一个集群
9. 修改集群名称:（随意起名）
10. 集群搭建完成