Hadoop生態圈-基於yum源的方式部署Cloudera Manager5.15.1
作者：尹正傑
版權聲明：原創作品，謝絕轉載！否則將追究法律責任。
我之前分享過關於離線方式部署Cloudera Manager5.15.1的筆記。本篇博客給大家分享基於yum源的方式部署Cloudera Manager。由於我的台式機內存有限，本次實驗就采用三台虛擬機進行實驗。
一.集群大數據生態圈集群環境准備（溫馨提示：最好保持集群中的每台機器的硬件配置和軟件配置都一致。即每台機器最好都配置一致！）
1>.3台主機配置以及主機名與IP對應關系
[root@node101 ~]# cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core) [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# uname -r 3.10.0-327.el7.x86_64 [root@node101 ~]# [root@node101 ~]# uname -m x86_64 [root@node101 ~]# [root@node101 ~]# hostname -i 172.30.1.101 [root@node101 ~]# [root@node101 ~]# free -m total used free shared buff/cache available Mem: 7808 134 7533 8 139 7503 Swap: 2047 0 2047 [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# more /etc/hosts | grep yinzhengjie 172.30.1.101 node101.yinzhengjie.org.cn 172.30.1.102 node102.yinzhengjie.org.cn 172.30.1.103 node103.yinzhengjie.org.cn [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# hostname node101.yinzhengjie.org.cn [root@node101 ~]#
2>.配置Windows的主機名與IP地址的映射關系
Windows的主機名配置文件路徑在：“C:\Windows\System32\drivers\etc”，在該目錄下有一個hosts的配置文件，我們對其進行編輯如下：
3>.創建部署集群的管理用戶並配置sudo權限為無秘鑰登錄（溫馨提示：這個步驟可以不做，咱們可以直接使用root用戶來部署即可，我推薦使用root用戶，用普通用戶的話對某些敏感文件都需要加sudo關鍵字，盡管我這里教大家配置無需密碼驗證我覺得麻煩，因此這里只是告訴大家一個配置管理用戶的方法！而接下來的實驗我都是以root用戶來配置的！如果你非要用普通用戶的話，需要按照我下面的步驟配置即可，如果遇到權限問題，直接使用sudo提權即可！）
[root@node101 ~]# useradd yinzhengjie #創建一個普通用戶 [root@node101 ~]# [root@node101 ~]# echo "yinzhengjie" | passwd yinzhengjie --stdin #為用戶創建密碼 Changing password for user yinzhengjie. passwd: all authentication tokens updated successfully. [root@node101 ~]# [root@node101 ~]# chmod u+w /etc/sudoers #添加些的權限 [root@node101 ~]# [root@node101 ~]# vi /etc/sudoers #具體修改的內容參考下圖 [root@node101 ~]# [root@node101 ~]# gpasswd -a yinzhengjie wheel #將你的用戶(普通用戶)調整至wheel用戶組里面,這樣就可以以后用戶執行sudo就不用輸入密碼了. Adding user yinzhengjie to group wheel [root@node101 ~]# [root@node101 ~]# chmod 440 /etc/sudoers #將對文件寫的權限關閉掉 [root@node101 ~]# [root@node101 ~]# ll /etc/sudoers -r--r----- 1 root root 4255 Oct 1 07:57 /etc/sudoers [root@node101 ~]# [root@node101 ~]# id yinzhengjie #查看“yinzhengjie”用戶的id信息 uid=1000(yinzhengjie) gid=1000(yinzhengjie) groups=1000(yinzhengjie),10(wheel) [root@node101 ~]# [root@node101 ~]# su yinzhengjie #接下來，我們進行測試 [yinzhengjie@node101 root]$ [yinzhengjie@node101 root]$ sudo useradd test #執行你想要執行的任何命令吧，只需要前面加個sudo，並且不需要輸入root密碼！ [yinzhengjie@node101 root]$ sudo userdel -r test [yinzhengjie@node101 root]$
創建“yinzhengjie”用戶后，想要給這個用戶配置管理員權限，就得使用vi編輯“/etc/sudoers”配置文件，並修改以下兩處。
4>.配置ssh無秘鑰登錄
[root@node101 ~]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa Generating public/private rsa key pair. Created directory '/root/.ssh'. Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: eb:ee:5b:4b:21:d6:24:d4:2c:2d:a5:a3:b9:63:14:4f root@node101.yinzhengjie.org.cn The key's randomart image is: +--[ RSA 2048]----+ | .=. | | .o.+ | | . Eo. | | * = | | + S o | | . o o . | | + . o | | . o o . | | o=.. | +-----------------+ [root@node101 ~]#
[root@node101 ~]# ssh-copy-id root@node101.yinzhengjie.org.cn The authenticity of host 'node101.yinzhengjie.org.cn (172.30.1.101)' can't be established. ECDSA key fingerprint is b6:44:e8:e7:76:d4:c2:4c:e0:02:7e:9c:d8:59:d8:13. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@node101.yinzhengjie.org.cn's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@node101.yinzhengjie.org.cn'" and check to make sure that only the key(s) you wanted were added. [root@node101 ~]#
[root@node101 ~]# ssh-copy-id root@node102.yinzhengjie.org.cn The authenticity of host 'node102.yinzhengjie.org.cn (172.30.1.102)' can't be established. ECDSA key fingerprint is b6:44:e8:e7:76:d4:c2:4c:e0:02:7e:9c:d8:59:d8:13. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@node102.yinzhengjie.org.cn's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@node102.yinzhengjie.org.cn'" and check to make sure that only the key(s) you wanted were added. [root@node101 ~]#
[root@node101 ~]# ssh-copy-id root@node103.yinzhengjie.org.cn The authenticity of host 'node103.yinzhengjie.org.cn (172.30.1.103)' can't be established. ECDSA key fingerprint is b6:44:e8:e7:76:d4:c2:4c:e0:02:7e:9c:d8:59:d8:13. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@node103.yinzhengjie.org.cn's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@node103.yinzhengjie.org.cn'" and check to make sure that only the key(s) you wanted were added. [root@node101 ~]#
5>.編寫集群的管理腳本
[root@node101 ~]# vi /usr/local/bin/xcall.sh [root@node101 ~]# [root@node101 ~]# chmod +x /usr/local/bin/xcall.sh [root@node101 ~]# [root@node101 ~]# more `which xcall.sh` #!/bin/bash #@author :yinzhengjie #blog:http://www.cnblogs.com/yinzhengjie #EMAIL:y1053419035@qq.com #判斷用戶是否傳參 if [ $# -lt 1 ];then echo "請輸入參數" exit fi #獲取用戶輸入的命令 cmd=$@ for (( i=101;i<=103;i++ )) do #使終端變綠色 tput setaf 2 echo ============= node${i}.yinzhengjie.org.cn : $cmd ============ #使終端變回原來的顏色，即白灰色 tput setaf 7 #遠程執行命令 ssh node${i}.yinzhengjie.org.cn $cmd #判斷命令是否執行成功 if [ $? == 0 ];then echo "命令執行成功" fi done [root@node101 ~]# [root@node101 ~]# xcall.sh ls -d /home/yinzhengjie/ ============= node101.yinzhengjie.org.cn : ls -d /home/yinzhengjie/ ============ /home/yinzhengjie/ 命令執行成功 ============= node102.yinzhengjie.org.cn : ls -d /home/yinzhengjie/ ============ /home/yinzhengjie/ 命令執行成功 ============= node103.yinzhengjie.org.cn : ls -d /home/yinzhengjie/ ============ /home/yinzhengjie/ 命令執行成功 [root@node101 ~]#
[root@node101 ~]# xcall.sh "yum -y install rsync" #很顯然，我已經安裝成功啦！如果初次安裝的話不會提示你已經安裝成功喲！ ============= node101.yinzhengjie.org.cn : yum -y install rsync ============ Loaded plugins: fastestmirror Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast Determining fastest mirrors * base: mirrors.huaweicloud.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn Package rsync-3.1.2-4.el7.x86_64 already installed and latest version Nothing to do 命令執行成功 ============= node102.yinzhengjie.org.cn : yum -y install rsync ============ Loaded plugins: fastestmirror Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast Determining fastest mirrors * base: mirror.bit.edu.cn * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn Package rsync-3.1.2-4.el7.x86_64 already installed and latest version Nothing to do 命令執行成功 ============= node103.yinzhengjie.org.cn : yum -y install rsync ============ Loaded plugins: fastestmirror Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast Determining fastest mirrors * base: mirrors.huaweicloud.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn Package rsync-3.1.2-4.el7.x86_64 already installed and latest version Nothing to do 命令執行成功 [root@node101 ~]#
[root@node101 ~]# vi /usr/local/bin/xrsync.sh [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# chmod +x /usr/local/bin/xrsync.sh [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# more `which xrsync.sh` #!/bin/bash #@author :yinzhengjie #blog:http://www.cnblogs.com/yinzhengjie #EMAIL:y1053419035@qq.com #判斷用戶是否傳參 if [ $# -lt 1 ];then echo "請輸入參數"; exit fi #獲取文件路徑 file=$@ #獲取子路徑 filename=`basename $file` #獲取父路徑 dirpath=`dirname $file` #獲取完整路徑 cd $dirpath fullpath=`pwd -P` #同步文件到DataNode for (( i=102;i<=103;i++ )) do #使終端變綠色 tput setaf 2 echo =========== node${i}.yinzhengjie.org.cn : $file =========== #使終端變回原來的顏色，即白灰色 tput setaf 7 #遠程執行命令 rsync -lr $filename `whoami`@node${i}.yinzhengjie.org.cn:$fullpath #判斷命令是否執行成功 if [ $? == 0 ];then echo "命令執行成功" fi done [root@node101 ~]#
[root@node101 ~]# xrsync.sh /etc/hosts =========== node102.yinzhengjie.org.cn : /etc/hosts =========== 命令執行成功 =========== node103.yinzhengjie.org.cn : /etc/hosts =========== 命令執行成功 [root@node101 ~]#
6>.關閉集群的防火牆
[root@node101 ~]# xcall.sh systemctl stop firewalld ============= node101.yinzhengjie.org.cn : systemctl stop firewalld ============ 命令執行成功 ============= node102.yinzhengjie.org.cn : systemctl stop firewalld ============ 命令執行成功 ============= node103.yinzhengjie.org.cn : systemctl stop firewalld ============ 命令執行成功 [root@node101 ~]#
[root@node101 ~]# xcall.sh systemctl disable firewalld ============= node101.yinzhengjie.org.cn : systemctl disable firewalld ============ Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. 命令執行成功 ============= node102.yinzhengjie.org.cn : systemctl disable firewalld ============ Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. 命令執行成功 ============= node103.yinzhengjie.org.cn : systemctl disable firewalld ============ Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. 命令執行成功 [root@node101 ~]#
7>.禁用集群的selinux
[root@node101 ~]# xcall.sh setenforce 0 #很顯然，我已經禁用了selinux，如果你沒有禁用的話，selinux設置為0，也是臨時的關閉喲！ ============= node101.yinzhengjie.org.cn : setenforce 0 ============ setenforce: SELinux is disabled ============= node102.yinzhengjie.org.cn : setenforce 0 ============ setenforce: SELinux is disabled ============= node103.yinzhengjie.org.cn : setenforce 0 ============ setenforce: SELinux is disabled [root@node101 ~]#
[root@node101 ~]# xcall.sh sed -i 's#SELINUX=enforcing#SELINUX=disabled#' /etc/selinux/config ============= node101.yinzhengjie.org.cn : sed -i s#SELINUX=enforcing#SELINUX=disabled# /etc/selinux/config ============ 命令執行成功 ============= node102.yinzhengjie.org.cn : sed -i s#SELINUX=enforcing#SELINUX=disabled# /etc/selinux/config ============ 命令執行成功 ============= node103.yinzhengjie.org.cn : sed -i s#SELINUX=enforcing#SELINUX=disabled# /etc/selinux/config ============ 命令執行成功 [root@node101 ~]#
8>.禁用IPV6（每台服務器都需要禁用ipv6，我下面就舉例了一台的案例）
1>.編輯/etc/sysctl.conf配置，增加net.ipv6.conf.all.disable_ipv6=1 [root@node101 ~]# cat /etc/sysctl.conf | grep -v ^# [root@node101 ~]# [root@node101 ~]# echo "#Add by yinzhengjie" >> /etc/sysctl.conf [root@node101 ~]# [root@node101 ~]# echo "net.ipv6.conf.all.disable_ipv6=1" >> /etc/sysctl.conf You have new mail in /var/spool/mail/root [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# cat /etc/sysctl.conf | grep -v ^# net.ipv6.conf.all.disable_ipv6=1 [root@node101 ~]# [root@node101 ~]# 2>.編輯/etc/sysconfig/network配置，增加 NETWORKING_IPV6=no [root@node101 ~]# cat /etc/sysconfig/network # Created by anaconda [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# cat /etc/sysconfig/network | grep -v ^# [root@node101 ~]# [root@node101 ~]# echo "#Add by yinzhengjie" >> /etc/sysconfig/network You have new mail in /var/spool/mail/root [root@node101 ~]# [root@node101 ~]# echo "NETWORKING_IPV6=no" >> /etc/sysconfig/network [root@node101 ~]# [root@node101 ~]# cat /etc/sysconfig/network | grep -v ^# NETWORKING_IPV6=no [root@node101 ~]# [root@node101 ~]# 3>.在配置網卡信息上再次禁用ipv6 [root@node101 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0 DEVICE=bond0 BOOTPROTO=static ONBOOT=yes BONDING_OPTS="miimon=100 mode=4" IPADDR=172.30.1.101 NETMASK=255.255.255.0 GATEWAY=172.30.1.254 [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# echo NETWORKING_IPV6=no >> /etc/sysconfig/network-scripts/ifcfg-bond0 [root@node101 ~]# [root@node101 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0 （我之所以修改這個配置文件是因為我的服務器雙網卡做的事bound） DEVICE=bond0 BOOTPROTO=static ONBOOT=yes BONDING_OPTS="miimon=100 mode=4" IPADDR=172.30.1.101 NETMASK=255.255.255.0 GATEWAY=172.30.1.254 NETWORKING_IPV6=no [root@node101 ~]# 4>.執行sysctl -p或者reboot重啟命令 [root@node101 ~]# sysctl -p net.ipv6.conf.all.disable_ipv6 = 1 [root@node101 ~]#
9>.卸載jdk
[root@node102 ~]# rpm -aq | grep java #查看操作系統下載的安裝包 tzdata-java-2018e-3.el7.noarch [root@node102 ~]# [root@node102 ~]# rpm -e --nodeps tzdata-java-2018e-3.el7.noarch #使用rpm工具卸載上面安裝的軟件 [root@node102 ~]# [root@node102 ~]# rpm -aq | grep java #再次驗證，是否卸載成功啦！ [root@node102 ~]# [root@node102 ~]#
10>.設置文件打開數據和用戶最大進程數（針對每個版本Cloudera版本參數設置可能會有一些小的的偏差，我這里給的是相對來說統統的配置。）
[root@node101 ~]# ulimit -n #查看默認指定同一時間最多可開啟的文件數，這個值顯然是不合理的，我們必須得修改它！ 1024 [root@node103 ~]# [root@node101 ~]# ulimit -u #查看默認用戶最多可開啟的程序數目，這個貌似是根據你的機器配置會有相應的變化 31152 [root@node101 ~]# [root@node101 ~]# ulimit -a #顯示目前資源限制的設定 core file size (blocks, -c) 0 #core文件的最大值為100 blocks。 data seg size (kbytes, -d) unlimited #進程的數據段可以任意大。 scheduling priority (-e) 0 #指定調度優先級 file size (blocks, -f) unlimited #文件可以任意大。 pending signals (-i) 31152 #最多有31152個待處理的信號。 max locked memory (kbytes, -l) 64 #一個任務鎖住的物理內存的最大值為64KB。 max memory size (kbytes, -m) unlimited #一個任務的常駐物理內存的最大值。 open files (-n) 1024 #一個任務最多可以同時打開1024的文件。 pipe size (512 bytes, -p) 8 #管道的最大空間為4096字節。 POSIX message queues (bytes, -q) 819200 #POSIX的消息隊列的最大值為819200字節。 real-time priority (-r) 0 #指定實時優先級 stack size (kbytes, -s) 8192 #進程的棧的最大值為8192字節。 cpu time (seconds, -t) unlimited #進程使用的CPU時間。 max user processes (-u) 31152 #當前用戶同時打開的進程（包括線程）的最大個數為31152。 virtual memory (kbytes, -v) unlimited #沒有限制進程的最大地址空間。 file locks (-x) unlimited #所能鎖住的文件的最大個數沒有限制。 [root@node101 ~]# [root@node101 ~]# vi /etc/security/limits.conf #往里面配置相應的參數，我這里就做了6項優化 [root@node101 ~]# cat /etc/security/limits.conf | grep -v ^# | grep -v ^$ * soft nofile 32768 * hard nofile 1048576 * soft nproc 65536 * hard nproc unlimited * soft memlock unlimited * hard memlock unlimited [root@node101 ~]# [root@node101 ~]# reboot #按照上面的配置完成后，需要重啟操作系統 Connection closed by foreign host. Disconnected from remote host(node101.yinzhengjie.org.cn) at 12:23:03. Type `help' to learn how to use Xshell prompt. [d:\~]$ Connecting to 172.30.1.101:22... Connection established. To escape to local shell, press 'Ctrl+Alt+]'. Last login: Mon Oct 1 07:50:21 2018 from 172.30.1.1 [root@node101 ~]# [root@node101 ~]# ulimit -a #重啟操作系統后，再次查看目前資源限制的設定，來我們一起找一下不同吧 core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 31152 max locked memory (kbytes, -l) unlimited #這個值發生變化了，之前是64，現在是沒有限制啦！ max memory size (kbytes, -m) unlimited open files (-n) 32768 #這個值發生了變化，之前是1024！ pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) unlimited #這個值也發生了變化，之前是31152，現在是沒有限制 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited [root@node101 ~]#
想要了解更多ulimit的小技巧，可參考：https://www.cnblogs.com/yinzhengjie/p/9734489.html。
11>.時間同步
詳情請參考：https://www.cnblogs.com/yinzhengjie/p/9480665.html。
二.構建平台並搭建配置本地yum源
1>.Cloudera manager安裝方式
1.1>.基於bin文件方式安裝
下載地址：http://archive.cloudera.com/cm5/installer/
1.2>.基於rpm包方式安裝
下載地址：http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/
1.3>.基於yum源的方式安裝（本篇博客采取的安裝方式，因為這種方式安裝很簡單）
下載地址：http://archive.cloudera.com/cm5/repo-as-tarball/
2>.啟動Http服務（當然，你也可以選取nginx來提點http服務）
[root@node101 ~]# yum -y install httpd Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.huaweicloud.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn Resolving Dependencies --> Running transaction check ---> Package httpd.x86_64 0:2.4.6-80.el7.centos.1 will be installed --> Processing Dependency: httpd-tools = 2.4.6-80.el7.centos.1 for package: httpd-2.4.6-80.el7.centos.1.x86_64 --> Processing Dependency: /etc/mime.types for package: httpd-2.4.6-80.el7.centos.1.x86_64 --> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.4.6-80.el7.centos.1.x86_64 --> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.4.6-80.el7.centos.1.x86_64 --> Running transaction check ---> Package apr.x86_64 0:1.4.8-3.el7_4.1 will be installed ---> Package apr-util.x86_64 0:1.5.2-6.el7 will be installed ---> Package httpd-tools.x86_64 0:2.4.6-80.el7.centos.1 will be installed ---> Package mailcap.noarch 0:2.1.41-2.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================= Package Arch Version Repository Size ========================================================================================================================================================= Installing: httpd x86_64 2.4.6-80.el7.centos.1 updates 2.7 M Installing for dependencies: apr x86_64 1.4.8-3.el7_4.1 base 103 k apr-util x86_64 1.5.2-6.el7 base 92 k httpd-tools x86_64 2.4.6-80.el7.centos.1 updates 90 k mailcap noarch 2.1.41-2.el7 base 31 k Transaction Summary ========================================================================================================================================================= Install 1 Package (+4 Dependent packages) Total download size: 3.0 M Installed size: 10 M Downloading packages: (1/5): apr-1.4.8-3.el7_4.1.x86_64.rpm | 103 kB 00:00:00 (2/5): httpd-tools-2.4.6-80.el7.centos.1.x86_64.rpm | 90 kB 00:00:00 (3/5): apr-util-1.5.2-6.el7.x86_64.rpm | 92 kB 00:00:00 (4/5): mailcap-2.1.41-2.el7.noarch.rpm | 31 kB 00:00:00 (5/5): httpd-2.4.6-80.el7.centos.1.x86_64.rpm | 2.7 MB 00:00:00 --------------------------------------------------------------------------------------------------------------------------------------------------------- Total 6.5 MB/s | 3.0 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : apr-1.4.8-3.el7_4.1.x86_64 1/5 Installing : apr-util-1.5.2-6.el7.x86_64 2/5 Installing : httpd-tools-2.4.6-80.el7.centos.1.x86_64 3/5 Installing : mailcap-2.1.41-2.el7.noarch 4/5 Installing : httpd-2.4.6-80.el7.centos.1.x86_64 5/5 Verifying : mailcap-2.1.41-2.el7.noarch 1/5 Verifying : httpd-tools-2.4.6-80.el7.centos.1.x86_64 2/5 Verifying : apr-util-1.5.2-6.el7.x86_64 3/5 Verifying : apr-1.4.8-3.el7_4.1.x86_64 4/5 Verifying : httpd-2.4.6-80.el7.centos.1.x86_64 5/5 Installed: httpd.x86_64 0:2.4.6-80.el7.centos.1 Dependency Installed: apr.x86_64 0:1.4.8-3.el7_4.1 apr-util.x86_64 0:1.5.2-6.el7 httpd-tools.x86_64 0:2.4.6-80.el7.centos.1 mailcap.noarch 0:2.1.41-2.el7 Complete! [root@node101 ~]#
[root@node101 ~]# systemctl start httpd #啟動Apache服務 [root@node101 ~]# [root@node101 ~]# systemctl enable httpd #設置Apache的開啟自啟 Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service. [root@node101 ~]#
啟動成功后，我們可以訪問webUI來查看是否真實啟動，如下圖：
3>.下載CM資源文件並安裝到Apache的webUI路徑中
[root@node101 ~]# mkdir -pv /yinzhengjie/download/ && cd /yinzhengjie/download mkdir: created directory ‘/yinzhengjie’ mkdir: created directory ‘/yinzhengjie/download/’ [root@node101 download]# [root@node101 download]# yum -y install wget #安裝下載工具wget，如果你已經裝了這一步驟可以省略 [root@node101 download]# [root@node101 download]# wget http://archive.cloudera.com/cm5/repo-as-tarball/5.15.1/cm5.15.1-centos7.tar.gz #下載cm的安裝包 --2018-10-01 15:01:49-- http://archive.cloudera.com/cm5/repo-as-tarball/5.15.1/cm5.15.1-centos7.tar.gz Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.108.167 Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.108.167|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 1038457027 (990M) [application/x-tar] Saving to: ‘cm5.15.1-centos7.tar.gz’ 100%[=============================================================================================================>] 1,038,457,027 902KB/s in 30m 30s 2018-10-01 15:12:19 (554 KB/s) - ‘cm5.15.1-centos7.tar.gz’ saved [1038457027/1038457027] [root@node101 download]# ll total 1014120 -rw-r--r-- 1 root root 1038457027 Aug 22 22:51 cm5.15.1-centos7.tar.gz [root@node101 download]# [root@node101 download]# mkdir -pv /yinzhengjie/softwares mkdir: created directory ‘/yinzhengjie/softwares’ [root@node101 download]# [root@node101 download]# tar -zxf cm5.15.1-centos7.tar.gz -C /yinzhengjie/softwares/ #將下載后的文件進行解壓 [root@node101 download]# [root@node101 download]# cd /yinzhengjie/softwares/cm/5.15.1/RPMS/x86_64/ #進入到解壓后的CM的rpm工具包中 [root@node101 x86_64]# ll total 1016664 -rw-rw-r-- 1 1106 592 9857532 Jul 31 18:05 cloudera-manager-agent-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 787321332 Jul 31 18:05 cloudera-manager-daemons-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 8712 Jul 31 18:05 cloudera-manager-server-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 10616 Jul 31 18:05 cloudera-manager-server-db-2-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 30603956 Jul 31 18:05 enterprise-debuginfo-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 1066 Jul 31 18:12 generated_index.html -rw-rw-r-- 1 1106 592 71204325 Jul 31 18:05 jdk-6u31-linux-amd64.rpm -rw-rw-r-- 1 1106 592 142039186 Jul 31 18:05 oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm [root@node101 x86_64]# [root@node101 x86_64]# mv ./* /var/www/html/cm-5.15.1/ #將資源安裝到Apache的webUI目錄中 [root@node101 x86_64]# [root@node101 x86_64]# cd /var/www/html/cm-5.15.1/ [root@node101 cm-5.15.1]# [root@node101 cm-5.15.1]# ll total 1016664 -rw-rw-r-- 1 1106 592 9857532 Jul 31 18:05 cloudera-manager-agent-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 787321332 Jul 31 18:05 cloudera-manager-daemons-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 8712 Jul 31 18:05 cloudera-manager-server-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 10616 Jul 31 18:05 cloudera-manager-server-db-2-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 30603956 Jul 31 18:05 enterprise-debuginfo-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 1066 Jul 31 18:12 generated_index.html -rw-rw-r-- 1 1106 592 71204325 Jul 31 18:05 jdk-6u31-linux-amd64.rpm -rw-rw-r-- 1 1106 592 142039186 Jul 31 18:05 oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm [root@node101 cm-5.15.1]#
配置成功后，我們可以在wen目錄上看到相應的信息。
4>.創建repodata數據文件
[root@node101 cm-5.15.1]# yum -y install createrepo Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.huaweicloud.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn Resolving Dependencies --> Running transaction check ---> Package createrepo.noarch 0:0.9.9-28.el7 will be installed --> Processing Dependency: python-deltarpm for package: createrepo-0.9.9-28.el7.noarch --> Processing Dependency: libxml2-python for package: createrepo-0.9.9-28.el7.noarch --> Processing Dependency: deltarpm for package: createrepo-0.9.9-28.el7.noarch --> Running transaction check ---> Package deltarpm.x86_64 0:3.6-3.el7 will be installed ---> Package libxml2-python.x86_64 0:2.9.1-6.el7_2.3 will be installed ---> Package python-deltarpm.x86_64 0:3.6-3.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================= Package Arch Version Repository Size ========================================================================================================================================================= Installing: createrepo noarch 0.9.9-28.el7 base 94 k Installing for dependencies: deltarpm x86_64 3.6-3.el7 base 82 k libxml2-python x86_64 2.9.1-6.el7_2.3 base 247 k python-deltarpm x86_64 3.6-3.el7 base 31 k Transaction Summary ========================================================================================================================================================= Install 1 Package (+3 Dependent packages) Total download size: 454 k Installed size: 2.0 M Downloading packages: (1/4): deltarpm-3.6-3.el7.x86_64.rpm | 82 kB 00:00:00 (2/4): python-deltarpm-3.6-3.el7.x86_64.rpm | 31 kB 00:00:00 (3/4): createrepo-0.9.9-28.el7.noarch.rpm | 94 kB 00:00:00 (4/4): libxml2-python-2.9.1-6.el7_2.3.x86_64.rpm | 247 kB 00:00:09 --------------------------------------------------------------------------------------------------------------------------------------------------------- Total 49 kB/s | 454 kB 00:00:09 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : deltarpm-3.6-3.el7.x86_64 1/4 Installing : python-deltarpm-3.6-3.el7.x86_64 2/4 Installing : libxml2-python-2.9.1-6.el7_2.3.x86_64 3/4 Installing : createrepo-0.9.9-28.el7.noarch 4/4 Verifying : createrepo-0.9.9-28.el7.noarch 1/4 Verifying : libxml2-python-2.9.1-6.el7_2.3.x86_64 2/4 Verifying : deltarpm-3.6-3.el7.x86_64 3/4 Verifying : python-deltarpm-3.6-3.el7.x86_64 4/4 Installed: createrepo.noarch 0:0.9.9-28.el7 Dependency Installed: deltarpm.x86_64 0:3.6-3.el7 libxml2-python.x86_64 0:2.9.1-6.el7_2.3 python-deltarpm.x86_64 0:3.6-3.el7 Complete! [root@node101 cm-5.15.1]#
[root@node101 cm-5.15.1]# pwd /var/www/html/cm-5.15.1 [root@node101 cm-5.15.1]# ll total 1016664 -rw-rw-r-- 1 1106 592 9857532 Jul 31 18:05 cloudera-manager-agent-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 787321332 Jul 31 18:05 cloudera-manager-daemons-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 8712 Jul 31 18:05 cloudera-manager-server-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 10616 Jul 31 18:05 cloudera-manager-server-db-2-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 30603956 Jul 31 18:05 enterprise-debuginfo-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 1066 Jul 31 18:12 generated_index.html -rw-rw-r-- 1 1106 592 71204325 Jul 31 18:05 jdk-6u31-linux-amd64.rpm -rw-rw-r-- 1 1106 592 142039186 Jul 31 18:05 oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm [root@node101 cm-5.15.1]# [root@node101 cm-5.15.1]# createrepo . #生成數據目錄，我們看在webUI中查看相應的數據目錄 Spawning worker 0 with 4 pkgs Spawning worker 1 with 3 pkgs Workers Finished Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete [root@node101 cm-5.15.1]# [root@node101 cm-5.15.1]# ll total 1016668 -rw-rw-r-- 1 1106 592 9857532 Jul 31 18:05 cloudera-manager-agent-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 787321332 Jul 31 18:05 cloudera-manager-daemons-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 8712 Jul 31 18:05 cloudera-manager-server-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 10616 Jul 31 18:05 cloudera-manager-server-db-2-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 30603956 Jul 31 18:05 enterprise-debuginfo-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm -rw-rw-r-- 1 1106 592 1066 Jul 31 18:12 generated_index.html -rw-rw-r-- 1 1106 592 71204325 Jul 31 18:05 jdk-6u31-linux-amd64.rpm -rw-rw-r-- 1 1106 592 142039186 Jul 31 18:05 oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm drwxr-xr-x 2 root root 4096 Oct 1 15:29 repodata #溫馨提示：注意看這里，發現沒有？這個目錄是新生成的！ [root@node101 cm-5.15.1]#
再次查看webUI是否有repodata目錄生成：
5>.配置集群各機器本地yum源
[root@node101 ~]# cd /etc/yum.repos.d/ [root@node101 yum.repos.d]# [root@node101 yum.repos.d]# ll #下面是默認的repo源 total 28 -rw-r--r--. 1 root root 1664 Dec 9 2015 CentOS-Base.repo -rw-r--r--. 1 root root 1309 Dec 9 2015 CentOS-CR.repo -rw-r--r--. 1 root root 649 Dec 9 2015 CentOS-Debuginfo.repo -rw-r--r--. 1 root root 290 Dec 9 2015 CentOS-fasttrack.repo -rw-r--r--. 1 root root 630 Dec 9 2015 CentOS-Media.repo -rw-r--r--. 1 root root 1331 Dec 9 2015 CentOS-Sources.repo -rw-r--r--. 1 root root 1952 Dec 9 2015 CentOS-Vault.repo [root@node101 yum.repos.d]# [root@node101 yum.repos.d]# mkdir repo-backup [root@node101 yum.repos.d]# mv ./*.repo repo-backup/ #備份默認的repo源，為了防止你操作失敗可以還原之前的狀態！ [root@node101 yum.repos.d]# ll total 4 drwxr-xr-x 2 root root 4096 Oct 1 15:50 repo-backup [root@node101 yum.repos.d]# [root@node101 yum.repos.d]# vi yinzhengjie-cm.repo [root@node101 yum.repos.d]# ll total 8 drwxr-xr-x 2 root root 4096 Oct 1 15:50 repo-backup -rw-r--r-- 1 root root 204 Oct 1 15:58 yinzhengjie-cm.repo [root@node101 yum.repos.d]# [root@node101 yum.repos.d]# more yinzhengjie-cm.repo #這個配置參考自：可參考：http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/cloudera-manager.repo [yinzhengjie-cm] # Packages for Cloudera Manager, Version 5, on RedHat or CentOS 7 x86_64 name=Cloudera Manager baseurl=https://node101.yinzhengjie.org.cn/cm-5.15.1 enabled = 1 gpgcheck = 0 [root@node101 yum.repos.d]# [root@node101 yum.repos.d]# xrsync.sh /etc/yum.repos.d/yinzhengjie-cm.repo #將配置同步到其它兩個節點中 =========== node102.yinzhengjie.org.cn : /etc/yum.repos.d/yinzhengjie-cm.repo =========== 命令執行成功 =========== node103.yinzhengjie.org.cn : /etc/yum.repos.d/yinzhengjie-cm.repo =========== 命令執行成功 [root@node101 yum.repos.d]#
三.Cloudera Manager服務安裝詳解
1>.Cloudera Manager服務架構設計
1.1>.Agent 1 .. .Agent n：
所有的Agent相當於集群。Agent部署在每一台服務器上，Server的控制命令最終由Agent來完成，而Agent通過Python調用shell腳本來控制具體的啟動或者停止的腳本執行。
1.2>.Server
相當於Cloudera Manager的主節點，它可以管理整個集群。它主要負責以下幾件事情：
第一：處理軟件和服務器的邏輯關系；
第二：處理服務的安裝配置，啟動，停止等；
第三：控制對應的角色的啟動和停止；
第四：管理集群（每一台Agent）
1.3>.Database
需要關系型數據庫存儲數據。
1.4>.Management Server
對整個集群運行的硬件和軟件做一些預警，監控，報告等。當你的圖標沒有出來的時候，你就得考慮它是否正常啟動了。
1.5>.Cloudera Repository
Cloudera官網的支持，可以下載軟件包。
1.6>.Clients
通過webUI或者調用對應的API來控制Server。如果你是通過webUI界面點擊了某個操作，比如讓某個集群停止或啟動，首先這個指令會發送給Server端，然后Server端根據自己的記錄去找相應的Agent執行相應的腳本來停止或者啟動應用。默認Server端會想Agent發送五次指令，如果這五次指令Agent都沒有相應，則Server端判定Agent端存在故障。
2>.安裝PostgreSQL數據庫
PostgreSQL是Cloudera Manager默認使用的數據庫。它負責存儲信息內容包括：
2.1>.保存平台所有的配置信息（比如對各個組件的參數配置等等），便於各個節點獲取數據庫的配置信息；
2.2>.所有節點的監控狀態（各個節點將自己的狀態信息存儲到數據庫中，以便主服務從數據庫中獲取數據信息，從而達到監控的目的）；
數據庫主要有以上兩個功能，具體安裝步驟如下：
[root@node101 packages]# yum -y install postgresql Loaded plugins: fastestmirror base | 3.6 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 Loading mirror speeds from cached hostfile * base: mirrors.tuna.tsinghua.edu.cn * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn Resolving Dependencies --> Running transaction check ---> Package postgresql.x86_64 0:9.2.24-1.el7_5 will be installed --> Processing Dependency: postgresql-libs(x86-64) = 9.2.24-1.el7_5 for package: postgresql-9.2.24-1.el7_5.x86_64 --> Processing Dependency: libpq.so.5()(64bit) for package: postgresql-9.2.24-1.el7_5.x86_64 --> Running transaction check ---> Package postgresql-libs.x86_64 0:9.2.24-1.el7_5 will be installed --> Finished Dependency Resolution Dependencies Resolved ====================================================================================================================================================================================================================== Package Arch Version Repository Size ====================================================================================================================================================================================================================== Installing: postgresql x86_64 9.2.24-1.el7_5 updates 3.0 M Installing for dependencies: postgresql-libs x86_64 9.2.24-1.el7_5 updates 234 k Transaction Summary ====================================================================================================================================================================================================================== Install 1 Package (+1 Dependent package) Total download size: 3.3 M Installed size: 17 M Downloading packages: (1/2): postgresql-libs-9.2.24-1.el7_5.x86_64.rpm | 234 kB 00:00:00 (2/2): postgresql-9.2.24-1.el7_5.x86_64.rpm | 3.0 MB 00:00:05 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 631 kB/s | 3.3 MB 00:00:05 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : postgresql-libs-9.2.24-1.el7_5.x86_64 1/2 Installing : postgresql-9.2.24-1.el7_5.x86_64 2/2 Verifying : postgresql-libs-9.2.24-1.el7_5.x86_64 1/2 Verifying : postgresql-9.2.24-1.el7_5.x86_64 2/2 Installed: postgresql.x86_64 0:9.2.24-1.el7_5 Dependency Installed: postgresql-libs.x86_64 0:9.2.24-1.el7_5 Complete! [root@node101 packages]#
3>.安裝Cloudera manager Server
[root@node101 ~]# cd /yinzhengjie/download/ [root@node101 download]# wget http://archive.cloudera.com/cm5/installer/5.15.1/cloudera-manager-installer.bin --2018-10-01 16:44:49-- http://archive.cloudera.com/cm5/installer/5.15.1/cloudera-manager-installer.bin Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.72.167 Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.72.167|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 521199 (509K) [application/octet-stream] Saving to: ‘cloudera-manager-installer.bin’ 100%[============================================================================================================================================================================>] 521,199 599KB/s in 0.8s 2018-10-01 16:44:50 (599 KB/s) - ‘cloudera-manager-installer.bin’ saved [521199/521199] [root@node101 download]# ll total 1014632 -rw-r--r-- 1 root root 521199 Aug 22 22:50 cloudera-manager-installer.bin -rw-r--r-- 1 root root 1038457027 Aug 22 22:51 cm5.15.1-centos7.tar.gz [root@node101 download]#
[root@node101 download]# pwd /yinzhengjie/download [root@node101 download]# [root@node101 download]# ll total 1014632 -rwxr-xr-x 1 root root 521199 Aug 22 22:50 cloudera-manager-installer.bin -rw-r--r-- 1 root root 1038457027 Aug 22 22:51 cm5.15.1-centos7.tar.gz [root@node101 download]# [root@node101 download]# chmod +x cloudera-manager-installer.bin [root@node101 download]# [root@node101 download]# ./cloudera-manager-installer.bin --skip_repo_package=1 #注意，后面的參數必須要寫，目的是為了防止cm去官網下載repo，我們之前的操作依據將repo下載到本地了，這里跳過驗證即可 [root@node101 download]#
關於安裝的詳細步驟請參考：https://www.cnblogs.com/yinzhengjie/p/9735095.html。安裝成功后，可以訪問webUI界面，如下圖：
[root@node101 ~]# systemctl cloudera-scm-server status Unknown operation 'cloudera-scm-server'. [root@node101 ~]# [root@node101 ~]# systemctl status cloudera-scm-server ● cloudera-scm-server.service - LSB: Cloudera SCM Server Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled) Active: active (exited) since Mon 2018-10-01 17:34:01 CST; 45min ago Docs: man:systemd-sysv-generator(8) Process: 4088 ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=0/SUCCESS) Oct 01 17:33:56 node101.yinzhengjie.org.cn systemd[1]: Starting LSB: Cloudera SCM Server... Oct 01 17:33:56 node101.yinzhengjie.org.cn cloudera-scm-server[4088]: /etc/rc.d/init.d/cloudera-scm-server: line 109: pstree: command not found Oct 01 17:33:56 node101.yinzhengjie.org.cn su[4110]: (to cloudera-scm) root on none Oct 01 17:34:01 node101.yinzhengjie.org.cn cloudera-scm-server[4088]: Starting cloudera-scm-server: [ OK ] Oct 01 17:34:01 node101.yinzhengjie.org.cn systemd[1]: Started LSB: Cloudera SCM Server. Hint: Some lines were ellipsized, use -l to show in full. [root@node101 ~]#
[root@node101 ~]# systemctl cloudera-scm-server status Unknown operation 'cloudera-scm-server'. [root@node101 ~]# [root@node101 ~]# systemctl status cloudera-scm-server ● cloudera-scm-server.service - LSB: Cloudera SCM Server Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled) Active: active (exited) since Mon 2018-10-01 17:34:01 CST; 45min ago Docs: man:systemd-sysv-generator(8) Process: 4088 ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=0/SUCCESS) Oct 01 17:33:56 node101.yinzhengjie.org.cn systemd[1]: Starting LSB: Cloudera SCM Server... Oct 01 17:33:56 node101.yinzhengjie.org.cn cloudera-scm-server[4088]: /etc/rc.d/init.d/cloudera-scm-server: line 109: pstree: command not found Oct 01 17:33:56 node101.yinzhengjie.org.cn su[4110]: (to cloudera-scm) root on none Oct 01 17:34:01 node101.yinzhengjie.org.cn cloudera-scm-server[4088]: Starting cloudera-scm-server: [ OK ] Oct 01 17:34:01 node101.yinzhengjie.org.cn systemd[1]: Started LSB: Cloudera SCM Server. Hint: Some lines were ellipsized, use -l to show in full. [root@node101 ~]# [root@node101 ~]# systemctl stop cloudera-scm-server [root@node101 ~]# [root@node101 ~]# systemctl status cloudera-scm-server ● cloudera-scm-server.service - LSB: Cloudera SCM Server Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled) Active: inactive (dead) since Mon 2018-10-01 18:21:18 CST; 2s ago Docs: man:systemd-sysv-generator(8) Process: 5894 ExecStop=/etc/rc.d/init.d/cloudera-scm-server stop (code=exited, status=0/SUCCESS) Process: 5792 ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=0/SUCCESS) Oct 01 18:19:58 node101.yinzhengjie.org.cn systemd[1]: Starting LSB: Cloudera SCM Server... Oct 01 18:19:58 node101.yinzhengjie.org.cn cloudera-scm-server[5792]: /etc/rc.d/init.d/cloudera-scm-server: line 109: pstree: command not found Oct 01 18:19:58 node101.yinzhengjie.org.cn su[5814]: (to cloudera-scm) root on none Oct 01 18:20:03 node101.yinzhengjie.org.cn systemd[1]: Started LSB: Cloudera SCM Server. Oct 01 18:20:03 node101.yinzhengjie.org.cn cloudera-scm-server[5792]: Starting cloudera-scm-server: [ OK ] Oct 01 18:21:15 node101.yinzhengjie.org.cn systemd[1]: Stopping LSB: Cloudera SCM Server... Oct 01 18:21:16 node101.yinzhengjie.org.cn cloudera-scm-server[5894]: /etc/rc.d/init.d/cloudera-scm-server: line 109: pstree: command not found Oct 01 18:21:18 node101.yinzhengjie.org.cn cloudera-scm-server[5894]: Stopping cloudera-scm-server: [ OK ] Oct 01 18:21:18 node101.yinzhengjie.org.cn systemd[1]: Stopped LSB: Cloudera SCM Server. Hint: Some lines were ellipsized, use -l to show in full. [root@node101 ~]#
[root@node101 ~]# systemctl status cloudera-scm-server ● cloudera-scm-server.service - LSB: Cloudera SCM Server Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled) Active: inactive (dead) since Mon 2018-10-01 18:21:18 CST; 2s ago Docs: man:systemd-sysv-generator(8) Process: 5894 ExecStop=/etc/rc.d/init.d/cloudera-scm-server stop (code=exited, status=0/SUCCESS) Process: 5792 ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=0/SUCCESS) Oct 01 18:19:58 node101.yinzhengjie.org.cn systemd[1]: Starting LSB: Cloudera SCM Server... Oct 01 18:19:58 node101.yinzhengjie.org.cn cloudera-scm-server[5792]: /etc/rc.d/init.d/cloudera-scm-server: line 109: pstree: command not found Oct 01 18:19:58 node101.yinzhengjie.org.cn su[5814]: (to cloudera-scm) root on none Oct 01 18:20:03 node101.yinzhengjie.org.cn systemd[1]: Started LSB: Cloudera SCM Server. Oct 01 18:20:03 node101.yinzhengjie.org.cn cloudera-scm-server[5792]: Starting cloudera-scm-server: [ OK ] Oct 01 18:21:15 node101.yinzhengjie.org.cn systemd[1]: Stopping LSB: Cloudera SCM Server... Oct 01 18:21:16 node101.yinzhengjie.org.cn cloudera-scm-server[5894]: /etc/rc.d/init.d/cloudera-scm-server: line 109: pstree: command not found Oct 01 18:21:18 node101.yinzhengjie.org.cn cloudera-scm-server[5894]: Stopping cloudera-scm-server: [ OK ] Oct 01 18:21:18 node101.yinzhengjie.org.cn systemd[1]: Stopped LSB: Cloudera SCM Server. Hint: Some lines were ellipsized, use -l to show in full. [root@node101 ~]# [root@node101 ~]# systemctl start cloudera-scm-server [root@node101 ~]# [root@node101 ~]# systemctl status cloudera-scm-server ● cloudera-scm-server.service - LSB: Cloudera SCM Server Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled) Active: active (exited) since Mon 2018-10-01 18:21:33 CST; 53s ago Docs: man:systemd-sysv-generator(8) Process: 5894 ExecStop=/etc/rc.d/init.d/cloudera-scm-server stop (code=exited, status=0/SUCCESS) Process: 5939 ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=0/SUCCESS) Oct 01 18:21:28 node101.yinzhengjie.org.cn systemd[1]: Starting LSB: Cloudera SCM Server... Oct 01 18:21:28 node101.yinzhengjie.org.cn cloudera-scm-server[5939]: /etc/rc.d/init.d/cloudera-scm-server: line 109: pstree: command not found Oct 01 18:21:28 node101.yinzhengjie.org.cn su[5961]: (to cloudera-scm) root on none Oct 01 18:21:33 node101.yinzhengjie.org.cn cloudera-scm-server[5939]: Starting cloudera-scm-server: [ OK ] Oct 01 18:21:33 node101.yinzhengjie.org.cn systemd[1]: Started LSB: Cloudera SCM Server. Hint: Some lines were ellipsized, use -l to show in full. [root@node101 ~]#
4>.下載parcels包並配置到CM主服務器目錄下（“/opt/cloudera/parcel-repo/”）
[root@node101 ~]# mkdir -pv /yinzhengjie/softwares/parcels mkdir: created directory ‘/yinzhengjie/softwares/parcels’ [root@node101 ~]# [root@node101 ~]# cd /yinzhengjie/softwares/parcels/ [root@node101 parcels]# [root@node101 parcels]# ll total 0 [root@node101 parcels]# [root@node101 parcels]# wget http://archive.cloudera.com/cdh5/parcels/5.15.1/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 #下載CDH校驗包 --2018-10-01 18:26:21-- http://archive.cloudera.com/cdh5/parcels/5.15.1/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.108.167 Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.108.167|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 41 [binary/octet-stream] Saving to: ‘CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1’ 100%[=====================================================================================================>] 41 --.-K/s in 0s 2018-10-01 18:26:22 (2.72 MB/s) - ‘CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1’ saved [41/41] [root@node101 parcels]#
[root@node101 parcels]# [root@node101 parcels]# wget http://archive.cloudera.com/cdh5/parcels/5.15.1/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel #下載CDH安裝包 --2018-10-01 18:26:42-- http://archive.cloudera.com/cdh5/parcels/5.15.1/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.228.167 Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.228.167|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2120090032 (2.0G) [binary/octet-stream] Saving to: ‘CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel’ 100%[=====================================================================================================>] 41 9.17 MB/s in 0s 2018-10-01 18:43:01 (3.07 MB/s) - ‘CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel’ saved [41/41] [root@node101 parcels]# [root@node101 parcels]# ll total 2070408 -rw-r--r-- 1 root root 2120090032 Aug 23 03:54 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel -rw-r--r-- 1 root root 41 Sep 13 01:44 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 [root@node101 parcels]#
[root@node101 parcels]# ll total 2070408 -rw-r--r-- 1 root root 2120090032 Aug 23 03:54 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel -rw-r--r-- 1 root root 41 Sep 13 01:44 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 [root@node101 parcels]# [root@node101 parcels]# mv CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha [root@node101 parcels]# [root@node101 parcels]# ll total 2070408 -rw-r--r-- 1 root root 2120090032 Aug 23 03:54 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel -rw-r--r-- 1 root root 41 Sep 13 01:44 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha [root@node101 parcels]#
[root@node101 parcels]# pwd /yinzhengjie/softwares/parcels [root@node101 parcels]# ll total 2070408 -rw-r--r-- 1 root root 2120090032 Aug 23 03:54 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel -rw-r--r-- 1 root root 41 Sep 13 01:44 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha [root@node101 parcels]# [root@node101 parcels]# mv * /opt/cloudera/parcel-repo/ #將資源拷貝到parcel-repo目錄下 [root@node101 parcels]# [root@node101 parcels]# ll total 0 [root@node101 parcels]# [root@node101 parcels]# [root@node101 parcels]# ll /opt/cloudera/parcel-repo/ total 2070408 -rw-r--r-- 1 root root 2120090032 Aug 23 03:54 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel -rw-r--r-- 1 root root 41 Sep 13 01:44 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha [root@node101 parcels]#
5>.檢查本地源並安裝Cloudera manager agent端
詳情強參考：https://www.cnblogs.com/yinzhengjie/p/9735451.html
6>.