一、需求
系統
CentOS 7 最小化安裝
JDK環境
JDK版本：1.8.0_91 jdk-8u91-linux-x64.rpm
下載地址：http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
CM包
CM版本：5.7.0 cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz
下載地址：http://archive.cloudera.com/cm5/cm/5/cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz
CDH包
CDH版本：5.7.0
CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 manifest.json
下載地址
http://archive.cloudera.com/cdh5/parcels/5.7/CDH-5.7.1-1.cdh5.7.1.p0.11-el7.parcel
http://archive.cloudera.com/cdh5/parcels/5.7/CDH-5.7.1-1.cdh5.7.1.p0.11-el7.parcel.sha1
http://archive.cloudera.com/cdh5/parcels/5.7/manifest.json
java-mysql連接jar包
下載地址：http://central.maven.org/maven2/mysql/mysql-connector-java/6.0.2/mysql-connector-java-6.0.2.jar
數據庫
類型：MariaDB
版本：5.5.47-MariaDB MariaDB Server
節點
| 主機名 |
IP地址 |
說明 |
| bigdata01.tfpay.com |
172.17.28.1 |
master datanode namenode |
| bigdata02.tfpay.com |
172.17.28.2 |
datanode |
| bigdata03.tfpay.com |
172.17.28.3 |
datanode |
| bigdata04.tfpay.com |
172.17.28.4 |
datanode |
節點的數量可以根據自己的需求變更，也可以在CDH安裝完成后，在管理界面自己拓展添加
二、底層環境准備
注：准備安裝前需要把相關的包文件下載下來，並上傳到服務器上
1、安裝JDK
所有節點
1）安裝系統原有java
# yum -y install java # java -version openjdk version "1.8.0_91" OpenJDK Runtime Environment (build 1.8.0_91-b14) OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode) # rpm -qa|grep java javapackages-tools-3.4.1-11.el7.noarch python-javapackages-3.4.1-11.el7.noarch java-1.8.0-openjdk-headless-1.8.0.91-0.b14.el7_2.x86_64 java-1.8.0-openjdk-1.8.0.91-0.b14.el7_2.x86_64 tzdata-java-2016d-1.el7.noarch
2）卸載OpenJDK
# rpm -e --nodeps java-1.8.0-openjdk-headless-1.8.0.91-0.b14.el7_2.x86_64
# rpm -e --nodeps java-1.8.0-openjdk-1.8.0.91-0.b14.el7_2.x86_64
3）安裝下載的java
# rpm -e --nodeps java-1.8.0-openjdk-headless-1.8.0.91-0.b14.el7_2.x86_64 # rpm -e --nodeps java-1.8.0-openjdk-1.8.0.91-0.b14.el7_2.x86_64
# rpm -ivh jdk-8u91-linux-x64.rpm Preparing... ################################# [100%] Updating / installing... 1:jdk1.8.0_91-2000:1.8.0_91-fcs ################################# [100%] Unpacking JAR files... tools.jar... plugin.jar... javaws.jar... deploy.jar... rt.jar... jsse.jar... charsets.jar... localedata.jar... jfxrt.jar... # java -version java version "1.8.0_91" Java(TM) SE Runtime Environment (build 1.8.0_91-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode) # javac -version javac 1.8.0_91
2、更改主機名和hosts
所有節點
注意每個主機名對應的IP，按實際情況更改。
# echo "bigdata0*.tfpay.com" > /etc/hostname # echo "172.17.28.1 bigdata01.tfpay.com bigdata01" >> /etc/hosts # echo "172.17.28.2 bigdata02.tfpay.com bigdata02" >> /etc/hosts # echo "172.17.28.3 bigdata03.tfpay.com bigdata03" >> /etc/hosts # echo "172.17.28.4 bigdata04.tfpay.com bigdata04" >> /etc/hosts
3、關閉防火牆
所有節點
# systemctl stop firewalld # systemctl disable firewalld Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service. # systemctl status firewalld ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled) Active: inactive (dead) Jun 20 16:05:43 bigdata05.tfpay.com systemd[1]: Starting firewalld - dynamic firewall daemon... Jun 20 16:05:45 bigdata05.tfpay.com systemd[1]: Started firewalld - dynamic firewall daemon. Jun 20 16:17:51 bigdata05.tfpay.com systemd[1]: Stopping firewalld - dynamic firewall daemon... Jun 20 16:17:52 bigdata05.tfpay.com systemd[1]: Stopped firewalld - dynamic firewall daemon. # sed -i "s/SELINUX=enforcing/SELINUX=disabled/" /etc/selinux/config # reboot
重啟服務器使hostname和selinux的配置生效
# getenforce Disabled # sestatus -v SELinux status: disabled
4、安裝NTP服務
所有節點
# yum -y install ntp
更改配置文件/etc/ntp.conf
在master節點namenode節點
注釋掉原有的server指向把時間服務器的域名地址指向一個靠譜可連的時間服務器，我選擇的是ntp.api.bz.
# vi /etc/ntp.conf …… #server 0.centos.pool.ntp.org iburst #server 1.centos.pool.ntp.org iburst #server 2.centos.pool.ntp.org iburst #server 3.centos.pool.ntp.org iburst server ntp.api.gz iburst ......
在其余的節點上，把時間服務器的server指向master namenode節點
# vi /etc/ntp.conf …… #server 0.centos.pool.ntp.org iburst #server 1.centos.pool.ntp.org iburst #server 2.centos.pool.ntp.org iburst #server 3.centos.pool.ntp.org iburst server 172.17.28.1 iburst ......
所有節點
# systemctl start ntpd
# systemctl enable ntpd
Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
5、SSH無密碼登陸
# systemctl start ntpd
# systemctl enable ntpd
Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
配置master namenode節點無密碼登陸到其他節點
在master namenode節點
# ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): 回車 Enter passphrase (empty for no passphrase): 回車 Enter same passphrase again: 回車 Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: bc:14:e2:59:42:1f:21:80:eb:49:d8:3b:7c:8d:b7:f9 root@bigdata05.tfpay.com The key's randomart image is: +--[ RSA 2048]----+ | ...o o. | | . . o . | | o . o + | |. + . * . | | + o oo S | | * o o. . | | o . o. | | o | | .E | +-----------------+ # ssh-copy-id 172.17.28.2 # ssh-copy-id 172.17.28.3 # ssh-copy-id 172.17.28.4
6、安裝數據庫
master namenode節點
# yum -y install mariadb-server # vi /etc/my.cnf [mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock symbolic-links=0 character-set-server=utf8 \\添加的行 [mysqld_safe] log-error=/var/log/mariadb/mariadb.log pid-file=/var/run/mariadb/mariadb.pid !includedir /etc/my.cnf.d # systemctl start mariadb # systemctl enable mariadb ln -s '/usr/lib/systemd/system/mariadb.service' '/etc/systemd/system/multi-user.target.wants/mariadb.service' # mysql_secure_installation /usr/bin/mysql_secure_installation: line 379: find_mysql_client: command not found NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY! In order to log into MariaDB to secure it, we'll need the current password for the root user. If you've just installed MariaDB, and you haven't set the root password yet, the password will be blank, so you should just press enter here. Enter current password for root (enter for none): OK, successfully used password, moving on... Setting the root password ensures that nobody can log into the MariaDB root user without the proper authorisation. Set root password? [Y/n] y \\設置數據庫root密碼 New password: \\設置密碼 Re-enter new password: \\再次輸入密碼 Password updated successfully! Reloading privilege tables.. ... Success! By default, a MariaDB installation has an anonymous user, allowing anyone to log into MariaDB without having to have a user account created for them. This is intended only for testing, and to make the installation go a bit smoother. You should remove them before moving into a production environment. Remove anonymous users? [Y/n] y \\移除匿名用戶 ... Success! Normally, root should only be allowed to connect from 'localhost'. This ensures that someone cannot guess at the root password from the network. Disallow root login remotely? [Y/n] y \\禁止root用戶遠程登錄 ... Success! By default, MariaDB comes with a database named 'test' that anyone can access. This is also intended only for testing, and should be removed before moving into a production environment. Remove test database and access to it? [Y/n] y \\刪除測試數據庫 - Dropping test database... ... Success! - Removing privileges on test database... ... Success! Reloading the privilege tables will ensure that all changes made so far will take effect immediately. Reload privilege tables now? [Y/n] // ... Success! Cleaning up... All done! If you've completed all of the above steps, your MariaDB installation should now be secure. Thanks for using MariaDB!
數據庫只是安裝了一下，並沒有做一些詳細的配置。如果有需求可以另行配置。
7、其他安裝與配置
所有節點
# yum -y install psmisc MySQL-python at bc bind-libs bind-utils cups-client cups-libs cyrus-sasl-gssapi cyrus-sasl-plain ed fuse fuse-libs httpd httpd-tools keyutils-libs-devel krb5-devel libcom_err-devel libselinux-devel libsepol-devel libverto-devel mailcap noarch mailx mod_ssl openssl-devel pcre-devel postgresql-libs python-psycopg2 redhat-lsb-core redhat-lsb-submod-security x86_64 spax time zlib-devel # chmod +x /etc/rc.d/rc.local # echo "echo 0 > /proc/sys/vm/swappiness" >>/etc/rc.d/rc.local # echo "echo never > /sys/kernel/mm/transparent_hugepage/defrag" >>/etc/rc.d/rc.local # echo 0 > /proc/sys/vm/swappiness # echo never > /sys/kernel/mm/transparent_hugepage/defrag # yum -y install rpcbind # systemctl start rpcbind # echo "systemctl start rpcbind" >> /etc/rc.d/rc.local
三、開始安裝CDH
1、傳寶、解包
把下載好的一下相應包文件上傳到master namenode節點內
mysql-java連接包在所有節點上都要上傳
# ls CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz manifest.json CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 jdk-8u91-linux-x64.rpm mysql-connector-java-6.0.2.jar # tar zxvf cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz -C /opt/ # ls /opt/ cloudera cm-5.7.0
2、創建數據庫
# mysql -uroot -p Enter password: \\輸入數據庫密碼 Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 10 Server version: 5.5.47-MariaDB MariaDB Server Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> create database monitor DEFAULT CHARSET utf8 COLLATE utf8_general_ci; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> grant all on *.* to root@"%" Identified by "1234.com"; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> exit # /opt/cm-5.7.0/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -p1234.com --scm-host localhost scm scm scm JAVA_HOME=/usr/java/jdk1.8.0_91 Verifying that we can write to /opt/cm-5.7.0/etc/cloudera-scm-server Creating SCM configuration file in /opt/cm-5.7.0/etc/cloudera-scm-server Executing: /usr/java/jdk1.8.0_91/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cm-5.7.0/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cm-5.7.0/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db. 2016-06-17 13:32:20,562 [main] INFO com.cloudera.enterprise.dbutil.DbCommandExecutor - Successfully connected to database. All done, your SCM database is configured correctly!
3、創建用戶
所有節點
#useradd --system --home=/opt/cm-5.7.0/run/cloudera-scm-server --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
4、制作本地源
master datanote節點執行
# ls CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz manifest.json CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 jdk-8u91-linux-x64.rpm mysql-connector-java-6.0.2.jar # cp CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 manifest.json /opt/cloudera/parcel-repo/ # mv CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha
5、拷貝jar包
所有節點
# ls
CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz manifest.json
CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 jdk-8u91-linux-x64.rpm mysql-connector-java-6.0.2.jar
# cp mysql-connector-java-6.0.2.jar /usr/share/java/mysql-connector-java.jar
# ls CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz manifest.json CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 jdk-8u91-linux-x64.rpm mysql-connector-java-6.0.2.jar # cp mysql-connector-java-6.0.2.jar /usr/share/java/mysql-connector-java.jar
master namenode節點
# cp mysql-connector-java-6.0.2.jar /opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hive/lib/ # cp mysql-connector-java-6.0.2.jar /opt/cm-5.7.0/share/cmf/lib/
6、更改配置
# sed -i "s/server_host=localhost/server_host=172.17.28.1/" /opt/cm-5.7.0/etc/cloudera-scm-agent/config.ini # mkdir /opt/cm-5.7.0/run/cloudera-scm-agent
7、啟動服務
master namenode節點
# /opt/cm-5.7.0/etc/init.d/cloudera-scm-server start
除了master namenode節點
# scp -r 172.17.28.1:/opt/cm-5.7.0 /opt
所有節點
# /opt/cm-5.7.0/etc/init.d/cloudera-scm-agent start
8、瀏覽頁面開始安裝
1）登陸
打開瀏覽器，訪問maser namenode節點的7180端口
輸入用戶名和密碼admin、admin
2）接受協議
3）選擇版本
本文檔選擇Cloudera Express免費開源版本
4）指定主機
5）選擇CDH版本
注：選擇在制作本地源時選擇的版本，如果選擇別的版本的話會去官網下載那樣安裝速度會很慢
6）檢查主機正確性
確保全部驗證通過
7）選擇安裝的服務
本文選擇的是安裝所有服務
8）角色分配
這里應該根據自己的實際需求和服務器的硬件參數自己選擇
9）數據庫設置
10）群集設置
可以根據自己的實際情況更改設置，本文采用默認配置。這些配置可以在集群安裝完成以后后續更改
11）開始安裝
無錯誤信息全部通過
12）安裝完成
第一次安裝完成后，會出現一些配置的警告信息。這些可以根據提示信息更改。
四、錯誤總結
1、pstree: command not found
錯誤展示
# /opt/cm-5.7.0/etc/init.d/cloudera-scm-server start /opt/cm-5.7.0/etc/init.d/cloudera-scm-server: line 109: pstree: command not found Starting cloudera-scm-server: [FAILED] # /opt/cm-5.7.0/etc/init.d/cloudera-scm-agent start /opt/cm-5.7.0/etc/init.d/cloudera-scm-agent: line 108: pstree: command not found Starting cloudera-scm-agent: [FAILED]
原因
因為系統是最小化安裝，默認沒有安裝
解決方法
# yum -y install psmisc
2、cloudera-scm-agent start 啟動失敗
錯誤展示
# /opt/cm-5.7.0/etc/init.d/cloudera-scm-agent start Starting cloudera-scm-agent: [FAILED]
查詢日志
、
# cd /opt/cm-5.7.0/log/cloudera-scm-agent/ # ls cloudera-scm-agent.out # cat cloudera-scm-agent.out [22/Jun/2016 11:02:09 +0000] 3695 MainThread agent INFO SCM Agent Version: 5.7.0 Unable to create the pidfile.
原因
經過查找后發現/opt/cm-5.7.0/run目錄下沒有cloudera-scm-agent文件夾
# cd /opt/cm-5.7.0/run/ # ls cloudera-scm-server
解決方法
# mkdir /opt/cm-5.7.0/run/cloudera-scm-agent
3、install: invalid user ‘cloudera-scm’
錯誤展示
# /opt/cm-5.7.0/etc/init.d/cloudera-scm-server start install: invalid user ‘cloudera-scm’
原因
沒有創建運行server的用戶
解決方法
#useradd --system --home=/opt/cm-5.7.0/run/cloudera-scm-server --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
4、檢查主機正確性警告
警告
解決方法
# echo 0 > /proc/sys/vm/swappiness # echo never > /sys/kernel/mm/transparent_hugepage/defrag # echo "echo 0 > /proc/sys/vm/swappiness" >>/etc/rc.d/rc.local # echo "echo never > /sys/kernel/mm/transparent_hugepage/defrag" >>/etc/rc.d/rc.local
5、啟動Hive失敗
錯誤展示
原因
根據提示發現是沒有mysql-java連接的驅動jar包
解決方法
# ls CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz manifest.json CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 jdk-8u91-linux-x64.rpm mysql-connector-java-6.0.2.jar # cp mysql-connector-java-6.0.2.jar /opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hive/lib/
6、啟動NFS Gateway失敗
錯誤展示
原因
解決方法
# yum -y install rpcbind # systemctl start rpcbind # echo "systemctl start rpcbind" >> /etc/rc.d/rc.local # chmod +x /etc/rc.d/rc.local
7、啟動Oozie,impala失敗
錯誤展示
原因
沒有mysql-java連接jar包
解決方法
# ls CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel cloudera-manager-centos7-cm5.7.0_x86_64.tar.gz manifest.json CDH-5.7.0-1.cdh5.7.0.p0.45-el7.parcel.sha1 jdk-8u91-linux-x64.rpm mysql-connector-java-6.0.2.jar # cp mysql-connector-java-6.0.2.jar /usr/share/java/mysql-connector-java.jar