集群節點數量3 個
192.168.1.170 cdh-master
192.168.1.171 cdh-slave-1
192.168.1.171 cdh-slave-2
一、安裝CentOS6.5 (64位)並搭建基本環境,包括:
(1)添加sudo 權限
(2)修改主機名、網關、靜態IP地址、DNS
(3)關閉SELINUX、防火牆
可參考文章
(4)修改系統時區、配置ntp服務(首先可選宿主機器配置ntp server,參考文章)
二、克隆為三台機器,分別修改靜態IP 地址、MAC、主機名、hosts文件:
主機名、網關修改 /etc/sysconfig/network文件
MAC 修改 /etc/udev/rules.d/70-persistent-net.rules
IP地址是修改 /etc/sysconfig/network-scripts/ifcfg-eth0
host 文件修改 /etc/hosts 需要添加集群內所有的主機名和ip的對應關系,而且主機名的名稱要和真實的主機名(/etc/sysconfig/network)完全一致
三、下載clouder-manger的安裝包及parcel文件:
cm 文件下載地址
cloudera-manager-daemons-5.2.1-1.cm521.p0.109.el6.x86_64.rpm
cloudera-manager-server-5.2.1-1.cm521.p0.109.el6.x86_64.rpm
cloudera-manager-agent-5.2.1-1.cm521.p0.109.el6.x86_64.rpm
jdk-6u31-linux-amd64.rpm
oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm
parcel 下載地址
CDH-5.2.1-1.cdh5.2.1.p0.12-el6.parcel
四、在master 節點安裝daemons、server、agent(先裝daemons)
yum --nogpgcheck localinstall cloudera-manager-daemons-5.2.1-1.cm521.p0.109.el6.x86_64.rpm
yum --nogpgcheck localinstall cloudera-manager-server-5.2.1-1.cm521.p0.109.el6.x86_64.rpm
yum --nogpgcheck localinstall cloudera-manager-agent-5.2.1-1.cm521.p0.109.el6.x86_64.rpm(注:agent安裝需要聯網)
五、在slave-1、slave-2節點安裝daemons、agent(先裝daemons)
yum --nogpgcheck localinstall cloudera-manager-daemons-5.2.1-1.cm521.p0.109.el6.x86_64.rpm
yum --nogpgcheck localinstall cloudera-manager-agent-5.2.1-1.cm521.p0.109.el6.x86_64.rpm(注:agent安裝需要聯網)
六、在master、slave-1、slave-2 節點安裝JDK、oraclejdk
rpm -ivh jdk-6u31-linux-amd64.rpm
七、在master節點安裝mysql 數據庫,並配置cdh需要的數據庫選項
yum install mysql-server mysql mysql-deve (注:需要聯網)
chkconfig mysqld on
service mysqld start
mysql –u root
use mysql
update user set password=password('1234') where user='root'
update user set password=password('1234') where host='localhost'
update user set password=password('1234') where host='cdh-mater'
service mysqld restart
mysql -u root -p1234
create database cloudera
八、在master節點配置cloudera manager 數據庫並啟動cm的server及agent程序
1.拷貝mysql-connector-java-5.1.7-bin.jar 到 /usr/share/java 下並重命名mysql-connector-java.jar
2.運行 /usr/share/cmf/schema/scm_prepare_database.sh -h cdh-master mysql cloudera root 1234
3.啟動cm server :service cloudera-scm-server start
4.添加cm server服務 :chkconfig cloudera-scm-server on
5.啟動cm agent :chkconfig cloudera-scm-agent on
6.添加cm agent服務 :service cloudera-scm-server start
九、修改所有節點的agent 配置文件
/etc/cloudera-scm-agent/config.ini 將配置文件中的host 改成 cdh-master
十、在slave節點配置cloudera manager agent程序
1.啟動cm agent :chkconfig cloudera-scm-agent on
2.添加cm agent服務 :service cloudera-scm-server start
十一、測試agent和server是否通信成功
service cloudera-scm-server status
service cloudera-scm-agent status
netstat –anp | grep 7182
# server 端開啟的是7182端口,用於和agent進行通訊
啟動失敗時可以查看日志
server 日志 /var/log/cloudera-scm-server
agent 日志 /var/log/cloudera-scm-agent
十二、將下載的parcel文件及官方網站上獲取對應版本的 sha 文件和 manifest.json文件上傳至/opt/cloudera/parcel-repo目錄下。
十三、啟動瀏覽器,開始集群web安裝配置