Centos7 安裝 CDH6.3.2

1. 安裝包

1.1 CM

https://archive.cloudera.com/cm6/6.3.1/repo-as-tarball/cm6.3.1-redhat7.tar.gz.md5
https://archive.cloudera.com/cm6/6.3.1/allkeys.asc

1.2 CDH

https://archive.cloudera.com/cdh6/6.3.2/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel
https://archive.cloudera.com/cdh6/6.3.2/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1
https://archive.cloudera.com/cdh6/6.3.2/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256
https://archive.cloudera.com/cdh6/6.3.2/parcels/manifest.json

2. 服務器配置

2.1 hosts配置

127.0.0.1   localhost localhost
192.168.5.246   dev-node01
192.168.5.247   dev-node02
192.168.5.248   dev-node03

2.2 關閉iptables、firewalld、selinux

2.3 免密登錄（主節點到所有節點，secondnode同樣要求）

2.4 對時ntpd

2.5 關閉swap、禁用透明頁，寫入/etc/profile

echo 0 >  /proc/sys/vm/swappiness
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
swapoff -a

2.6 安裝jdk1.8

2.7 下載mysql-jdbc包

mkdir /usr/share/java/
wget http://central.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar
mv mysql-connector-java-8.0.16.jar mysql-connector-java.jar

2.8 mysql 數據庫創建

create database cmserver default charset utf8 collate utf8_general_ci;
grant all on cmserver.* to 'cmserveruser'@'%' identified by 'root';

create database hive default charset utf8 collate utf8_general_ci;
grant all on hive.* to 'hiveuser'@'%' identified by 'root';

create database amon default charset utf8 collate utf8_general_ci;
grant all on amon.* to 'amonuser'@'%' identified by 'root';

create database rman default charset utf8 collate utf8_general_ci;
grant all on rman.* to 'rmanuser'@'%' identified by 'root';

create database oozie default charset utf8 collate utf8_general_ci;
grant all on oozie.* to 'oozieuser'@'%' identified by 'root';

create database hue default charset utf8 collate utf8_general_ci;
grant all on hue.* to 'hueuser'@'%' identified by 'root';

2.9 安裝httpd服務

yum install httpd
systemctl start httpd
systemctl enable httpd.service

3. CM安裝

3.1 配置本地yum源

mkdir -p /var/www/html/cloudera-repos/
tar -xzvf cm6.3.1-redhat7.tar.gz -C /var/www/html/cloudera-repos/
wget https://archive.cloudera.com/cm6/6.3.1/allkeys.asc

vim /etc/yum.repos.d/cloudera-manager.repo
[cloudera-manager]
name = Cloudera Manager, Version cm6.3.1
baseurl = http://dev-node01/cloudera-repos/cm6.3.1
gpgcheck = 1

yum clean all
yum makecache

rpm --import http://dev-node01/cloudera-repos/cm6.3.1/RPM-GPG-KEY-cloudera

3.2 安裝cm

yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
# 生成sha
sha1sum CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel | awk '{print $1}' > CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha

 

 3.3 配置cm

# 初始化數據庫
/opt/cloudera/cm/schema/scm_prepare_database.sh  mysql -h192.168.5.231 -P3306 cmserver cmserveruser root
# 啟動cm
systemctl restart cloudera-scm-server

3.4 web安裝步驟（admin/admin） 

 

4. FAQ 

4.1  fail to format namenode

問題原因
是由於之前初始化 namenode 在 /dfs/nn 留下了殘留數據（失效數據），從而影響再次初始化

解決方法

namenode節點：
rm -rf /dfs/nn

datanode節點：
rm -rf /dfs/dn

4.2 主機運行狀況不良

rm -f /var/lib/cloudera-scm-agent/cm_guid
systemctl restart cloudera-scm-agent