【CDH部署】CDH6.2安裝配置第二篇：CDH安裝的前期配置

本篇介紹cdh安裝之前需要的一些必要配置，當然這些配置也可以用shell腳本來配置。在安裝之前請先配置好yum源，在文中用的統一都是阿里源。在安裝的時候，要確保主機的內存是4G以上，要不然會無限重啟cm服務

 另外2篇傳送門：

CDH6.2安裝配置第一篇：CDH配置本地http服務

CDH6.2安裝配置第三篇：前台頁面配置講解

其他配置的傳送門：

LINUX的ssh免密碼配置

LINUX之ntp時間同步服務配置

CDH版本：6.2 

系統：centos7（el6是代表centos6系統，centos7則用el7表示）

主機3台：

master.com.cn 192.168.1.110   2C4G

salver1.com.cn 192.168.1.111   1C2G

salver2.com.cn 192.168.1.112  1C2G

yum服務器：yum.com.cn 192.168.1.100。這里如果不知道怎么配置的朋友，可以參考第一篇：https://www.cnblogs.com/yaowentao/p/12296329.html

 

基本配置

1. 主機名在系統安裝的配置完畢，如果沒有配置，可以進系統在修改

$ vi /etc/hostname

2. 配置host文件

$ vi /etc/hosts
192.168.1.100 yum.com.cn
192.168.1.110 master.com.cn
192.168.1.111 salver1.com.cn
192.168.1.112 salver2.com.cn

3. 關閉防火牆，selinux。配置完后重啟

$ sudo systemctl disable firewalld        #停止防火牆
$ sudo systemctl stop firewalld           #禁止開機啟動防火牆

$ vi /etc/selinux/config
SELINUX=disable　　　　　　　　　　　　　　　　#關閉selinux

 

4. 配置ntp服務與ssh免密碼登陸配置，請參考我的另外2篇文檔

 ssh免密碼配置傳送門：https://www.cnblogs.com/yaowentao/p/12300321.html

 ntp配置傳送門：https://www.cnblogs.com/yaowentao/p/12300408.html

5. 如果有本地的yum源，就配置。如果沒有就使用公網的yum源。

5.1 配置yum 本地庫,也可以下載官網的repo文件，一個是從網上下載，一個是從本地yum庫下載

[cloudera-repo]
name=cloudera-repo
baseurl=http://192.168.10.225/cloudera-repos/cm6/6.2.1/redhat7/yum/
enabled=1
gpgcheck=0

5.2 網上yum源，直接下載即可

$ sudo wget https://archive.cloudera.com/cm6/6.2.1/redhat7/yum/cloudera-manager.repo -P /etc/yum.repos.d/ 

6. 安裝cdh專用jdk（需要在所有主機上都要安裝）

$ sudo yum -y install oracle-j2sdk1.8

7. 配置本地的mysql數據庫，便於存放cdh服務器的一些配置，以及hadoop組件中需要的一些配置，如hive的源數據庫。這里一般推薦用mysql，當然oracle等其他的數據庫也可以。

1. 直接用yum源安裝

2. $ wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
   $ sudo rpm -ivh mysql-community-release-el7-5.noarch.rpm
   $ sudo yum update
   $ sudo yum install mysql-server
   $ sudo systemctl start mysqld
   
   

   
3. 配置mysql數據，並進行初始化，設置root密碼

4. $vi /etc/my.cnf
   
   [mysqld]
   datadir=/var/lib/mysql
   socket=/var/lib/mysql/mysql.sock
   transaction-isolation = READ-COMMITTED
   # Disabling symbolic-links is recommended to prevent assorted security risks;
   # to do so, uncomment this line:
   symbolic-links = 0
   
   
   key_buffer_size = 32M
   max_allowed_packet = 32M
   thread_stack = 256K
   thread_cache_size = 64
   query_cache_limit = 8M
   query_cache_size = 64M
   query_cache_type = 1
   
   
   max_connections = 550
   #expire_logs_days = 10
   #max_binlog_size = 100M
   
   
   #log_bin should be on a disk with enough free space.
   #Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
   #system and chown the specified folder to the mysql user.
   log_bin=/var/lib/mysql/mysql_binary_log
   
   
   #In later versions of MySQL, if you enable the binary log and do not set
   #a server_id, MySQL will not start. The server_id must be unique within
   #the replicating group.
   server_id=1
   
   
   binlog_format = mixed
   
   
   read_buffer_size = 2M
   read_rnd_buffer_size = 16M
   sort_buffer_size = 8M
   join_buffer_size = 8M
   
   
   # InnoDB settings
   innodb_file_per_table = 1
   innodb_flush_log_at_trx_commit  = 2
   innodb_log_buffer_size = 64M
   innodb_buffer_pool_size = 4G
   innodb_thread_concurrency = 8
   innodb_flush_method = O_DIRECT
   innodb_log_file_size = 512M
   
   
   [mysqld_safe]
   log-error=/var/log/mysqld.log
   pid-file=/var/run/mysqld/mysqld.pid
   
   
   sql_mode=STRICT_ALL_TABLES

5. 配置開機自啟動

6. $ sudo systemctl enable mysqld
   $ sudo systemctl start mysqld

7. 初始化mysql，設置root密碼

8. $ sudo /usr/bin/mysql_secure_installation
   [...]
   Enter current password for root (enter for none):
   OK, successfully used password, moving on...
   [...]
   Set root password? [Y/n] Y
   New password:
   Re-enter new password:
   Remove anonymous users? [Y/n] Y
   [...]
   Disallow root login remotely? [Y/n] N
   [...]
   Remove test database and access to it [Y/n] Y
   [...]
   Reload privilege tables now? [Y/n] Y
   All done!

   

    

    

    

9. 安裝mysql的jdbc

10. $ wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
    $ tar zxvf mysql-connector-java-5.1.46.tar.gz
    
    $ sudo mkdir -p /usr/share/java/
    $ cd mysql-connector-java-5.1.46
    $ sudo cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar

11. 給root授遠程登陸權限

12. mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' WITH GRANT OPTION;
    mysql> FLUSH PRIVILEGES;

13. 創建cm服務，hadoop組件需要的庫。用戶統一用root。

14. CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON scm.* TO 'root'@'%' IDENTIFIED BY '123456';
    CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON amon.* TO 'root'@'%' IDENTIFIED BY '123456';
    CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON rman.* TO 'root'@'%' IDENTIFIED BY '123456';
    CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON hue.* TO 'root'@'%' IDENTIFIED BY '123456';
    CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON metastore.* TO 'root'@'%' IDENTIFIED BY '123456';
    CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON sentry.* TO 'root'@'%' IDENTIFIED BY '123456';
    CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON nav.* TO 'root'@'%' IDENTIFIED BY '123456';
    CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON navms.* TO 'root'@'%' IDENTIFIED BY '123456';
    CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON oozie.* TO 'root'@'%' IDENTIFIED BY '123456';

    

15.  安裝cm服務器文件

16. master:
    $ sudo yum -y install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
    

    

     

     

    salver:
    $ sudo yum -y install cloudera-manager-daemons cloudera-manager-agent

     
17. scm初始化
    1. 　　

       $ sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm root

       
18. 完成以上配置，就能正常登陸cm服務器的前台了。下面進入服務器的配置。請看第三篇。
    1. 　　啟動cm服務

       $ sudo systemctl start cloudera-scm-server
       $ sudo tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

       

啟動成功： 

 

 

各位小伙伴，對以上如有問題，請在下方留言溝通。以上都是我親自測試通過安裝成功的。

 

轉發，請注明出處！