Ubuntu14.04下完美安裝cloudermanage多種方式（圖文詳解）（博主推薦）

 

 

　　

說在前面的話

　　我的機器是總共4台，分別為ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4。

 

　　ClouderaManager官網

https://www.cloudera.com/documentation/manager/5-0-x/Cloudera-Manager-Installation-Guide/cm5ig_install_path_B.html

 

 

 

 

　　在ubuntucmbigdata1上執行了

sudo apt-get install cloudera-manager-daemons cloudera-manager-server 


sudo apt-get install cloudera-manager-daemons  cloudera-manager-agent 

　　即，ubuntucmbigdata1是既做server，又做agent。

 

 

　　在ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4上執行了

sudo apt-get install cloudera-manager-daemons  cloudera-manager-agent 

　　即，ubuntucmbigdata2、3、和4是只做agent。

 

 

　　在ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3、ubuntucmbigdata4上

新建文件sudo vi /etc/apt/sources.list.d/cloudera-manager.list，復制以下內容到cloudera.list里

# Packages for Cloudera Manager, Version 5, on Ubuntu 14.04 x86_64
deb [arch=amd64] http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib
deb-src http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib

 

　　並且在在ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3、ubuntucmbigdata4上

　　向apt添加cloudera源公鑰，在所有節點執行如下命令：

curl -s http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key | sudo apt-key add -

然后，再

sudo apt-get update

　

 

 

 

 

方式1、將CDH5的軟件源添加到Apt中，通過在線安裝。

參考

Ubuntu14.04用apt在線/離線安裝CDH5.1.2[Apache Hadoop 2.3.0]

 https://www.zybuluo.com/ncepuwanghui/note/474966

 配置Cloudera軟件倉庫

Cloudera官方文檔提供了兩種管理倉庫策略： 
• Standard Cloudera repositories 
• Internally hosted repositories

本文采用Standard Cloudera repositories，步驟如下： 
① 在ubuntucmbigdata1節點下載cloudera.list

wget https://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/cloudera.list?_ga=1.37493987.2113753785.1471598714

 

② 在ubuntucmbigdata1節點新建文件sudo vi /etc/apt/sources.list.d/cloudera-manager.list，復制cloudera.list的內容

# Packages for Cloudera Manager, Version 5, on Ubuntu 14.04 x86_64
deb [arch=amd64] http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib
deb-src http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib

③ 將cloudera-manager.list分發到所有slave節點：

scp /etc/apt/sources.list.d/cloudera-manager.list bigdata@ubuntucmbigdata2:/etc/apt/sources.list.d/cloudera-manager.list
scp /etc/apt/sources.list.d/cloudera-manager.list bigdata@ubuntucmbigdata3:/etc/apt/sources.list.d/cloudera-manager.list
scp /etc/apt/sources.list.d/cloudera-manager.list bigdata@ubuntucmbigdata4:/etc/apt/sources.list.d/cloudera-manager.list

 

1. ④ 向apt添加cloudera源公鑰，在所有節點執行如下命令：

curl -s http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key | sudo apt-key add -

 

1. ⑤ 在所有節點，更新源

sudo apt-get update

 

 

若更新出錯，提示如下：

1. W: GPG error: http://archive.cloudera.com trusty-cm5 InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 327574EE02A818DD

可執行如下命令，添加cloudera源的公鑰，重新更新源：

1. curl -s http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key | sudo apt-key add -

 　　

 

　　

 

安裝Cloudera Manager Server

• 安裝Cloudera Manager Server軟件包 
  在maubuntucmbigdata1ster節點上執行：

  1. sudo apt-get install cloudera-manager-daemons cloudera-manager-server

• 創建Cloudera Manager Server數據庫scm

• mysql -u root -p
  create database scm DEFAULT CHARACTER SET utf8;
  grant all on scm.* TO 'scm'@'%' IDENTIFIED BY 'scm_password';
  grant all on *.* TO 'scm'@'ubuntucmbigdata1' IDENTIFIED BY 'scm_password';
  grant all on scm.* TO 'scm'@'localhost' IDENTIFIED BY 'scm_password';
  flush privileges;

   

• 同時，關於這里的mysql配置，還要加上，我這篇博客里的mysql內容

  Cloudera Manager安裝之Cloudera Manager安裝前准備(Ubuntu14.04）（一）

  • mysql> create database hive DEFAULT CHARACTER SET utf8;
    Ignoring query to other database
    mysql> grant all on hive.* TO 'hive'@'localhost' IDENTIFIED BY 'hive_password';
    Ignoring query to other database
    mysql> grant all on *.* TO 'hive'@'ubuntucmbigdata1' IDENTIFIED BY 'hive_password';
    Ignoring query to other database
    mysql> grant all on hive.* TO 'hive'@'%' IDENTIFIED BY 'hive_password';  
    Ignoring query to other database
    mysql> create database oozie DEFAULT CHARACTER SET utf8;
    Ignoring query to other database
    mysql> grant all on oozie.* TO 'oozie'@'localhost' IDENTIFIED BY 'oozie_password';
    Ignoring query to other database
    mysql> grant all on *.* TO 'oozie'@'ubuntucmbigdata1' IDENTIFIED BY 'oozie_password';
    Ignoring query to other database
    mysql> grant all on oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie_password';
    Ignoring query to other database
    mysql> create database amon DEFAULT CHARACTER SET utf8;
    Ignoring query to other database
    mysql> grant all on amon.* TO 'amon'@'localhost' IDENTIFIED BY 'amon_password';
    Ignoring query to other database
    mysql> grant all on *.* TO 'amon'@'ubuntucmbigdata1' IDENTIFIED BY 'amon_password';
    Ignoring query to other database
    mysql> grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'amon_password';
    Ignoring query to other database
    mysql> create database smon DEFAULT CHARACTER SET utf8;
    Ignoring query to other database
    mysql> grant all on smon.* TO 'smon'@'localhost' IDENTIFIED BY 'smon_password';
    Ignoring query to other database
    mysql> grant all on *.* TO 'smon'@'ubuntucmbigdata1' IDENTIFIED BY 'smon_password';
    Ignoring query to other database
    mysql> grant all on smon.* TO 'smon'@'%' IDENTIFIED BY 'smon_password';
    Ignoring query to other database
    mysql> create database rman DEFAULT CHARACTER SET utf8;
    Ignoring query to other database
    mysql> grant all on rman.* TO 'rman'@'localhost' IDENTIFIED BY 'rman_password';
    Ignoring query to other database
    mysql> grant all on *.* TO 'rman'@'ubuntucmbigdata1' IDENTIFIED BY 'rman_password';
    Ignoring query to other database
    mysql> grant all on rman.* TO 'rman'@'%' IDENTIFIED BY 'rman_password';
    Ignoring query to other database
    mysql> create database hmon DEFAULT CHARACTER SET utf8;
    Ignoring query to other database
    mysql> grant all on hmon.* TO 'hmon'@'localhost' IDENTIFIED BY 'hmon_password';
    Ignoring query to other database
    mysql> grant all on *.* TO 'hmon'@'ubuntucmbigdata1' IDENTIFIED BY 'hmon_password';
    Ignoring query to other database
    mysql> grant all on hmon.* TO 'hmon'@'%' IDENTIFIED BY 'hmon_password';
    Ignoring query to other database
    mysql> create database nav DEFAULT CHARACTER SET utf8;
    Ignoring query to other database
    mysql> grant all on nav.* TO 'nav'@'localhost' IDENTIFIED BY 'nav_password';
    Ignoring query to other database
    mysql> grant all on *.* TO 'nav'@'ubuntucmbigdata1' IDENTIFIED BY 'nav_password';
    Ignoring query to other database
    mysql> grant all on nav.* TO 'nav'@'%' IDENTIFIED BY 'nav_password';
    Ignoring query to other database
    mysql> flush privileges;
    

    mysql> exit;
    Bye
    root@ubuntucmbigdata1:~# 

     

• 配置啟動Cloudera Manager Server （只在ubuntucmbigdata1上執行）

  1. sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm scm_password

   

• 也許大家會出現這個錯誤。

• root@ubuntucmbigdata1:~# sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm scm_password
  +======================================================================+
  |      Error: JAVA_HOME is not set and Java could not be found         |
  +----------------------------------------------------------------------+
  | Please download the latest Oracle JDK from the Oracle Java web site  |
  |  > http://www.oracle.com/technetwork/java/javase/index.html <        |
  |                                                                      |
  | Cloudera Manager requires Java 1.6 or later.                         |
  | NOTE: This script will find Oracle Java whether you install using    |
  |       the binary or the RPM based installer.                         |
  +======================================================================+
  root@ubuntucmbigdata1:~#

  則，需要安裝jdk。

• 成功的話，則是如下

• bigdata@ubuntucmbigdata1:/usr/java$ sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm scm_password
  JAVA_HOME=/usr/java/jdk1.7.0_80
  Verifying that we can write to /etc/cloudera-scm-server
  Creating SCM configuration file in /etc/cloudera-scm-server
  Executing:  /usr/java/jdk1.7.0_80/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
  [                          main] DbCommandExecutor              INFO  Successfully connected to database.
  All done, your SCM database is configured correctly!
  bigdata@ubuntucmbigdata1:/usr/java$ 

   

• 安裝JDK1.7 （所有節點）
  CDH要求至少是Oracle JDK7，Ubuntu自帶的是OpenJDK，Ubuntu沒有自帶的Oracle JDK,必須添加外部源才能安裝JDK。 
  ① 采用Cloudera軟件源 
  在上一步配置Cloudera軟件源后，可直接執行下面命令安裝：

  1. sudo apt-get install oracle-j2sdk1.7

  ② 采用PPP源 
  添加PPA repository到系統,配置PPP源安裝：

  1. sudo add-apt-repository ppa:webupd8team/java
  2. sudo apt-get update
  3. sudo apt-get upgrade
  4. sudo apt-get install oracle-java7-installer
  5. sudo apt-get install oracle-java7-set-default
  6. java -version

  注：若提示如下信息：

  1. sha256sum mismatch jdk-7u80-linux-x64.tar.gz
  2. Oracle JDK 7 is NOT installed.

  可能是jdk軟件包沒有下載完全，可執行如下命令刪除后重新安裝：

  1. sudo apt-get purge oracle-java7-installer

  ③ 手動安裝 
  從Oracle官網下載JDK7u80：jdk-7u80-linux-x64.tar.gz，解壓到/usr/java/jdk.1.7.0_80

  1. mkdir /usr/java
  2. sudo tar -zxvf jdk-7u80-linux-x64.tar.gz -C /usr/java

  配置JDK環境變量： 

• vim /etc/profile

  • source /etc/profile

    #jdk
    export JAVA_HOME=/usr/java/jdk1.7.0_80
    export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
    export PATH=$PATH:$JAVA_HOME/bin

     

 

　　如果上述安裝錯了，查看和卸載命令。

Ubuntu系統里如何正確卸載oracle-j2sdk1.6（圖文詳解）

　　則為

dpkg --list | grep -i jdk
sudo apt-get remove oracle-j2sdk1.6
sudo apt-get purge oracle-java7-set-default

sudo apt-get purge  oracle-java7-installer  

 

　　　　安裝orcal-java7-installer出錯，則見

　　orcal-java7-installer中斷錯誤 CSDN

 

 

 

 

• 　sudo service cloudera-scm-server start（只在ubuntucmbigdata1上執行）
• 成功，則出現

• bigdata@ubuntucmbigdata1:/usr/java$ sudo service cloudera-scm-server start
  Starting cloudera-scm-server:  * cloudera-scm-server started
  
  bigdata@ubuntucmbigdata1:/usr/java$ 

   

•  

  若啟動報錯：

  1. /etc/init.d/cloudera-scm-server: line 292: [[: [error] JVM PermGen is set less than 256m, CM server may run out of PermGen space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.: syntax error: operand expected (error token is "[error] JVM PermGen is set less than 256m, CM server may run out of PermGen space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.")
  2. /etc/init.d/cloudera-scm-server: line 292: [[: [error] JVM MaxHeap is set less than 2G, CM server may run out of Heap space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.: syntax error: operand expected (error token is "[error] JVM MaxHeap is set less than 2G, CM server may run out of Heap space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.")

  修改配置文件/etc/default/cloudera-scm-server，找到export CM_JAVA_OPTS部分，將-Xmx2G修改為-Xmx4G

• 查看日志，驗證是否成功啟動

  1. tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

 

 

 

 

 

安裝Cloudera Manager Agents

• 安裝Cloudera Manager Agent軟件包 
  在所有節點（包括ubuntucmbigdata1）上執行：

  1. sudo apt-get install cloudera-manager-agent cloudera-manager-daemons

• 配置Cloudera Manager Agent 
  在所有節點上，修改配置文件sudo vi /etc/cloudera-scm-agent/config.ini，配置Cloudera Manager Server的主機和端口：

  1. server_host=ubuntucmbigdata1
  2. server_port=7182 #默認端口即可

• 啟動Cloudera Manager Agent（包含ubuntucmbigdata1）

  1. sudo service cloudera-scm-agent start

 

bigdata@ubuntucmbigdata1:~$ sudo service cloudera-scm-agent start
cloudera-scm-agent is already running
bigdata@ubuntucmbigdata1:~$ 

bigdata@ubuntucmbigdata2:~$ sudo service cloudera-scm-agent start
cloudera-scm-agent is already running bigdata@ubuntucmbigdata1:~$ 

bigdata@ubuntucmbigdata3:~$ sudo service cloudera-scm-agent start
cloudera-scm-agent is already running bigdata@ubuntucmbigdata1:~$ 

bigdata@ubuntucmbigdata4:~$ sudo service cloudera-scm-agent start
cloudera-scm-agent is already running bigdata@ubuntucmbigdata1:~$ 

 

 

 

 　　

 

 

 

 

 

　　

方式2、放在/opt/softwares/cm-5.6.0

參考

Cloudera Manager安裝之Cloudera Manager 5.3.X安裝（三）（tar方式、rpm方式和yum方式）

　　

 

　　先不用安裝什么jdk，以及什么其他的。只如下執行就是

 

 

　　執行cloudera-manager-installer.bin

bigdata@ubuntucmbigdata1:~$ sudo mkdir -p /opt/softwares/cm-5.6.0
[sudo] password for bigdata: 
bigdata@ubuntucmbigdata1:~$ cd /opt/softwares/cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ pwd
/opt/softwares/cm-5.6.0
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 8
drwxr-xr-x 2 root root 4096  6月 25 22:19 ./
drwxr-xr-x 3 root root 4096  6月 25 22:19 ../
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chown -R bigdata:bigdata /opt/softwares/*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 8
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 22:19 ./
drwxr-xr-x 3 root    root    4096  6月 25 22:19 ../
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rw-r--r-- 1 root    root    517878  6月 25 22:17 cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chown -R bigdata:bigdata cloudera-manager-installer.bin 
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rw-r--r-- 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chmod +x ./cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rwxr-xr-x 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ 

 

 

bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ pwd
/etc/apt/sources.list.d
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 12
drwxr-xr-x 2 root root 4096  6月 25 20:26 ./
drwxr-xr-x 6 root root 4096  6月 25 20:26 ../
-rw-r--r-- 1 root root  136  6月 25 20:26 webupd8team-java-trusty.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo curl "http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/cloudera.list" -o /etc/apt/sources.list.d/cloudera_precise.list
[sudo] password for bigdata: 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   263  100   263    0     0    422      0 --:--:-- --:--:-- --:--:--   422
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 16
drwxr-xr-x 2 root root 4096  6月 25 22:26 ./
drwxr-xr-x 6 root root 4096  6月 25 20:26 ../
-rw-r--r-- 1 root root  263  6月 25 22:26 cloudera_precise.list
-rw-r--r-- 1 root root  136  6月 25 20:26 webupd8team-java-trusty.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo curl -s http://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -
OK
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo apt-get update

 

 

 

 

 

bigdata@ubuntucmbigdata1:~$ cd /etc/apt/sources.list.d
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 16
drwxr-xr-x 2 root root 4096 Jun 26 11:10 ./
drwxr-xr-x 6 root root 4096 Jun 26 11:10 ../
-rw-r--r-- 1 root root  175 Oct 20  2015 cloudera-cm5.list
-rw-r--r-- 1 root root  263 Jun 26 11:10 cloudera_precise.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$

 

 

 

 

 

bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rwxr-xr-x 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo ./cloudera-manager-installer.bin

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 　　省略。

 

 

　　也許中間，會報錯，這個一一解決就是了

 

 

bigdata@ubuntucmbigdata1:~$ cat /var/log/cloudera-manager-installer/3.install-oracle-j2sdk1.7.log
Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
  libntdb1 python-ntdb Use 'apt-get autoremove' to remove them. The following NEW packages will be installed: oracle-j2sdk1.7 0 upgraded, 1 newly installed, 0 to remove and 6 not upgraded. 2 not fully installed or removed. Need to get 142 MB of archives. After this operation, 292 MB of additional disk space will be used. Get:1 http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/ trusty-cm5/contrib oracle-j2sdk1.7 amd64 1.7.0+update67-1 [142 MB] Fetched 142 MB in 15min 9s (157 kB/s) Selecting previously unselected package oracle-j2sdk1.7. (Reading database ... 172281 files and directories currently installed.) Preparing to unpack .../oracle-j2sdk1.7_1.7.0+update67-1_amd64.deb ... Unpacking oracle-j2sdk1.7 (1.7.0+update67-1) ... Setting up oracle-java7-installer (7u80+7u60arm-0~webupd8~1) ... Downloading Oracle Java 7... --2017-06-26 00:20:41-- http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz Resolving download.oracle.com (download.oracle.com)... 23.2.16.218, 23.2.16.121 Connecting to download.oracle.com (download.oracle.com)|23.2.16.218|:80... connected. HTTP request sent, awaiting response... 302 Moved Temporarily Location: https://edelivery.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz [following] --2017-06-26 00:20:41-- https://edelivery.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz Resolving edelivery.oracle.com (edelivery.oracle.com)... 96.16.169.83, 2402:4f00:4001:197::2d3e, 2402:4f00:4001:180::2d3e Connecting to edelivery.oracle.com (edelivery.oracle.com)|96.16.169.83|:443... connected. HTTP request sent, awaiting response... 302 Moved Temporarily Location: http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz?AuthParam=1498407762_def59b5e0df725a8c8cee2b6d12ac74e [following] --2017-06-26 00:20:42-- http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz?AuthParam=1498407762_def59b5e0df725a8c8cee2b6d12ac74e Connecting to download.oracle.com (download.oracle.com)|23.2.16.218|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2017-06-26 00:20:43 ERROR 404: Not Found. download failed Oracle JDK 7 is NOT installed. dpkg: error processing package oracle-java7-installer (--configure): subprocess installed post-installation script returned error exit status 1 dpkg: dependency problems prevent configuration of oracle-java7-set-default: oracle-java7-set-default depends on oracle-java7-installer; however: Package oracle-java7-installer is not configured yet. dpkg: error processing package oracle-java7-set-default (--configure): dependency problems - leaving unconfigured Setting up oracle-j2sdk1.7 (1.7.0+update67-1) ... No apport report written because the error message indicates its a followup error from a previous failure. Errors were encountered while processing: oracle-java7-installer oracle-java7-set-default E: Sub-process /usr/bin/dpkg returned an 

 

 

 

　　解決這個問題，還是得自己手動下載來安裝，才得以解決。

 　　它的下載鏈接

http://download.oracle.com/otn/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz

 

 

 

#java
export JAVA_HOME=/usr/java/jdk1.7.0_80 export JRE_HOME=$JAVA_HOME/jre export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib export PATH=$PATH:$JAVA_HOME/bin

 

 

 

 

 

bigdata@ubuntucmbigdata1:/usr/java$ sudo vim ~/.bashrc

 

 

 

 

 

 

總結

　　對於這步安裝jdk，需要注意，見

https://www.zybuluo.com/ncepuwanghui/note/474966

Ubuntu Server 14.04 安裝部署 CDH5.7.2

 

 

 

 

 

 

 

 

 

 

 

 

 

 

　　這里，我也覺得很奇怪，在ubuntu系統里，是localhost，可我明明注銷掉了。這跟centos下安裝有點不同。

 

 

 　　但是呢，竟然可以用ubuntucmbigdata1:7810或者192.168.86.31來訪問。

 

 

 

　　當安裝完成，Cloudera Manager 管理控制台會提供一個完成的url包括默認端口7180。（這個安裝過程可能持續很長時間，網速較慢，可能會持續一上午，甚至一天）
這時候把這個地址輸入瀏覽器地址欄，就能看到頁面了。默認用戶名和密碼都為admin.

 

 

 

 

 

 　　成功！

 

 

 　　若是通過bin方式安裝的話，則

 

bigdata@ubuntucmbigdata1:/etc/init.d$ pwd
/etc/init.d
bigdata@ubuntucmbigdata1:/etc/init.d$ ls
acpid    apparmor      bluetooth            cloudera-scm-server-db  cups          dns-clean          halt        killprocs  mysql       ondemand    procps      rc.local  reboot      rsync    sendsigs  speech-dispatcher  thermald  umountnfs.sh         urandom
anacron  apport        brltty               console-setup           cups-browsed  friendly-recovery  irqbalance  kmod       networking  postgresql  pulseaudio  rcS       resolvconf  rsyslog  single    ssh                udev      umountroot           x11-common
apache2  avahi-daemon  cloudera-scm-server  cron                    dbus          grub-common        kerneloops  lightdm    ntp         pppd-dns    rc          README    rpcbind     saned    skeleton  sudo               umountfs  unattended-upgrades
bigdata@ubuntucmbigdata1:/etc/init.d$ sudo /etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:  * cloudera-scm-server started

bigdata@ubuntucmbigdata1:/etc/init.d$ sudo /etc/init.d/cloudera-scm-server-db start
 * Cloudera manager database already started
bigdata@ubuntucmbigdata1:/etc/init.d$ 

 

 

 

 

 

bigdata@ubuntucmbigdata3:~$ cd /etc/init.d/
bigdata@ubuntucmbigdata3:/etc/init.d$ ls
acpid    apparmor      bluetooth           console-setup  cups-browsed  friendly-recovery  irqbalance  kmod        ntp       procps      rc.local  reboot      rsync    sendsigs  speech-dispatcher  thermald  umountnfs.sh         urandom
anacron  apport        brltty              cron           dbus          grub-common        kerneloops  lightdm     ondemand  pulseaudio  rcS       resolvconf  rsyslog  single    ssh                udev      umountroot           x11-common
apache2  avahi-daemon  cloudera-scm-agent  cups           dns-clean     halt               killprocs   networking  pppd-dns  rc          README    rpcbind     saned    skeleton  sudo               umountfs  unattended-upgrades
bigdata@ubuntucmbigdata3:/etc/init.d$ sudo ./cloudera-scm-agent start
cloudera-scm-agent is already running
bigdata@ubuntucmbigdata3:/etc/init.d$ 

 

 

 

 

 

 

 

 

 

 

 

 方式3，通過tar,gz方法

其實很簡單，網上寫的資料各種雜活。就是如下！（博主，我是很喜歡這個人寫的這篇博客）！！！！

在線和離線安裝Cloudera CDH 5.6.0

 

　　的確，我也跟這位博主一樣，也需要在實驗室里部署。以及也是4台機器。（至於原因，大家都懂得，中國高校研究生喜歡4、3、2、1來進行加速比做實驗論文）

 

     在實驗室安裝過社區版的Hadoop，當時用的是普通PC機，安裝的過程遇到了好多問題。當解決了大部分的安裝問題之后由於實驗室的需求，需要安裝一個商業發行版的hadoop ，我們就采用的Cloudera 公司發行的版本。首先要說明的是CDH對內存要求比較高，當我們采用原來的4G內存的PC機，只要啟動基本內存就滿了，並且會拖慢整個集群的運算速度。所以說安裝CDH對硬件的要求比較高。安裝CDH 有三種安裝方式：在線安裝、半離線安裝、離線安裝。在線安裝和離線安裝我都安裝過，雖然在線安裝貌似簡單但是實際用起來還是沒有離線安裝方便，我還是建議大家采用離線安裝的方式。不過下面都會介紹在線安裝的方式和離線安裝的方式。希望對別人有所幫助。

 

 　　關於，前期的介紹、硬件基礎、安裝Ubuntu14.04系統、設置hosts、關閉防火牆、設置靜態IP、安裝Oracle JDK 8（所有節點）、所有節點免密碼SSH登陸、建立數據庫（僅主機）、設置集群時間同步(所有節點）。我都上面已經做好了。

 

 

 

相關軟件和下載

Cloudera Manager : http://archive.cloudera.com/cm5/cm/5/   
Cloudera CDH：http://archive.cloudera.com/cdh5 
MySql的JDBC驅動jar包 

 

 

　　因為，我這里是ubuntu14.04，所以

 

 

 

　　同理，我這里，也不去一枚地追求新，采用的是5.6.0

 

 

 

 　　這個網上多的是，大家隨便去下載就是了

 

 

 

 

 

 

 

根據系統下載所相應的版本，下載的文件如下

cloudera-manager-trusty-cm5.6.0_amd64.tar.gz
CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel
CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel.sha
manifest.json 
mysql-connector-java-5.1.32-bin.jar

 

 

 

 

 　　

　　解壓 cloudera-manager-trusty-cm5.6.0_amd64.tar.gz 到 /opt 目錄下

 　　先弄清楚，為什么這么做，是因為，在線安裝的話，默認是安裝在/opt下。所以，我這里帶大家離線安裝Cloudera安裝搭建部署大數據集群，更是如此，要放在/opt下了。

 

 

 

 

 

bigdata@ubuntucmbigdata1:/opt$ pwd
/opt
bigdata@ubuntucmbigdata1:/opt$ ll
total 8
drwxr-xr-x  2 root root 4096  2月 18  2016 ./
drwxr-xr-x 23 root root 4096  6月 24 15:09 ../
bigdata@ubuntucmbigdata1:/opt$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt$ ll
total 505696
drwxr-xr-x  2 root root      4096  6月 25 18:37 ./
drwxr-xr-x 23 root root      4096  6月 24 15:09 ../
-rw-r--r--  1 root root 517818680  6月 25 18:22 cloudera-manager-trusty-cm5.6.0_amd64.tar.gz
bigdata@ubuntucmbigdata1:/opt$ sudo tar -zxvf cloudera-manager-trusty-cm5.6.0_amd64.tar.gz 

 

 

 

bigdata@ubuntucmbigdata1:/opt$ pwd
/opt
bigdata@ubuntucmbigdata1:/opt$ ll
total 505700
drwxr-xr-x  3 1106  592      4096  2月 12  2016 ./
drwxr-xr-x 23 root root      4096  6月 24 15:09 ../
-rw-r--r--  1 root root 517818680  6月 25 18:22 cloudera-manager-trusty-cm5.6.0_amd64.tar.gz
drwxr-xr-x  8 1106  592      4096  2月 12  2016 cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt$ sudo rm cloudera-manager-trusty-cm5.6.0_amd64.tar.gz 
bigdata@ubuntucmbigdata1:/opt$ sudo chown -R bigdata:bigdata cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt$ ll
total 12
drwxr-xr-x  3    1106     592 4096  6月 25 18:46 ./
drwxr-xr-x 23 root    root    4096  6月 24 15:09 ../
drwxr-xr-x  8 bigdata bigdata 4096  2月 12  2016 cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt$ 

 

 

 

 

 

　　添加mysql connector包到/opt/cm-5.6.0/share/cmf/lib下

 

 

bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/share/cmf/lib$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/share/cmf/lib$ ls
activation-1.1.1.jar                       commons-fileupload-1.2.2.jar       gson-2.2.jar                                   jaxb-impl-2.1.13.jar                     mx4j-3.0.1.jar                            spring-security-crypto-3.1.0.RELEASE.jar
agent-5.6.0.jar                            commons-httpclient-3.1.jar         guava-14.0.jar                                 jaxp-api-1.4.jar                         mx4j-tools-3.0.1.jar                      spring-security-ldap-3.1.0.RELEASE.jar
alarms-5.6.0.jar                           commons-io-2.4.jar                 hawtjni-runtime-1.8.jar                        jboss-logging-3.1.0.GA.jar               mysql-connector-java-5.1.32-bin.jar       spring-security-saml2-core-1.0.1.RELEASE.jar
alertpublisher-5.6.0.jar                   commons-lang-2.6.jar               headlamp-5.6.0.jar                             jcommander-1.30.jar                      nav-common-5.6.0.jar                      spring-security-web-3.1.0.RELEASE.jar

 

 

 

 

 　　下面，參考這篇博客，非常方便

Cloudera Manager 5和CDH5離線安裝

 

 

 

創建用戶(所有節點)

　　創建cloudera-scm用戶 
 　　即，在我這里就是，

 

bigdata@ubuntucmbigdata1:~$ sudo useradd --system --home=/opt/cm-5.3.3/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

　  我這里，僅拿ubuntucmbigdata1來說，其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一樣的。

 

 

 

 

 

 

 　　初始化CM5的數據庫數據庫(j僅ubuntucmbigdata1節點)

bigdata@ubuntucmbigdata1:~$ sudo /opt/cm-5.6.0/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -prootroot --scm-host localhost scm scm scm
+======================================================================+
|      Error: JAVA_HOME is not set and Java could not be found         |
+----------------------------------------------------------------------+
| Please download the latest Oracle JDK from the Oracle Java web site  |
|  > http://www.oracle.com/technetwork/java/javase/index.html <        |
|                                                                      |
| Cloudera Manager requires Java 1.6 or later.                         |
| NOTE: This script will find Oracle Java whether you install using    |
|       the binary or the RPM based installer.                         |
+======================================================================+
bigdata@ubuntucmbigdata1:~$ 

 

 

 

 

 

Agent 配置

 　　修改/opt/cm-5.6.0/etc/cloudera-scm-agent/config.ini中的server_host為主節點的主機名。 

bigdata@ubuntucmbigdata1:~$ sudo vim /opt/cm-5.6.0/etc/cloudera-scm-agent/config.ini

 

 

 

 

 

 

[General]
# Hostname of the CM server.
server_host=ubuntucmbigdata1

# Port that the CM server is listening on.
server_port=7182

 

 

 

 

 

 

Parcels離線包設置，用以安裝（僅主節點）

　　將下載好的

   CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel 
　  CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel.sha 
　　manifest.json

      這三個文件移至/opt/cloudera/parcel-repo/中,沒有這個目錄的話，手動創建。

 

 

 

 

 

bigdata@ubuntucmbigdata1:/opt$ pwd
/opt
bigdata@ubuntucmbigdata1:/opt$ ll
total 12
drwxr-xr-x  3    1106     592 4096  6月 25 18:46 ./
drwxr-xr-x 23 root    root    4096  6月 24 15:09 ../
drwxr-xr-x  8 bigdata bigdata 4096  2月 12  2016 cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt$ sudo mkdir -p /opt/cloudera/parcel-repo/
bigdata@ubuntucmbigdata1:/opt$ cd /opt/cloudera/parcel-repo/
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ pwd
/opt/cloudera/parcel-repo
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 8
drwxr-xr-x 2 root root 4096  6月 25 19:43 ./
drwxr-xr-x 3 root root 4096  6月 25 19:43 ../
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ 

 

 

 

 

 

 

bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ pwd
/opt/cloudera/parcel-repo
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 8
drwxr-xr-x 2 root root 4096  6月 25 19:43 ./
drwxr-xr-x 3 root root 4096  6月 25 19:43 ../
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo chown -R bigdata:bigdata /opt/cloudera/parcel-repo/
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 8
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:43 ./
drwxr-xr-x 3 root    root    4096  6月 25 19:43 ../
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 1497448
drwxr-xr-x 2 bigdata bigdata       4096  6月 25 19:48 ./
drwxr-xr-x 3 root    root          4096  6月 25 19:43 ../
-rw-r--r-- 1 root    root    1533315179  6月 25 18:31 CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel
-rw-r--r-- 1 root    root            41  6月 25 18:22 CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel.sha1
-rw-r--r-- 1 root    root         49913  6月 25 18:32 manifest.json
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo chown -R bigdata:bigdata /opt/cloudera/parcel-repo/*
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 1497448
drwxr-xr-x 2 bigdata bigdata       4096  6月 25 19:48 ./
drwxr-xr-x 3 root    root          4096  6月 25 19:43 ../
-rw-r--r-- 1 bigdata bigdata 1533315179  6月 25 18:31 CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel
-rw-r--r-- 1 bigdata bigdata         41  6月 25 18:22 CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel.sha1
-rw-r--r-- 1 bigdata bigdata      49913  6月 25 18:32 manifest.json
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ 

 

 

 

 

 

相關啟動腳本 
通過/opt/cm-5.3.3/etc/init.d/cloudera-scm-server start啟動服務端。 
通過/opt/cm-5.3.3/etc/init.d/cloudera-scm-agent start啟動Agent服務。 
我們啟動的其實是個service腳本，需要停止服務將以上的start參數改為stop就可以了，重啟是restart。 

 

bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo /opt/cm-5.6.0/etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:  * Couldn't start cloudera-scm-server

bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo /opt/cm-5.6.0/etc/init.d/cloudera-scm-agent start 
Starting cloudera-scm-agent:  * cloudera-scm-agent started

bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo /opt/cm-5.6.0/etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:  * Couldn't start cloudera-scm-server

bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ 

 

 　　查看日志

注：查看安裝目錄下log文件夾下cloudera-scm-agent/cloudera-scm-agent.out 看Agent 是否啟動成功 。 

 

bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ pwd
/opt/cm-5.6.0/log
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ l
cloudera-scm-agent/  cloudera-scm-server/  cmf/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ ll
total 20
drwxr-xr-x 5 bigdata bigdata 4096  2月 12  2016 ./
drwxr-xr-x 8 bigdata bigdata 4096  2月 12  2016 ../
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:53 cloudera-scm-agent/
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:52 cloudera-scm-server/
drwxr-xr-x 2 bigdata bigdata 4096  2月 12  2016 cmf/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ cd cloudera-scm-server/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-server$ ll
total 12
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:52 ./
drwxr-xr-x 5 bigdata bigdata 4096  2月 12  2016 ../
-rw-r--r-- 1 root    root     730  6月 25 19:53 cloudera-scm-server.out
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-server$ cat cloudera-scm-server.out 
+======================================================================+
|      Error: JAVA_HOME is not set and Java could not be found         |
+----------------------------------------------------------------------+
| Please download the latest Oracle JDK from the Oracle Java web site  |
|  > http://www.oracle.com/technetwork/java/javase/index.html <        |
|                                                                      |
| Cloudera Manager requires Java 1.6 or later.                         |
| NOTE: This script will find Oracle Java whether you install using    |
|       the binary or the RPM based installer.                         |
+======================================================================+

 　　查看日志，可以看出，JAVA_HOME，沒有設置。（是這種對ubuntu系統不適用，但是對centos系統是可以的。無解）

 

 

 

 

 

然后，發現，得用

 　　原來，在

在線和離線安裝Cloudera CDH 5.6.0

　　里

 

 

bigdata@ubuntucmbigdata1:~$ sudo add-apt-repository ppa:webupd8team/java

bigdata@ubuntucmbigdata1:~$ sudo add-apt-repository ppa:webupd8team/java
 Oracle Java (JDK) Installer (automatically downloads and installs Oracle JDK7 / JDK8 / JDK9). There are no actual Java files in this PPA.

Important -> Why Oracle Java 7 And 6 Installers No Longer Work: http://www.webupd8.org/2017/06/why-oracle-java-7-and-6-installers-no.html

More info (and Ubuntu installation instructions):
- for Oracle Java 7: http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html
- for Oracle Java 8: http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html

Debian installation instructions:
- Oracle Java 7: http://www.webupd8.org/2012/06/how-to-install-oracle-java-7-in-debian.html
- Oracle Java 8: http://www.webupd8.org/2014/03/how-to-install-oracle-java-8-in-debian.html

Oracle Java 9 (for both Ubuntu and Debian): http://www.webupd8.org/2015/02/install-oracle-java-9-in-ubuntu-linux.html

For JDK9, the PPA uses standard builds from: http://jdk.java.net/9/ (and not the Jigsaw builds!).

Important!!! For now, you should continue to use Java 8 because Oracle Java 9 is available as an early access release! You should only use Oracle Java 9 if you explicitly need it, because it may contain bugs and it might not include the latest security patches! Also, some Java options were removed in JDK9, so you may encounter issues with various Java apps. More information and installation instructions (Ubuntu / Linux Mint / Debian): http://www.webupd8.org/2015/02/install-oracle-java-9-in-ubuntu-linux.html
 More info: https://launchpad.net/~webupd8team/+archive/ubuntu/java
Press [ENTER] to continue or ctrl-c to cancel adding it （回車）

gpg: keyring `/tmp/tmpel2bk0b8/secring.gpg' created
gpg: keyring `/tmp/tmpel2bk0b8/pubring.gpg' created
gpg: requesting key EEA14886 from hkp server keyserver.ubuntu.com
gpg: /tmp/tmpel2bk0b8/trustdb.gpg: trustdb created
gpg: key EEA14886: public key "Launchpad VLC" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
OK
bigdata@ubuntucmbigdata1:~$ 

 　　 我這里，僅拿ubuntucmbigdata1來說，其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一樣的。

 

 

 

 

bigdata@ubuntucmbigdata1:~$ sudo apt-get update

 　　 我這里，僅拿ubuntucmbigdata1來說，其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一樣的。

 

 

 

 

 

 

 

 

 

bigdata@ubuntucmbigdata1:~$ sudo apt-get upgrade
...
Do you want to continue? [Y/n] y
...
*** sudoers (Y/I/N/O/D/Z) [default=N] ? y

 

 

 

 

 

 

 

bigdata@ubuntucmbigdata1:~$ sudo apt-get install oracle-java7-installer
[sudo] password for bigdata: 
Do you want to continue? [Y/n] y

 

 

 

 

 

 

 

 

 

 

bigdata@ubuntucmbigdata1:~$ sudo apt-get install oracle-java7-set-default

 

 

 

 

 

　　最后，我們需要如下，就可以了

bigdata@ubuntucmbigdata1:/etc/cloudera-scm-server$ dpkg --list | grep -i jdk
iF  oracle-java7-installer                                7u80+7u60arm-0~webupd8~1                            all          Oracle Java(TM) Development Kit (JDK) 7
iU  oracle-java7-set-default                              7u80+7u60arm-0~webupd8~1                            all          Set Oracle JDK 7 as default Java
bigdata@ubuntucmbigdata1:/etc/cloudera-scm-server$ 

 

 　　如果有多余的，如1.6版本的話，則

Ubuntu系統里如何正確卸載oracle-j2sdk1.6（圖文詳解）

 

 

 

 

 

 

 

 

 

 

 

bigdata@ubuntucmbigdata1:~$ sudo /opt/cm-5.6.0/etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:  * cloudera-scm-server started

bigdata@ubuntucmbigdata1:~$ 

 　　成功！

 

 

 

 

 

 

 

 　　查看cloudera-scm-agent的運行日志

bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ ll
total 20
drwxr-xr-x 5 bigdata bigdata 4096  2月 12  2016 ./
drwxr-xr-x 8 bigdata bigdata 4096  2月 12  2016 ../
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:53 cloudera-scm-agent/
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:52 cloudera-scm-server/
drwxr-xr-x 2 bigdata bigdata 4096  2月 12  2016 cmf/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ cd cloudera-scm-agent/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-agent$ ll
total 40
drwxr-xr-x 2 bigdata bigdata  4096  6月 25 19:53 ./
drwxr-xr-x 5 bigdata bigdata  4096  2月 12  2016 ../
-rw-r--r-- 1 root    root    14757  6月 25 19:57 cloudera-scm-agent.log
-rw-r--r-- 1 root    root      935  6月 25 19:53 cloudera-scm-agent.out
-rw-r--r-- 1 root    root      202  6月 25 19:53 cmf_listener.log
-rw-r--r-- 1 root    root      539  6月 25 19:53 supervisord.log
-rw-r--r-- 1 root    root      449  6月 25 19:53 supervisord.out
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-agent$ cat cloudera-scm-agent.out 
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     SCM Agent Version: 5.6.0
[25/Jun/2017 19:53:01 +0000] 9651 MainThread util         INFO     Generating Agent UUID for the first time: '/opt/cm-5.6.0/lib/cloudera-scm-agent/uuid'
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     Adding env vars that start with CMF_AGENT_
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     explicitly passing env var to supervisee: MGMT_HOME: /opt/cm-5.6.0/share/cmf
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     explicitly passing env var to supervisee: ROOT: /opt/cm-5.6.0/lib/cmf
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     Logging to /opt/cm-5.6.0/log/cloudera-scm-agent/cloudera-scm-agent.log
Error: could not find config file /opt/cm-5.6.0/run/cloudera-scm-agent/supervisor/supervisord.conf
For help, use /opt/cm-5.6.0/lib/cmf/agent/src/cmf/agent.py -h
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-agent$ 

 

 

 

 

　　當cloudera-scm-server、cloudera-scm-agent start啟動成功之后，可通過http://主節點IP:7180來訪問CM，用戶名密碼都為admin。剩下的就是一直點下一步和根據具體的需求來配置集群了。

 

 

 

 

 　　然后，到這一步，我的

cloudera manager的7180 web界面訪問不了的解決辦法（圖文詳解）

 

　　轉換成，在線安裝

　　如下

　　放在

/opt/softwares/cm-5.6.0

bigdata@ubuntucmbigdata1:~$ sudo mkdir -p /opt/softwares/cm-5.6.0
[sudo] password for bigdata: 
bigdata@ubuntucmbigdata1:~$ cd /opt/softwares/cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ pwd
/opt/softwares/cm-5.6.0
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 8
drwxr-xr-x 2 root root 4096  6月 25 22:19 ./
drwxr-xr-x 3 root root 4096  6月 25 22:19 ../
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chown -R bigdata:bigdata /opt/softwares/*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 8
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 22:19 ./
drwxr-xr-x 3 root    root    4096  6月 25 22:19 ../
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rw-r--r-- 1 root    root    517878  6月 25 22:17 cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chown -R bigdata:bigdata cloudera-manager-installer.bin 
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rw-r--r-- 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chmod +x ./cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rwxr-xr-x 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ 

 

 

bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ pwd
/etc/apt/sources.list.d
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 12
drwxr-xr-x 2 root root 4096  6月 25 20:26 ./
drwxr-xr-x 6 root root 4096  6月 25 20:26 ../
-rw-r--r-- 1 root root  136  6月 25 20:26 webupd8team-java-trusty.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo curl "http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/cloudera.list" -o /etc/apt/sources.list.d/cloudera_precise.list
[sudo] password for bigdata: 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   263  100   263    0     0    422      0 --:--:-- --:--:-- --:--:--   422
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 16
drwxr-xr-x 2 root root 4096  6月 25 22:26 ./
drwxr-xr-x 6 root root 4096  6月 25 20:26 ../
-rw-r--r-- 1 root root  263  6月 25 22:26 cloudera_precise.list
-rw-r--r-- 1 root root  136  6月 25 20:26 webupd8team-java-trusty.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo curl -s http://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -
OK
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo apt-get update

 

 

 　　然后，到這一步，我的

cloudera manager的7180 web界面訪問不了的解決辦法（圖文詳解）

 

 

 

 

 

 

 參考

Cloudera Manager5及CDH5安裝指導（終極在線安裝）

 

 

離線安裝CM5和CDH5.13完全教程

 https://blog.csdn.net/u013019431/article/details/78540683

 

 

 

 

 

 

 

 

 

 

歡迎大家，加入我的微信公眾號：大數據躺過的坑     免費給分享

 

 

 

同時，大家可以關注我的個人博客：

   http://www.cnblogs.com/zlslch/   和  http://www.cnblogs.com/lchzls/ 

 

　　人生苦短，我願分享。本公眾號將秉持活到老學到老學習無休止的交流分享開源精神，匯聚於互聯網和個人學習工作的精華干貨知識，一切來於互聯網，反饋回互聯網。
　　目前研究領域：大數據、機器學習、深度學習、人工智能、數據挖掘、數據分析。 語言涉及：Java、Scala、Python、Shell、Linux等 。同時還涉及平常所使用的手機、電腦和互聯網上的使用技巧、問題和實用軟件。 只要你一直關注和呆在群里，每天必須有收獲

 

       以及對應本平台的QQ群：161156071（大數據躺過的坑）