一、簡介
hue是一個開源的apache hadoop ui系統，由cloudear desktop演化而來，最后cloudera公司將其貢獻給了apache基金會的hadoop社區，它基於python框架django實現的。
通過使用hue，我們可以使用可視化的界面在web瀏覽器上與hadoop集群交互來分析處理數據，例如操作hdfs上的數據，運行MapReduce Job，查看HBase中的數據
二、安裝
（1）下載
http://archive.cloudera.com/cdh5/cdh/5/
從這里下載cdh5.11.1的最新版本的hue，3.9.0版本，到本地，並上傳到服務器，解壓縮到app目錄下
（2）必要的組件准備
需要先安裝好mysql數據庫
需要安裝好下面的組件
sudo yum install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libtidy libxml2-devel libxslt-devel openldap-devel python-devel python-simplejson sqlite-devel gmp-devel -y
（3）編譯
到hue的根目錄下，運行
make apps
三、配置
（1）hue基本配置
打開desktop/conf/hue.ini文件
[desktop]
# Set this to a random string, the longer the better.
# This is used for secure hashing in the session store.
secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
# Webserver listens on this address and port
http_host=hadoop001
http_port=8888
# Time zone name
time_zone=Asia/Shanghai
# Enable or disable Django debug mode.
django_debug_mode=false
# Enable or disable backtrace for server error
http_500_debug_mode=false
# Enable or disable memory profiling.
## memory_profiler=false
# Server email for internal error messages
## django_server_email='hue@localhost.localdomain'
# Email backend
## django_email_backend=django.core.mail.backends.smtp.EmailBackend
# Webserver runs as this user
server_user=hue
server_group=hue
# This should be the Hue admin and proxy user
## default_user=hue
# This should be the hadoop cluster admin
#default_hdfs_superuser=hadoop
（2）配置hue集成hadoop
首先hadoop里設置代理用戶，需要配置hadoop的core-site.xml
<property>
<name>hadoop.proxyuser.hue.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hue.groups</name>
<value>*</value>
</property>
加入這兩個屬性即可。
然后重啟hadoop集群
sbin/stop-dfs.sh
sbin/stop-yarn.sh
sbin/start-dfs.sh
sbin/start-yarn.sh
[hadoop]
# Configuration for HDFS NameNode
# ------------------------------------------------------------------------
[[hdfs_clusters]]
# HA support by using HttpFs
[[[default]]]
# Enter the filesystem uri
fs_defaultfs=hdfs://hadoop001:8020
# NameNode logical name.
## logical_name=
# Use WebHdfs/HttpFs as the communication mechanism.
# Domain should be the NameNode or HttpFs host.
# Default port is 14000 for HttpFs.
webhdfs_url=http://hadoop001:50070/webhdfs/v1
# Change this if your HDFS cluster is Kerberos-secured
## security_enabled=false
# Default umask for file and directory creation, specified in an octal value.
## umask=022
# Directory of the Hadoop configuration
hadoop_conf_dir=/home/hadoop/app/hadoop/etc/hadoop
# Configuration for YARN (MR2)
# ------------------------------------------------------------------------
[[yarn_clusters]]
[[[default]]]
# Enter the host on which you are running the ResourceManager
resourcemanager_host=hadoop002
# The port where the ResourceManager IPC listens on
resourcemanager_port=8032
# Whether to submit jobs to this cluster
submit_to=True
# Resource Manager logical name (required for HA)
## logical_name=
# Change this if your YARN cluster is Kerberos-secured
## security_enabled=false
# URL of the ResourceManager API
resourcemanager_api_url=http://hadoop002:8088
# URL of the ProxyServer API
proxy_api_url=http://hadoop002:8088
# URL of the HistoryServer API
history_server_api_url=http://hadoop002:19888
# In secure mode (HTTPS), if SSL certificates from Resource Manager's
# Rest Server have to be verified against certificate authority
## ssl_cert_ca_verify=False
# HA support by specifying multiple clusters
# e.g.
# [[[ha]]]
# Resource Manager logical name (required for HA)
## logical_name=my-rm-name
# Configuration for MapReduce (MR1)
（3）配置hue集成hive
[beeswax]
# Host where HiveServer2 is running.
# If Kerberos security is enabled, use fully-qualified domain name (FQDN).
hive_server_host=hadoop001
# Port where HiveServer2 Thrift server runs on.
hive_server_port=10000
# Hive configuration directory, where hive-site.xml is located
hive_conf_dir=/home/hadoop/app/hive/conf
# Timeout in seconds for thrift calls to Hive service
server_conn_timeout=120
（4）配置hue集成hbase
###########################################################################
# Settings to configure HBase Browser
###########################################################################
[hbase]
# Comma-separated list of HBase Thrift servers for clusters in the format of '(name|host:port)'.
# Use full hostname with security.
# If using Kerberos we assume GSSAPI SASL, not PLAIN.
hbase_clusters=(Cluster|hadoop004:9090)
# HBase configuration directory, where hbase-site.xml is located.
hbase_conf_dir=/home/hadoop/app/hbase/conf
# Hard limit of rows or columns per row fetched before truncating.
## truncate_limit = 500
# 'buffered' is the default of the HBase Thrift Server and supports security.
# 'framed' can be used to chunk up responses,
# which is useful when used in conjunction with the nonblocking server in Thrift.
## thrift_transport=buffered
（5）配置hue集成oozie
###########################################################################
# Settings to configure liboozie
###########################################################################
[liboozie]
# The URL where the Oozie service runs on. This is required in order for
# users to submit jobs. Empty value disables the config check.
oozie_url=http://hadoop004:11000/oozie
# Requires FQDN in oozie_url if enabled
## security_enabled=false
# Location on HDFS where the workflows/coordinator are deployed when submitted.
#remote_deployement_dir=/user/hue/oozie/deployments
（6）配置hue使用mysql數據庫
先配置一個mysql數據庫
[[database]]
# Database engine is typically one of:
# postgresql_psycopg2, mysql, sqlite3 or oracle.
#
# Note that for sqlite3, 'name', below is a path to the filename. For other backends, it is the database name
# Note for Oracle, options={"threaded":true} must be set in order to avoid crashes.
# Note for Oracle, you can use the Oracle Service Name by setting "host=" and "port=" and then "name=<host>:<port>/<service_name>".
# Note for MariaDB use the 'mysql' engine.
engine=mysql
host=hadoop004
port=3306
user=root
password=123456
name=hue
# conn_max_age option to make database connection persistent value in seconds
# https://docs.djangoproject.com/en/1.9/ref/databases/#persistent-connections
## conn_max_age=0
# Execute this script to produce the database password. This will be used when 'password' is not set.
## password_script=/path/script
## name=desktop/desktop.db
## options={}
# Database schema, to be used only when public schema is revoked in postgres
## schema=public
然后去mysql中，建立一個數據庫，就是上面的配置文件中配置的數據庫oozie
create database hue default character set utf8 default collate utf8_general_ci;
接着，進入hue的build/env/bin目錄下，運行：
./hue syncdb
./hue migrate
完畢之后，數據庫中的表其實默認引擎是MyISAM，需要調整成InnoDB，否則在hue的首頁會出現下面的警告信息：
PREFERRED_STORAGE_ENGINE We recommend MySQL InnoDB engine over MyISAM which does not support transactions.
那么運行下面的語句即可批量調整：
mysql -u root -proot -e \
"SELECT CONCAT('ALTER TABLE ',table_schema,'.',table_name,' engine=InnoDB;') \
FROM information_schema.tables \
WHERE engine = 'MyISAM' AND table_schema = 'hue';" \
| grep "ALTER TABLE hue" > /tmp/set_engine_innodb.ddl
mysql -u root -proot < /tmp/set_engine_innodb.ddl
（注意空格上面是一句，空格下面是一句，分兩次執行）
（7）啟動hue
先啟動hive的metastore服務，和hiveserver2服務
nohup hive --service metastore &
nohup hive --service hiveserver2 &
再啟動hue
nohup /home/hadoop/app/hue/build/env/bin/supervisor &
（6）訪問hue
http://hadoop004:8888
四、可能會遇到的問題
Failed to contact an active Resource Manager: YARN RM returned a failed response: { "RemoteException" : { "message" : "User: hue is not allowed to impersonate admin", "exception" : "AuthorizationException", "javaClassName" : "org.apache.hadoop.security.authorize.AuthorizationException" } } (error 403)
這個問題是hadoop的core-site.xml配置的代理的用戶和hue配置文件的用戶不一致造成的。
比如，hadoop的core-site.xml是這樣配置的
<property>
<name>hadoop.proxyuser.hue.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hue.groups</name>
<value>*</value>
</property>
代理用戶是hue。
而hue里面是這樣配置的：
# Webserver runs as this user
#server_user=hue
#server_group=hue
需要把server_user和server_group設置成hue，即可