轉:http://blog.itpub.net/30089851/viewspace-2121221/
微視頻鏈接: Apache Kylin初識
1.版本(當前實驗版本組合,版本一定要兼容,否則各種問題)
Hadoop2.7.2+HBase1.1.5+Hive2.0.0
kylin-1.5.2.1 (apache-kylin-1.5.1-HBase1.1.3-bin.tar.gz)
2.Hadoop環境編譯以支持Snappy解壓縮庫
重新編譯hadoop-2.7.2-src的native以支持Snappy解壓壓縮庫
3.環境准備
Hadoop-2.7.2+zookeeper-3.4.6完全分布式環境搭建(HDFS、YARN HA)
Hadoop2.7實戰v1.0之HBase1.1.5 HA分布式搭建
配置Hadoop2.7.2和Hbase1.1.5支持Snappy解壓壓縮庫
Hadoop2.7實戰v1.0之Hive-2.0.0+MySQL本地模式安裝
4. 配置Hadoop2.7.2和Hbase1.1.5支持Snappy解壓壓縮庫
5. 主機規划
IP Host 安裝軟件 進程
172.16.101.55 sht-sgmhadoopnn-01 apache-kylin-1.5.1-HBase1.1.3-bin.tar.gz RunJar
6. 添加hive_dependency和KYLIN_HOME環境變量
點擊(此處)折疊或打開
- [root@sht-sgmhadoopnn-01 ~]# vi /etc/profile
- export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
- export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
- export HADOOP_HOME=/hadoop/hadoop
- export HBASE_HOME=/hadoop/hbase
- export ZOOKEEPER_HOME=/hadoop/zookeeper
- export HIVE_HOME=/hadoop/hive
- export hive_dependency=/hadoop/hive/conf:/hadoop/hive/lib/*:/hadoop/hive/hcatalog/share/hcatalog/hive-hcatalog-core-2.0.0.jar
- export KYLIN_HOME=/hadoop/kylin
- export PATH=.:$HADOOP_HOME/bin:$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$KYLIN_HOME/bin:$PATH
- "/etc/profile" 99L, 2592C written
- [root@sht-sgmhadoopnn-01 ~]# source /etc/profile
7.同步/etc/profile到集群其他節點,生效
8.下載安裝配置kylin
點擊(此處)折疊或打開
- # 在sht-sgmhadoopnn-01上執行以下命令
- [root@sht-sgmhadoopnn-01 ~]# cd /hadoop
- [root@sht-sgmhadoopnn-01 hadoop]# wget https://dist.apache.org/repos/dist/release/kylin/apache-kylin-1.5.2.1/apache-kylin-1.5.2.1-HBase1.x-bin.tar.gz
- [root@sht-sgmhadoopnn-01 hadoop]# tar -zxvf apache-kylin-1.5.1-HBase1.1.3-bin.tar.gz
- [root@sht-sgmhadoopnn-01 hadoop]# ln -s /hadoop/apache-kylin-1.5.2.1-bin /hadoop/kylin
- [root@sht-sgmhadoopnn-01 hadoop]# vi /hadoop/kylin/bin/kylin.sh
- export KYLIN_HOME=/hadoop/kylin # 改成絕對路徑
- export HBASE_CLASSPATH_PREFIX=${tomcat_root}/bin/bootstrap.jar:${tomcat_root}/bin/tomcat-juli.jar:${tomcat_root}/lib/*:$hive_dependency:$HBASE_CLASSPATH_PREFIX #在路徑中添加$hive_dependency
9.修改kylin.properties
點擊(此處)折疊或打開
- [root@sht-sgmhadoopnn-01 conf]# vi $KYLIN_HOME/conf/kylin.properties
- ######修改
- # List of web servers in use, this enables one web server instance to sync up with other servers.
- kylin.rest.servers=sht-sgmhadoopnn-01:7070
- ######新增
- kylin.job.jar=$KYLIN_HOME/lib/kylin-job-1.5.2.1.jar
- kylin.coprocessor.local.jar=$KYLIN_HOME /lib/kylin-coprocessor-1.5.2.1.jar
10.啟動kylin
[root@sht-sgmhadoopnn-01 kylin]# $KYLIN_HOME/bin/kylin.sh start
.........
.........
KYLIN_DEBUG_SETTINGS is not set, will not enable remote debuging
KYLIN_LD_LIBRARY_SETTINGS is not set, Usually it's okay unless you want to specify your own native path
A new Kylin instance is started by root, stop it using "kylin.sh stop"
Please visit <a href="http://http://:7070/kylin
You can check the log at<span "=""> /hadoop/kylin/logs/kylin.log<a href="http://http://:7070/kylin
You can check the log at<span "="">
###http://172.16.101.55:7070/kylin 賬號: ADMIN 密碼:KYLIN
11.檢查kylin是否啟動成功
點擊(此處)折疊或打開
- [root@sht-sgmhadoopnn-01 kylin]# netstat -nlp |grep 7070
- tcp 0 0 0.0.0.0:7070 0.0.0.0:* LISTEN 30939/java
- [root@sht-sgmhadoopnn-01 kylin]#
12.導入官網測試案例
點擊(此處)折疊或打開
- [root@sht-sgmhadoopnn-01 kylin]# $KYLIN_HOME/bin/kylin.sh stop
- [root@sht-sgmhadoopnn-01 kylin]# $KYLIN_HOME/bin/sample.sh
- [root@sht-sgmhadoopnn-01 kylin]# $KYLIN_HOME/bin/kylin.sh start
1>.選中'kylin_sales_cube'示例立方體,點擊'Actions'->'Build',選擇一個截止日期,本試驗中選擇的是'2016-01-10'
(具體小時,分,秒隨便選,因為最終hive sql的語句類似 WHERE (KYLIN_SALES.PART_DT >= '2012-01-01' AND KYLIN_SALES.PART_DT < '2016-01-10');)
2>.在'Monitor'標簽中通過刷新頁面檢查進度條,直到100%
3>.在'Insight'標簽中執行下面的SQL查詢:
select part_dt, sum(price) as total_selled, count(distinct seller_id) as sellers from kylin_sales group by part_dt order by part_dt;
###耗時2.87s
4>.在hive中執行同一個SQL查詢,驗證kylin的查詢結果(會開啟MapReduce Job計算)
###耗時65.205s
查詢執行和結果如圖所示
Build成功后,hive中建立了3+n個表,如圖所示(3個官網案例hive表,n個build的hive表)
Build成功后,hbase中建立了1+n個表,如圖所示(1個元數據表,n個build的hbase表)