Hive擴展功能(一)--Parquet

軟件環境:

linux系統: CentOS6.7
Hadoop版本: 2.6.5
zookeeper版本: 3.4.8

##主機配置: ######一共m1, m2, m3這五部機, 每部主機的用戶名都為centos ``` 192.168.179.201: m1 192.168.179.202: m2 192.168.179.203: m3

m1: Zookeeper, Namenode, DataNode, ResourceManager, NodeManager, Master, Worker
m2: Zookeeper, Namenode, DataNode, ResourceManager, NodeManager, Worker
m3: Zookeeper, DataNode, NodeManager, Worker

#####資料:

官方資料:
https://cwiki.apache.org/confluence/display/Hive/Parquet

#####注意:

Hive擴展Parquet有以下兩種方案:(hive自0.13版本以后已自帶paruqet功能, 不必再擴展parquet功能)

---
<br>
####1.方案一: 
####(1) 下載Parquet擴展包
 通過yum下載parquet資源包 (此方案大多數情況下無法找到parquet, 原因是沒有配置YUM源, 如何配置YUM源請自行查找資料)

sudo yum -y install parquet

######(2)將下載好的擴展包拷貝到指定位置
將下載到`/usr/lib/parquet`目錄下的`jar`(除`javadoc.jar`和`sources.jar`外)拷貝到$HIVE_HOME/lib目錄下

cp /usr/lib/parquet/* $HIVE_HOME/lib

---
<br>
####2.方案二: 
####(1)打開```$HIVE_HOME/lib```目錄

cd /home/centos/soft/hive/lib

####(2)下載Parquet資源包
執行以下指令, 下載parquet資源包(注意:`parquet1.2.5`版本對應`Hive 0.12`版本,每個版本的Hive對應的Parquet版本不同, 具體Hive所對應的版本請自行下載源碼查看)

for f in parquet-avro parquet-cascading parquet-column parquet-common parquet-encoding parquet-generator parquet-hadoop parquet-hive parquet-pig parquet-pig-bundle parquet-scrooge parquet-test-hadoop2 parquet-thrift
do
curl -O https://oss.sonatype.org/service/local/repositories/releases/content/com/twitter/${f}/1.2.5/${f}-1.2.5.jar
curl -O https://oss.sonatype.org/service/local/repositories/releases/content/com/twitter/parquet-format/1.2.5/parquet-format-1.2.5.jar
done

---
<br>
<br>
<br>