Mac上Hive環境搭建

本文介紹在Mac上搭建Hive環境。
建議首先配置好Hadoop，搭建與配置可以參考我之前的博文Mac Hadoop的安裝與配置。
當然你也可以選擇使用Docker搭建環境，本文不作介紹。

安裝

對於MacOs，推薦使用HomeBrew安裝hive，一步到位。

$ brew install hive

創建元數據庫

Hive默認用derby作為元數據庫。這里我們用mysql來存儲元數據，下面作一些初始化

mysql> create database metastore;
mysql> create user 'hive'@'localhost' identified by '123456';
mysql> grant select,insert,update,delete,alter,create,index,references on metastore.* to 'hive'@'localhost';
mysql> flush privileges;

創建並修改配置

進入Hive的安裝目錄,創建hive-site.xml文件。（下面命令中hive路徑替換為自己實際安裝的路徑即可）

$ cd /usr/local/Cellar/hive/2.1.1/libexec/conf
$ cp hive-default.xml.template hive-site.xml

在配置文件中找到以下幾個property修改value

<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://localhost/metastore</value>
</property>

<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
</property>

<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive(填上述mysql中創建的用戶名)</value>
</property>

<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>123456(填上述mysql中創建的用戶密碼)</value>
</property>

<property>
  <name>hive.exec.local.scratchdir</name>
  <value>/tmp/hive</value>
</property>

<property>
  <name>hive.querylog.location</name>
  <value>/tmp/hive</value>
</property>

<property>
  <name>hive.downloaded.resources.dir</name>
  <value>/tmp/hive</value>
</property>

<property>
  <name>hive.server2.logging.operation.log.location</name>
  <value>/tmp/hive</value>
</property>

拷貝mysql-connector到hive

給Hive的lib目錄下拷貝一個mysql-connector

$ curl -L 'http://www.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.42.tar.gz/from/http://mysql.he.net/' | tar xz
$ cp mysql-connector-java-5.1.42/mysql-connector-java-5.1.42-bin.jar /usr/local/Cellar/hive/2.1.1/libexec/lib/

初始化庫

初始化一下metastore庫

$ schematool -initSchema -dbType mysql

成功之后我們可以看到hive用usr/local/Cellar/hive/2.1.1/libexec/scripts/metastore/upgrade/mysql/hive-schema-2.1.0.mysql.sql的初始化sql腳本在我們配置的元數據庫中初始化好了如下表：

mysql> show tables;
+---------------------------+
| Tables_in_metastore       |
+---------------------------+
| AUX_TABLE                 |
| BUCKETING_COLS            |
| CDS                       |
| COLUMNS_V2                |
| COMPACTION_QUEUE          |
| COMPLETED_COMPACTIONS     |
| COMPLETED_TXN_COMPONENTS  |
| DATABASE_PARAMS           |
| DBS                       |
| DB_PRIVS                  |
| DELEGATION_TOKENS         |
| FUNCS                     |
| FUNC_RU                   |
| GLOBAL_PRIVS              |
| HIVE_LOCKS                |
| IDXS                      |
| INDEX_PARAMS              |
| KEY_CONSTRAINTS           |
| MASTER_KEYS               |
| NEXT_COMPACTION_QUEUE_ID  |
| NEXT_LOCK_ID              |
| NEXT_TXN_ID               |
| NOTIFICATION_LOG          |
| NOTIFICATION_SEQUENCE     |
| NUCLEUS_TABLES            |
| PARTITIONS                |
| PARTITION_EVENTS          |
| PARTITION_KEYS            |
| PARTITION_KEY_VALS        |
| PARTITION_PARAMS          |
| PART_COL_PRIVS            |
| PART_COL_STATS            |
| PART_PRIVS                |
| ROLES                     |
| ROLE_MAP                  |
| SDS                       |
| SD_PARAMS                 |
| SEQUENCE_TABLE            |
| SERDES                    |
| SERDE_PARAMS              |
| SKEWED_COL_NAMES          |
| SKEWED_COL_VALUE_LOC_MAP  |
| SKEWED_STRING_LIST        |
| SKEWED_STRING_LIST_VALUES |
| SKEWED_VALUES             |
| SORT_COLS                 |
| TABLE_PARAMS              |
| TAB_COL_STATS             |
| TBLS                      |
| TBL_COL_PRIVS             |
| TBL_PRIVS                 |
| TXNS                      |
| TXN_COMPONENTS            |
| TYPES                     |
| TYPE_FIELDS               |
| VERSION                   |
| WRITE_SET                 |
+---------------------------+
57 rows in set (0.00 sec)

啟動hive

接下來，啟動hive

$ hive

參考

《Hive編程指南》