前言：該文章是基於 Hadoop2.7.6_01_部署 進行的。
1. Hive基本概念
1.1. 什么是Hive
Hive是基於Hadoop的一個數據倉庫工具，可以將結構化的數據文件映射為一張數據庫表，並提供類SQL查詢功能。
1.2. 為什么使用Hive
直接使用hadoop所面臨的問題
人員學習成本太高
項目周期要求太短
MapReduce實現復雜查詢邏輯開發難度太大
為什么要使用Hive
操作接口采用類SQL語法，提供快速開發的能力。
避免了去寫MapReduce，減少開發人員的學習成本。
擴展功能很方便。
1.3. Hive的特點
可擴展
Hive可以自由的擴展集群的規模，一般情況下不需要重啟服務。
延展性
Hive支持用戶自定義函數，用戶可以根據自己的需求來實現自己的函數。
容錯
良好的容錯性，節點出現問題SQL仍可完成執行。
1.4. 架構圖
1.5. 基本組成
用戶接口：包括 CLI、JDBC/ODBC、WebGUI。
元數據存儲：通常是存儲在關系數據庫如 mysql , derby中。
解釋器、編譯器、優化器、執行器。
1.6. 各組件的基本功能
用戶接口主要由三個：CLI、JDBC/ODBC和WebGUI。其中，CLI為shell命令行；JDBC/ODBC是Hive的JAVA實現，與傳統數據庫JDBC類似；WebGUI是通過瀏覽器訪問Hive。
元數據存儲：Hive 將元數據存儲在數據庫中。Hive 中的元數據包括表的名字，表的列和分區及其屬性，表的屬性（是否為外部表等），表的數據所在目錄等。
解釋器、編譯器、優化器完成 HQL 查詢語句從詞法分析、語法分析、編譯、優化以及查詢計划的生成。生成的查詢計划存儲在 HDFS 中，並在隨后有 MapReduce 調用執行。
1.7. Hive的數據存儲
1、Hive中所有的數據都存儲在 HDFS 中，沒有專門的數據存儲格式（可支持Text，SequenceFile，ParquetFile，RCFILE等）
2、只需要在創建表的時候告訴 Hive 數據中的列分隔符和行分隔符，Hive 就可以解析數據。
3、Hive 中包含以下數據模型：DB、Table，External Table，Partition，Bucket。
db：在hdfs中表現為${hive.metastore.warehouse.dir}目錄下一個文件夾
table：在hdfs中表現所屬db目錄下一個文件夾
external table：外部表, 與table類似，不過其數據存放位置可以在任意指定路徑
普通表: 刪除表后, hdfs上的文件都刪了
External外部表刪除后, hdfs上的文件沒有刪除, 只是把數據庫中的元數據【描述細信息】刪除了
partition：在hdfs中表現為table目錄下的子目錄
bucket：桶, 在hdfs中表現為同一個表目錄下根據hash散列之后的多個文件, 會根據不同的文件把數據放到不同的文件中
2. 主機規划
| 主機名稱 |
外網IP |
內網IP |
操作系統 |
安裝軟件 |
| mini01 |
10.0.0.11 |
172.16.1.11 |
CentOS 7.4 |
Hadoop 【NameNode SecondaryNameNode】、Hive |
| mini02 |
10.0.0.12 |
172.16.1.12 |
CentOS 7.4 |
Hadoop 【ResourceManager】 |
| mini03 |
10.0.0.13 |
172.16.1.13 |
CentOS 7.4 |
Hadoop 【DataNode NodeManager】、Mariadb |
| mini04 |
10.0.0.14 |
172.16.1.14 |
CentOS 7.4 |
Hadoop 【DataNode NodeManager】 |
| mini05 |
10.0.0.15 |
172.16.1.15 |
CentOS 7.4 |
Hadoop 【DataNode NodeManager】 |
Linux添加hosts信息，保證每台都可以相互ping通
1 [yun@mini03 ~]$ cat /etc/hosts 2 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 3 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 4 5 10.0.0.11 mini01 6 10.0.0.12 mini02 7 10.0.0.13 mini03 8 10.0.0.14 mini04 9 10.0.0.15 mini05
Windows的hosts文件修改
1 # 文件位置C:\Windows\System32\drivers\etc 在hosts中追加如下內容 2 ……………… 3 10.0.0.11 mini01 4 10.0.0.12 mini02 5 10.0.0.13 mini03 6 10.0.0.14 mini04 7 10.0.0.15 mini05
3. Mariadb安裝與配置
3.1. 數據庫安裝
1 # 在mini03 機器安裝mariadb 2 [root@mini03 ~]# cat /etc/redhat-release # 也可以使用其他版本 3 CentOS Linux release 7.4.1708 (Core) 4 [root@mini03 ~]# yum install -y mariadb mariadb-server # CentOS7的mysql數據庫為mariadb 5 ……………… 6 [root@mini03 ~]# systemctl status mariadb.service 7 ● mariadb.service - MariaDB database server 8 Loaded: loaded (/usr/lib/systemd/system/mariadb.service; disabled; vendor preset: disabled) 9 Active: inactive (dead) 10 [root@mini03 ~]# systemctl enable mariadb.service # 加入開機自啟動 11 Created symlink from /etc/systemd/system/multi-user.target.wants/mariadb.service to /usr/lib/systemd/system/mariadb.service. 12 [root@mini03 ~]# systemctl start mariadb.service # 啟動mariadb 13 [root@mini03 ~]# systemctl status mariadb.service # 查看mariadb服務狀態 14 ● mariadb.service - MariaDB database server 15 Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled) 16 Active: active (running) since Mon 2018-07-02 23:36:37 CST; 2s ago 17 Process: 2072 ExecStartPost=/usr/libexec/mariadb-wait-ready $MAINPID (code=exited, status=0/SUCCESS) 18 Process: 1992 ExecStartPre=/usr/libexec/mariadb-prepare-db-dir %n (code=exited, status=0/SUCCESS) 19 Main PID: 2071 (mysqld_safe) 20 ………………
3.2. 數據庫配置查看
1 # 進入數據庫 2 [root@mini03 ~]# mysql 3 Welcome to the MariaDB monitor. Commands end with ; or \g. 4 Your MariaDB connection id is 2 5 Server version: 5.5.56-MariaDB MariaDB Server 6 7 Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others. 8 9 Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. 10 11 # 查看數據庫版本 12 MariaDB [(none)]> select version(); 13 +----------------+ 14 | version() | 15 +----------------+ 16 | 5.5.56-MariaDB | 17 +----------------+ 18 1 row in set (0.00 sec) 19 20 # 支持哪些字符集 21 MariaDB [(none)]> show CHARACTER SET; 22 ## 字符集 描述 默認校對規則 最大長度 23 +----------+-----------------------------+---------------------+--------+ 24 | Charset | Description | Default collation | Maxlen | 25 +----------+-----------------------------+---------------------+--------+ 26 | big5 | Big5 Traditional Chinese | big5_chinese_ci | 2 | 27 | dec8 | DEC West European | dec8_swedish_ci | 1 | 28 | cp850 | DOS West European | cp850_general_ci | 1 | 29 | hp8 | HP West European | hp8_english_ci | 1 | 30 | koi8r | KOI8-R Relcom Russian | koi8r_general_ci | 1 | 31 | latin1 | cp1252 West European | latin1_swedish_ci | 1 | 32 | latin2 | ISO 8859-2 Central European | latin2_general_ci | 1 | 33 | swe7 | 7bit Swedish | swe7_swedish_ci | 1 | 34 | ascii | US ASCII | ascii_general_ci | 1 | 35 | ujis | EUC-JP Japanese | ujis_japanese_ci | 3 | 36 | sjis | Shift-JIS Japanese | sjis_japanese_ci | 2 | 37 | hebrew | ISO 8859-8 Hebrew | hebrew_general_ci | 1 | 38 | tis620 | TIS620 Thai | tis620_thai_ci | 1 | 39 | euckr | EUC-KR Korean | euckr_korean_ci | 2 | 40 | koi8u | KOI8-U Ukrainian | koi8u_general_ci | 1 | 41 | gb2312 | GB2312 Simplified Chinese | gb2312_chinese_ci | 2 | 42 | greek | ISO 8859-7 Greek | greek_general_ci | 1 | 43 | cp1250 | Windows Central European | cp1250_general_ci | 1 | 44 | gbk | GBK Simplified Chinese | gbk_chinese_ci | 2 | 45 | latin5 | ISO 8859-9 Turkish | latin5_turkish_ci | 1 | 46 | armscii8 | ARMSCII-8 Armenian | armscii8_general_ci | 1 | 47 | utf8 | UTF-8 Unicode | utf8_general_ci | 3 | 48 | ucs2 | UCS-2 Unicode | ucs2_general_ci | 2 | 49 | cp866 | DOS Russian | cp866_general_ci | 1 | 50 | keybcs2 | DOS Kamenicky Czech-Slovak | keybcs2_general_ci | 1 | 51 | macce | Mac Central European | macce_general_ci | 1 | 52 | macroman | Mac West European | macroman_general_ci | 1 | 53 | cp852 | DOS Central European | cp852_general_ci | 1 | 54 | latin7 | ISO 8859-13 Baltic | latin7_general_ci | 1 | 55 | utf8mb4 | UTF-8 Unicode | utf8mb4_general_ci | 4 | 56 | cp1251 | Windows Cyrillic | cp1251_general_ci | 1 | 57 | utf16 | UTF-16 Unicode | utf16_general_ci | 4 | 58 | cp1256 | Windows Arabic | cp1256_general_ci | 1 | 59 | cp1257 | Windows Baltic | cp1257_general_ci | 1 | 60 | utf32 | UTF-32 Unicode | utf32_general_ci | 4 | 61 | binary | Binary pseudo charset | binary | 1 | 62 | geostd8 | GEOSTD8 Georgian | geostd8_general_ci | 1 | 63 | cp932 | SJIS for Windows Japanese | cp932_japanese_ci | 2 | 64 | eucjpms | UJIS for Windows Japanese | eucjpms_japanese_ci | 3 | 65 +----------+-----------------------------+---------------------+--------+ 66 39 rows in set (0.00 sec) 67 68 # 當前數據庫默認字符集 69 MariaDB [(none)]> show variables like '%character_set%'; 70 +--------------------------+----------------------------+ 71 | Variable_name | Value | 72 +--------------------------+----------------------------+ 73 | character_set_client | utf8 | ## 客戶端來源數據使用的字符集 74 | character_set_connection | utf8 | ## 連接層字符集 75 | character_set_database | latin1 | ## 當前選中數據庫的默認字符集 76 | character_set_filesystem | binary | 77 | character_set_results | utf8 | ## 查詢結果返回字符集 78 | character_set_server | latin1 | ## 默認的內部操作字符集【服務端（數據庫）字符】 79 | character_set_system | utf8 | ## 系統元數據(字段名等)字符集【Linux系統字符集】 80 | character_sets_dir | /usr/share/mysql/charsets/ | 81 +--------------------------+----------------------------+ 82 8 rows in set (0.00 sec)
注意：最好不要修改數據庫的字符集，因為hive建庫時需要使用latin1字符集。
3.3. 建庫並授權
1 # 這里沒有創建數據庫hive，該數據庫可以等hive啟動時自行創建 2 MariaDB [(none)]> show databases; 3 +--------------------+ 4 | Database | 5 +--------------------+ 6 | information_schema | 7 | mysql | 8 | performance_schema | 9 | test | 10 +--------------------+ 11 4 rows in set (0.00 sec) 12 13 # 創建用戶與授權用於遠程訪問 格式為：mysql -hmini03 -uhive -phive 或者 mysql -h10.0.0.13 -uhive -phive 14 MariaDB [(none)]> grant all on hive.* to hive@'%' identified by 'hive'; 15 Query OK, 0 rows affected (0.00 sec) 16 17 # 創建用戶與授權用於本地訪問 格式為：mysql -hmini03 -uhive -phive 或者 mysql -h10.0.0.13 -uhive -phive 18 MariaDB [(none)]> grant all on hive.* to hive@'mini03' identified by 'hive'; 19 Query OK, 0 rows affected (0.00 sec) 20 21 # 刷新權限信息 22 MariaDB [(none)]> flush privileges; 23 Query OK, 0 rows affected (0.00 sec) 24 25 MariaDB [(none)]> show grants for hive@'%' ; 26 +-----------------------------------------------------------------------------------------------------+ 27 | Grants for hive@% | 28 +-----------------------------------------------------------------------------------------------------+ 29 | GRANT USAGE ON *.* TO 'hive'@'%' IDENTIFIED BY PASSWORD '*4DF1D66463C18D44E3B001A8FB1BBFBEA13E27FC' | 30 | GRANT ALL PRIVILEGES ON `hive`.* TO 'hive'@'%' | 31 +-----------------------------------------------------------------------------------------------------+ 32 2 rows in set (0.00 sec) 33 34 # 查看用戶表信息 35 MariaDB [(none)]> select user,host from mysql.user; 36 +------+-----------+ 37 | user | host | 38 +------+-----------+ 39 | hive | % | 40 | root | 127.0.0.1 | 41 | root | ::1 | 42 | | localhost | 43 | root | localhost | 44 | | mini03 | 45 | hive | mini03 | 46 | root | mini03 | 47 +------+-----------+ 48 8 rows in set (0.00 sec)
4. hive-1.2.1的安裝與配置
4.1. 軟件安裝
1 [yun@mini01 software]$ pwd 2 /app/software 3 [yun@mini01 software]$ ll 4 total 421924 5 -rw-r--r-- 1 yun yun 92834839 May 14 00:23 apache-hive-1.2.1-bin.tar.gz 6 -rw-r--r-- 1 yun yun 198811365 Jun 8 16:36 CentOS7.4_hadoop-2.7.6.tar.gz 7 -rw-r--r-- 1 yun yun 1004838 Jul 2 22:59 mysql-connector-java-5.1.46.jar 8 drwxrwxr-x 2 yun yun 19 Jun 18 10:48 zhangliang 9 [yun@mini01 software]$ 10 [yun@mini01 software]$ tar xf apache-hive-1.2.1-bin.tar.gz 11 [yun@mini01 software]$ mv apache-hive-1.2.1-bin /app/hive-1.2.1 12 [yun@mini01 software]$ cd /app/ 13 [yun@mini01 ~]$ ln -s hive-1.2.1/ hive 14 [yun@mini01 ~]$ ll 15 total 8 16 drwxr-xr-x 3 yun yun 23 May 26 14:30 bigdata 17 lrwxrwxrwx 1 yun yun 13 Jun 8 20:43 hadoop -> hadoop-2.7.6/ 18 drwxr-xr-x 11 yun yun 172 Jun 9 17:54 hadoop-2.7.6 19 lrwxrwxrwx 1 yun yun 11 Jul 3 14:23 hive -> hive-1.2.1/ 20 drwxrwxr-x 8 yun yun 159 Jul 3 14:22 hive-1.2.1 21 lrwxrwxrwx 1 yun yun 12 May 26 11:18 jdk -> jdk1.8.0_112 22 drwxr-xr-x 8 yun yun 255 Sep 23 2016 jdk1.8.0_112
4.2. 環境變量
1 # 使用root權限 2 [root@mini01 profile.d]# pwd 3 /etc/profile.d 4 [root@mini01 profile.d]# cat hive.sh 5 export HIVE_HOME="/app/hive" 6 export PATH=$HIVE_HOME/bin:$PATH
4.3. 配置修改
1 [yun@mini01 conf]$ pwd 2 /app/hive/conf 3 [yun@mini01 conf]$ vim hive-site.xml # 默認沒有，新建該配置文件 4 <configuration> 5 <property> 6 <name>javax.jdo.option.ConnectionURL</name> 7 <!-- 如果hive庫不存在，則創建 --> 8 <value>jdbc:mysql://mini03:3306/hive?createDatabaseIfNotExist=true</value> 9 <description>JDBC connect string for a JDBC metastore</description> 10 </property> 11 12 <property> 13 <name>javax.jdo.option.ConnectionDriverName</name> 14 <value>com.mysql.jdbc.Driver</value> 15 <description>Driver class name for a JDBC metastore</description> 16 </property> 17 18 <property> 19 <name>javax.jdo.option.ConnectionUserName</name> 20 <value>hive</value> 21 <description>username to use against metastore database</description> 22 </property> 23 24 <property> 25 <name>javax.jdo.option.ConnectionPassword</name> 26 <value>hive</value> 27 <description>password to use against metastore database</description> 28 </property> 29 30 <!-- 顯示當前使用的數據庫 --> 31 <property> 32 <name>hive.cli.print.current.db</name> 33 <value>true</value> 34 <description>Whether to include the current database in the Hive prompt.</description> 35 </property> 36 37 </configuration>
4.4. 啟動
1 # 添加了環境變量 2 [yun@mini01 ~]$ hive 3 4 Logging initialized using configuration in jar:file:/app/hive-1.2.1/lib/hive-common-1.2.1.jar!/hive-log4j.properties 5 Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient 6 at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522) 7 at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677) 8 at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621) 9 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 10 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 11 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 12 at java.lang.reflect.Method.invoke(Method.java:498) 13 at org.apache.hadoop.util.RunJar.run(RunJar.java:221) 14 at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 15 Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient 16 at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523) 17 at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86) 18 at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132) 19 at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) 20 at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005) 21 at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024) 22 at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503) 23 ... 8 more 24 Caused by: java.lang.reflect.InvocationTargetException 25 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 26 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 27 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 28 at java.lang.reflect.Constructor.newInstance(Constructor.java:423) 29 at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521) 30 ... 14 more 31 Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional connection factory 32 NestedThrowables: 33 java.lang.reflect.InvocationTargetException 34 at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:587) 35 at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788) 36 at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333) 37 at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202) 38 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 39 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 40 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 41 at java.lang.reflect.Method.invoke(Method.java:498) 42 at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965) 43 at java.security.AccessController.doPrivileged(Native Method) 44 at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960) 45 at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166) 46 at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808) 47 at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701) 48 at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365) 49 at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394) 50 at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291) 51 at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258) 52 at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) 53 at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) 54 at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:57) 55 at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66) 56 at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:593) 57 at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:571) 58 at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624) 59 at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461) 60 at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66) 61 at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72) 62 at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762) 63 at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199) 64 at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) 65 ... 19 more 66 Caused by: java.lang.reflect.InvocationTargetException 67 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 68 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 69 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 70 at java.lang.reflect.Constructor.newInstance(Constructor.java:423) 71 at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631) 72 at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:325) 73 at org.datanucleus.store.AbstractStoreManager.registerConnectionFactory(AbstractStoreManager.java:282) 74 at org.datanucleus.store.AbstractStoreManager.<init>(AbstractStoreManager.java:240) 75 at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:286) 76 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 77 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 78 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 79 at java.lang.reflect.Constructor.newInstance(Constructor.java:423) 80 at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631) 81 at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301) 82 at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187) 83 at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356) 84 at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775) 85 ... 48 more 86 Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "BONECP" plugin to create a ConnectionPool gave an error : The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver. 87 at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:259) 88 at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:131) 89 at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:85) 90 ... 66 more 91 Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException: The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver. 92 at org.datanucleus.store.rdbms.connectionpool.AbstractConnectionPoolFactory.loadDriver(AbstractConnectionPoolFactory.java:58) 93 at org.datanucleus.store.rdbms.connectionpool.BoneCPConnectionPoolFactory.createConnectionPool(BoneCPConnectionPoolFactory.java:54) 94 at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:238) 95 ... 68 more 96 97 # 原因： 缺少連接數據庫包，把 /app/software/mysql-connector-java-5.1.46.jar 拷貝過去 98 [yun@mini01 ~]$ cp -a /app/software/mysql-connector-java-5.1.46.jar /app/hive/lib/ 99 [yun@mini01 ~]$ hive # 再次啟動hive 100 101 Logging initialized using configuration in jar:file:/app/hive-1.2.1/lib/hive-common-1.2.1.jar!/hive-log4j.properties 102 # 啟動成功 103 hive> show databases; 104 OK 105 default 106 Time taken: 0.774 seconds, Fetched: 1 row(s)
4.4.1. 通過Navicat查看，hive庫已經被創建
4.4.2. 在mini03查看MySQL建表語句
1 [root@mini03 ~]# mysql 2 Welcome to the MariaDB monitor. Commands end with ; or \g. 3 Your MariaDB connection id is 20 4 Server version: 5.5.56-MariaDB MariaDB Server 5 6 Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others. 7 8 Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. 9 10 MariaDB [(none)]> 11 MariaDB [(none)]> show databases; 12 +--------------------+ 13 | Database | 14 +--------------------+ 15 | information_schema | 16 | hive | 17 | mysql | 18 | performance_schema | 19 | test | 20 +--------------------+ 21 5 rows in set (0.00 sec) 22 23 MariaDB [(none)]> show create database hive; 24 # 注意字符集為 latin1，不然使用時可能報錯★★★ 25 +----------+-----------------------------------------------------------------+ 26 | Database | Create Database | 27 +----------+-----------------------------------------------------------------+ 28 | hive | CREATE DATABASE `hive` /*!40100 DEFAULT CHARACTER SET latin1 */ | 29 +----------+-----------------------------------------------------------------+ 30 1 row in set (0.00 sec)