presto需要訪問kudu數據源,但是impala可以直接支持多數據庫存儲,但是presto不能原生支持,按照presto的官網設置了然而並不起作用。
官方文檔:
到官方github提問了,然后並沒有答復。怎么設置都是無法起作用,有點無從下手的。
查看了官方kudu文檔,無意中看到一個關於 Using the Hive Metastore with Kudu 的功能,文檔中大概提到:
With the Hive Metastore integration disabled, Kudu presents tables as a single flat namespace, with no hierarchy or concept of a database. Additionally, Kudu’s only restriction on table names is that they be a valid UTF-8 encoded string. When the HMS integration is enabled in Kudu, both of these properties change in order to match the HMS model: the table name must indicate the table’s membership of a Hive database, and table name identifiers (i.e. the table name and database name) are subject to the Hive table name identifier constraints.
大概意思就是在禁用Hive Metastore集成的情況下,Kudu將表顯示為單個平面名稱空間,沒有數據庫的層次結構或概念。但是我本地的環境是CDH,查看了CDH 的官方文檔在6.3.X的版本才能支持該功能。我本地環境 CDH6.2 版本,抱着試一試的心態。進行了如下操作:
- 先把CDH 6.2 升級到 6.3,具體過程參照 cdh版本升級(5.14 -> 6.2)
- 升級好了以后,按照官方文檔設置好。
Setup using Cloudera Manager
-
- When the Hive Metastore is configured with fine-grained authorization using Apache Sentry, and the Sentry HDFS Sync feature is enabled, the Kudu admin need to be able to access and modify directories that are created for Kudu by the HMS. This can be done by adding the Kudu admin user to the group of the Hive service users, e.g. by running the usermod -aG hive kudu command on the HMS nodes.
- Go to the Hive service.
- Click the Configuration tab.
- Select the Kudu Service with which the Hive Metastore will synchronize the Kudu tables.
居然可以了,但是到這里並沒有結束。
開啟了 HMS 涉及到原來表的升級,具體可以參照 hms 升級現有表的功能,以及這里面有個需要注意的細節就是,在開啟HMS之前必須先把presto 按照官方文檔設置好了
kudu.schema-emulation.enabled=true
and kudu.schema-emulation.prefix=
kudu生成了$schemas表以后再開啟HMS功能才能支持,如果順序不對,依然是無法成功的。