如果一個表要實現update和delete功能,該表就必須支持ACID,而支持ACID,就必須滿足以下條件:
1、表的存儲格式必須是ORC(STORED AS ORC);
2、表必須進行分桶(CLUSTERED BY (col_name, col_name, ...) INTO num_buckets BUCKETS);
3、Table property中參數transactional必須設定為True(tblproperties('transactional'='true'));
4、以下配置項必須被設定:
Hive->配置->類別->高級
Client端:hive-site.xml 的 Hive 客戶端高級配置代碼段(安全閥)
hive.support.concurrency – true
hive.enforce.bucketing – true
hive.exec.dynamic.partition.mode – nonstrict
hive.txn.manager – org.apache.hadoop.hive.ql.lockmgr.DbTxnManager
服務端:hive-site.xml 的 Hive 服務高級配置代碼段(安全閥)
hive.compactor.initiator.on – true
hive.compactor.worker.threads – 1
hive.txn.manager – org.apache.hadoop.hive.ql.lockmgr.DbTxnManager
hive>set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
創建表
create table t1(id int, name string)
clustered by (id) into 8 buckets
stored as orc TBLPROPERTIES ('transactional'='true');
create table test_trancaction
(user_id Int,name String)
clustered by (user_id) into 3 buckets
stored as orc TBLPROPERTIES ('transactional'='true');
hive> create table test_insert_test(id int,name string) row format delimited fields TERMINATED BY ','; ---臨時表
hive> LOAD DATA LOCAL INPATH '/data/test.txt' OVERWRITE INTO TABLE test_insert_test;
hive> select * from test_insert_test;
OK
1,jerrick
2,tom
3,jerry
4,lily
5,hanmei
6,limlei
7,lucky
hive>insert into test_trancaction select * from test_insert_test;
FAILED: SemanticException [Error 10265]: This command is not allowed on an ACID table merge_data.transactions with a non-ACID transaction manager. Failed
hive>set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
hive>update test_trancaction set name='ccx' where user_id=2;