hive實現update和delete功能


如果一個表要實現update和delete功能,該表就必須支持ACID,而支持ACID,就必須滿足以下條件:
1、表的存儲格式必須是ORC(STORED AS ORC);
2、表必須進行分桶(CLUSTERED BY (col_name, col_name, ...)  INTO num_buckets BUCKETS);
3、Table property中參數transactional必須設定為True(tblproperties('transactional'='true'));
4、以下配置項必須被設定:
Hive->配置->類別->高級
Client端:hive-site.xml 的 Hive 客戶端高級配置代碼段(安全閥)
hive.support.concurrency – true
hive.enforce.bucketing – true
hive.exec.dynamic.partition.mode – nonstrict  
hive.txn.manager – org.apache.hadoop.hive.ql.lockmgr.DbTxnManager  
服務端:hive-site.xml 的 Hive 服務高級配置代碼段(安全閥)
hive.compactor.initiator.on – true
hive.compactor.worker.threads – 1
hive.txn.manager – org.apache.hadoop.hive.ql.lockmgr.DbTxnManager

hive>set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;

創建表
create table t1(id int, name string)
clustered by (id) into 8 buckets
stored as orc TBLPROPERTIES ('transactional'='true');

create table test_trancaction
(user_id Int,name String)
clustered by (user_id) into 3 buckets
stored as orc TBLPROPERTIES ('transactional'='true');

hive> create table test_insert_test(id int,name string) row format delimited fields TERMINATED BY ','; ---臨時表
hive> LOAD DATA LOCAL INPATH '/data/test.txt' OVERWRITE INTO TABLE test_insert_test;
hive> select * from test_insert_test;
OK
1,jerrick
2,tom
3,jerry
4,lily
5,hanmei
6,limlei
7,lucky

hive>insert into test_trancaction select * from test_insert_test;
FAILED: SemanticException [Error 10265]: This command is not allowed on an ACID table merge_data.transactions with a non-ACID transaction manager. Failed

hive>set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
hive>update test_trancaction set name='ccx' where user_id=2;


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM