第十章 Hive調優 【嚴格模式】

1. 嚴格模式
    1. 什么是嚴格模式
        hive對sql語法的一些安全性的限制

2. 分區表查詢時必須指定分區

    -- 開啟限制(默認為 false)
    set hive.strict.checks.no.partition.filter=true;

-- 測試

-- 測試
create table `partab` ( `occur_date` string comment '日期' ) comment 'tab' PARTITIONED BY (dt string) row format delimited fields terminated by '\t' lines terminated by '\n' stored as orc; set hive.strict.checks.no.partition.filter=true; select * from partab limit 111; FAILED: SemanticException [Error 10056]: Queries against partitioned tables without a partition filter are disabled for safety reasons. If you know what you are doing, please set hive.strict.checks.no.partition. filter to false and make sure that hive.mapred.mode is not set to 'strict' to proceed. Note that you may get errors or incorrect results if you make a mistake while using some of the unsafe features. No partition predicate for Alias "partab" Table "partab" select * from partab where dt='11' limit 111; Time taken: 0.77 seconds

3. order by必須指定limit

-- 開啟限制(默認為false)
    set hive.strict.checks.orderby.no.limit=true; 說明 : order by 為全局排序,所有數據只有一個reduceTask來處理,防止單個reduce運行時間過長,而導致任務阻塞

-- 測試

-- 測試
    set hive.strict.checks.orderby.no.limit=true; select * from arraytab order by name; FAILED: SemanticException 1:36
    Order by-s without limit are disabled for safety reasons. If you know what you are doing, please set hive.strict.checks.orderby.no.limit to false and make sure that hive.mapred.mode is not set to 'strict' to proceed. Note that you may get errors or incorrect results if you make a mistake while using some of the unsafe features.. Error encountered near token 'name'

4. 限制 笛卡爾積

    -- 開啟限制(默認為false)
    set hive.strict.checks.cartesian.product=true; 說明 : 出發笛卡爾積時,join操作會在一個reduceTask中執行

-- 測試

-- 測試
        set hive.strict.checks.cartesian.product=true; select t1.*,t2.*
        from arraytab as t1 inner join arraytab as t2; FAILED: SemanticException Cartesian products are disabled for safety reasons. If you know what you are doing, please set hive.strict.checks.cartesian.product to false and make sure that hive.mapred.mode is not set to 'strict' to proceed. Note that you may get errors or incorrect results if you make a mistake while using some of the unsafe features.