第十章 Hive调优 【严格模式】

1. 严格模式
    1. 什么是严格模式
        hive对sql语法的一些安全性的限制

2. 分区表查询时必须指定分区

    -- 开启限制(默认为 false)
    set hive.strict.checks.no.partition.filter=true;

-- 测试

-- 测试
create table `partab` ( `occur_date` string comment '日期' ) comment 'tab' PARTITIONED BY (dt string) row format delimited fields terminated by '\t' lines terminated by '\n' stored as orc; set hive.strict.checks.no.partition.filter=true; select * from partab limit 111; FAILED: SemanticException [Error 10056]: Queries against partitioned tables without a partition filter are disabled for safety reasons. If you know what you are doing, please set hive.strict.checks.no.partition. filter to false and make sure that hive.mapred.mode is not set to 'strict' to proceed. Note that you may get errors or incorrect results if you make a mistake while using some of the unsafe features. No partition predicate for Alias "partab" Table "partab" select * from partab where dt='11' limit 111; Time taken: 0.77 seconds

3. order by必须指定limit

-- 开启限制(默认为false)
    set hive.strict.checks.orderby.no.limit=true; 说明 : order by 为全局排序,所有数据只有一个reduceTask来处理,防止单个reduce运行时间过长,而导致任务阻塞

-- 测试

-- 测试
    set hive.strict.checks.orderby.no.limit=true; select * from arraytab order by name; FAILED: SemanticException 1:36
    Order by-s without limit are disabled for safety reasons. If you know what you are doing, please set hive.strict.checks.orderby.no.limit to false and make sure that hive.mapred.mode is not set to 'strict' to proceed. Note that you may get errors or incorrect results if you make a mistake while using some of the unsafe features.. Error encountered near token 'name'

4. 限制 笛卡尔积

    -- 开启限制(默认为false)
    set hive.strict.checks.cartesian.product=true; 说明 : 出发笛卡尔积时,join操作会在一个reduceTask中执行

-- 测试

-- 测试
        set hive.strict.checks.cartesian.product=true; select t1.*,t2.*
        from arraytab as t1 inner join arraytab as t2; FAILED: SemanticException Cartesian products are disabled for safety reasons. If you know what you are doing, please set hive.strict.checks.cartesian.product to false and make sure that hive.mapred.mode is not set to 'strict' to proceed. Note that you may get errors or incorrect results if you make a mistake while using some of the unsafe features.