Hive的嚴格模式


在hive里面可以通過嚴格模式防止用戶執行那些可能產生意想不到的不好的效果的查詢,從而保護hive的集群。

用戶可以通過 set hive.mapred.mode=strict 來設置嚴格模式,改成unstrict則為飛嚴格模式。  在嚴格模式下,用戶在運行如下query的時候會報錯。

1. 分區表的查詢沒有使用分區字段來限制。

select * from mart_catering.dim_shopca_reduction_ss limit 1000;

得到的錯誤是

CliDriver update main thread name to 4105daa5-e5a7-49d6-8e02-52c9184732f9
16/08/29 11:20:54 INFO CliDriver: CliDriver update main thread name to 4105daa5-e5a7-49d6-8e02-52c9184732f9

Logging initialized using configuration in jar:file:/opt/meituan/versions/mthive-0.13-package/lib/hive-common-0.13.1.jar!/hive-log4j.properties
FAILED: SemanticException [Error 10041]: No partition predicate found for Alias 'dim_shopca_reduction_ss' Table 'dim_shopca_reduction_ss'
query fails

 

2. 使用了笛卡爾積

當用戶寫代碼將表的別名寫錯的時候會引起笛卡爾積,例如

SELECT *
FROM origindb.promotion__campaign c
JOIN origindb.promotion__campaignex ce
ON c.id = c.id
limit 1000

得到的錯誤信息

CliDriver update main thread name to 9a962abc-afea-470f-9738-dceda72ff1fd
16/08/29 11:38:23 INFO CliDriver: CliDriver update main thread name to 9a962abc-afea-470f-9738-dceda72ff1fd

Logging initialized using configuration in jar:file:/opt/meituan/versions/mthive-0.13-package/lib/hive-common-0.13.1.jar!/hive-log4j.properties
FAILED: SemanticException [Error 10052]: In strict mode, cartesian product is not allowed. If you really want to perform the operation, set hive.mapred.mode=nonstrict
query fails

3. order by 的時候沒有使用limit

SELECT *
FROM origindb.promotion__campaign 
order by id

得到的錯誤信息

CliDriver update main thread name to 5efafee6-1ce8-4985-9b98-c8f5ff88bccd
16/08/29 11:40:45 INFO CliDriver: CliDriver update main thread name to 5efafee6-1ce8-4985-9b98-c8f5ff88bccd

Logging initialized using configuration in jar:file:/opt/meituan/versions/mthive-0.13-package/lib/hive-common-0.13.1.jar!/hive-log4j.properties
FAILED: SemanticException 3:9 In strict mode, if ORDER BY is specified, LIMIT must also be specified. Error encountered near token 'id'
query fails

當使用的limit的時候錯誤消除。

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM