hive grouping sets多維度報錯


An additional MR job is introduced since the cardinality of grouping sets is more than hive.new.job.grouping.set.cardinality. This functionality is not supported with distincts. Either set hive.new.job.grouping.set.cardinality to a high number (higher than the number of rows per input row due to grouping sets in the query), or rewrite the query to not use distincts. The number of rows per input row due to grouping sets is 32

如果你的 grouping sets大於等於5個維度,將會報如上的錯誤;解決辦法:

1.在你的hql語句前面加上 set hive.new.job.grouping.set.cardinality=xx;(例如我這里是5個維度,一共32個grouping sets,xx我寫的64 )

2.可以通過在子查詢中用group by去重,避免在聚合中用到distinct

 

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM