Spark2 Dataset多维度统计cube与rollup


val df6 = spark.sql("select gender,children,max(age),avg(age),count(age) from Affairs group by Cube(gender,children) order by 1,2")
df6.show
+------+--------+--------+--------+----------+                                  
|gender|children|max(age)|avg(age)|count(age)|
+------+--------+--------+--------+----------+
|  null|    null|    57.0|    34.0|        10|
|  null|      no|    37.0|    27.0|         6|
|  null|     yes|    57.0|    44.5|         4|
|female|    null|    32.0|    29.0|         5|
|female|      no|    32.0|    27.0|         3|
|female|     yes|    32.0|    32.0|         2|
|  male|    null|    57.0|    39.0|         5|
|  male|      no|    37.0|    27.0|         3|
|  male|     yes|    57.0|    57.0|         2|
+------+--------+--------+--------+----------+


val df7 = spark.sql("select gender,children,max(age),avg(age),count(age) from Affairs group by rollup(gender,children) order by 1,2")

df7.show
+------+--------+--------+--------+----------+                                  
|gender|children|max(age)|avg(age)|count(age)|
+------+--------+--------+--------+----------+
|  null|    null|    57.0|    34.0|        10|
|female|    null|    32.0|    29.0|         5|
|female|      no|    32.0|    27.0|         3|
|female|     yes|    32.0|    32.0|         2|
|  male|    null|    57.0|    39.0|         5|
|  male|      no|    37.0|    27.0|         3|
|  male|     yes|    57.0|    57.0|         2|
+------+--------+--------+--------+----------+

 


免责声明!

本站转载的文章为个人学习借鉴使用,本站对版权不负任何法律责任。如果侵犯了您的隐私权益,请联系本站邮箱yoyou2525@163.com删除。



 
粤ICP备18138465号  © 2018-2025 CODEPRJ.COM