Spark_4_2：Spark函數之collect、toArray和collectAsMap

collect、toArray

將RDD轉換為Scala的數組。

collectAsMap

與collect、toArray相似。collectAsMap將key-value型的RDD轉換為Scala的map。

注意：map中如果有相同的key，其value只保存最后一個值。

# 創建一個2分區的RDD
scala> var z = sc.parallelize(List( ("cat",2), ("cat", 5), ("mouse", 4),("cat", 12), ("dog", 12), ("mouse", 2)), 2)
z: org.apache.spark.rdd.RDD[(String, Int)] = ParallelCollectionRDD[129] at parallelize at <console>:21

# 輸出所有分區的數據
scala> z.collect
res44: Array[(String, Int)] = Array((cat,2), (cat,5), (mouse,4), (cat,12), (dog,12), (mouse,2))

# 轉化為字典
scala> z.collectAsMap
res45: scala.collection.Map[String,Int] = Map(dog -> 12, cat -> 12, mouse -> 2)
scala>