spark通過idea遠程提交job到yarn:
Caused by: java.lang.ClassCastException: cannot assign instance of scala.collection.immutable.List$SerializationProxy to field org.apache.spark.rdd.RDD.org$apache$spark$rdd$RDD$$dependencies_ of type scala.collection.Seq in instance of org.apache.spark.rdd.MapPartitionsRDD
1.大概是由於沒有設置要提交的jar包路徑,遠程提交是需要設置SPARK_HOME的環境變量,所以idea遠程提交應該是類似於spark-shell --master yarn的提交方式,需要執行的代碼要yarn集群的executor端