1.打包過程發現錯誤如下:
xception in thread "main" org.apache.spark.SparkException: Application application_1487576876020_5626 finished with failed status at org.apache.spark.deploy.yarn.Client.run(Client.scala:1167) at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1213) at org.apache.spark.deploy.yarn.Client.main(Client.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
首先快速在沒有運行完前,查看log中報錯:
Unsupported major.minor version 52.0
2
原因:是scala的jre和java的jdk不匹配,需要將idea中的scala版本有默認版本改為合適的對應版本,對應版本需要自己下載,下載鏡像安裝包,下載好后win下會自動配置環境變量,無需手動安裝。