翻滾吧,Spark (錯誤記錄)


1) 本地運行報錯: 

Exception in thread "main" org.apache.spark.SparkException: A master URL must be set in your configuration

解決方案:

第一種, 在代碼里設置:

val conf= new SparkConf(). setAppName("SimpleApp"). setMaster("local")

第二種,在IDE里設置為本地單線程運行(-Dspark.master=local)

 2) 調用函數出錯:

Exception in thread "main" java.lang.ExceptionInInitializerError
	at main.scala.test$.main(test.scala:13)
	at main.scala.test.main(test.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: org.apache.spark.SparkException: Only one SparkContext may be running in this JVM (see SPARK-2243). To ignore this error, set spark.driver.allowMultipleContexts = true. The currently running SparkContext was created at:
org.apache.spark.SparkContext.<init>(SparkContext.scala:80)

原因:main函數和另一個獨立的函數我都定義了SparkContext

解決:把main函數里面的SparkContext定義去掉。直接用函數里面的sc 即可。

3)自定義class名 與 Spark內置的類名 重復報錯。

Warning:(1, 64) 
imported `NaiveBayes' is permanently hidden by definition of object NaiveBayes
import org.apache.spark.mllib.classification.{NaiveBayesModel, NaiveBayes}
                                                               ^

  


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM