出現的異常是:java.sql.SQLException: No suitable driver。是因為缺少mysql包:
代碼中:
val alterDs: DataFrame = sparkSession.read.format("jdbc")
.option("url", url)
.option("driver","com.mysql.jdbc.Driver")->有mysql驅動
.option("dbtable", altertable)
.option("user", user)
.option("password", password)
.load()
提交命令:
spark-submit \
--name OnLineLogAnalysisTest1 \
--class com.importpack.OnLineLogAnalysis \
--master yarn \
--deploy-mode cluster \
--driver-memory 2G \
--executor-memory 2G \
--executor-cores 1 \
--num-executors 1 \
--packages org.influxdb:influxdb-java:2.5,org.apache.spark:spark-streaming-kafka-0-10_2.11:2.4.2 \
--jars /home/hadoop/lib/fastjson-1.2.58.jar,/home/hadoop/lib/mysql-connector-java-5.1.43-bin.jar \-------->在executor中加入mysql驅動的包
--driver-class-path /home/hadoop/lib/fastjson-1.2.58.jar,/home/hadoop/lib/mysql-connector-java-5.1.43-bin.jar \------>在driver中也加入mysql驅動的包
/home/hadoop/lib/ruozetest-1.0.jar