spark任務添加依賴的方式：
1 如果是local方式運行，可以通過--jars來添加依賴；
2 如果是yarn方式運行，可以通過spark.yarn.jars來添加依賴；
這兩種方式在oozie上都行不通，首先oozie上沒辦法也不應該通過local運行，其次通過spark.yarn.jars方式配置你會發現根本不會生效，來看為什么
查看LauncherMapper的日志
Spark Version 2.1.1
Spark Action Main class : org.apache.spark.deploy.SparkSubmit
Oozie Spark action configuration
=================================================================
...
--conf
spark.yarn.jars=hdfs://hdfs_name/jarpath/*.jar
--conf
spark.yarn.jars=hdfs://hdfs_name/oozie/share/lib_20180801121138/spark/spark-yarn_2.11-2.1.1.jar
可見oozie會自己添加一個新的spark.yarn.jars配置，如果提供兩個相同的key，spark會如何處理
org.apache.spark.deploy.SparkSubmit
val appArgs = new SparkSubmitArguments(args)
org.apache.spark.launcher.SparkSubmitOptionParser
if (!handle(name, value)) {
org.apache.spark.deploy.SparkSubmitArguments
override protected def handle(opt: String, value: String): Boolean = {
...
case CONF =>
value.split("=", 2).toSeq match {
case Seq(k, v) => sparkProperties(k) = v
case _ => SparkSubmit.printErrorAndExit(s"Spark config without '=': $value")
}
可見會直接覆蓋，使用最后一個配置，即oozie的配置，而不是應用自己提供的配置，這樣就需要應用自己將特殊依賴打包到應用jar中，具體使用maven的maven-assembly-plugin，配置其中的<dependencySets><dependencySet><includes><include>，詳細配置如下：
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
<!-- TODO: a jarjar format would be better -->
<id>jar-with-dependencies</id>
<formats>
<format>jar</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<outputDirectory>/</outputDirectory>
<useProjectArtifact>true</useProjectArtifact>
<unpack>true</unpack>
<scope>runtime</scope>
<includes>
<include>redis.clients:jedis</include>
<include>org.apache.commons:commons-pool2</include>
</includes>
</dependencySet>
</dependencySets>
</assembly>
這里只是將默認提供的jar-with-dependencies.xml內容拷貝出來添加includes配置；