hadoop開發setjar方法

 

屏蔽

//job.setJar("/Users/sumeng/IdeaProjects/cloudHadoop/out/artifacts/cloudHadoop_jar/cloudHadoop.jar");
job.setJarByClass(WordCount.class);

報錯：

Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class WordCount$TokenizerMapper not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2199)
    at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:745)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: Class WordCount$TokenizerMapper not found
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2105)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2197)
    ... 8 more

屏蔽

job.setJar("/Users/sumeng/IdeaProjects/cloudHadoop/out/artifacts/cloudHadoop_jar/cloudHadoop.jar");
//job.setJarByClass(WordCount.class);

不會報錯

 

//通過Configuration對象獲取job對象，該job對象會組織所有的該mapreduce的所有各種組件
Job job = Job.getInstance(conf);
//指定jar包所在路徑，本地模式需要這樣指定，如果不是本地，則使用setJarByClass指定所在class文件即可
//job.setJarByClass("wordcountJar/wordcount.jar")
job.setJar("wordcountJar/wordcount.jar");
--------------------- 
作者：夜下探戈 
來源：CSDN 
原文：https://blog.csdn.net/dudefu011/article/details/79586191 
版權聲明：本文為博主原創文章，轉載請附上博文鏈接！

 

可以確定的是，

情況一：如果采用打成jar包，在hadoop集群中通過hadoop jar命令運行，則只需要寫job.setJarByClass

情況二：本地idea開發mapreduce程序提交到hadoop集群執行，參考https://blog.csdn.net/dream_an/article/details/84342770

https://www.jianshu.com/p/2c9c22130225也是把jar包設置到了本機

https://blog.csdn.net/shirukai/article/details/81021872講解了打成jar包的過程

 

還是不是很明白setjar 與 setJarbyClass 的區別和意義

https://www.codetd.com/article/664330

https://my.oschina.net/zhzhenqin/blog/163158