问题1:
Exception in thread "main" org.apache.hadoop.mapred.InvalidJobConfException: Output directory not set
原因:
将FileOutputFormat.setOutputPath(job, new Path(args[1]));写成了FileInputFormat
问题2:
[demo1@master hadoop-2.7.3]$ hadoop jar wordCount.jar org.demo1.hadoop.wordcount.WordMain /test.txt /word
/test.txt
/word
20/02/25 20:00:57 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.122.1:8032
20/02/25 20:00:58 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
20/02/25 20:00:59 INFO input.FileInputFormat: Total input paths to process : 1
20/02/25 20:01:00 INFO mapreduce.JobSubmitter: number of splits:1
20/02/25 20:01:00 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1582682268936_0001
20/02/25 20:01:01 INFO impl.YarnClientImpl: Submitted application application_1582682268936_0001
20/02/25 20:01:01 INFO mapreduce.Job: The url to track the job: http://master:8088/proxy/application_1582682268936_0001/
20/02/25 20:01:01 INFO mapreduce.Job: Running job: job_1582682268936_0001
20/02/25 20:01:12 INFO mapreduce.Job: Job job_1582682268936_0001 running in uber mode : false
20/02/25 20:01:12 INFO mapreduce.Job: map 0% reduce 0%
20/02/25 20:01:20 INFO mapreduce.Job: Task Id : attempt_1582682268936_0001_m_000000_0, Status : FAILED
Error: java.io.IOException: Initialization of all the collectors failed. Error in last collector was :class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$Text
at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:414)
at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:698)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassCastException: class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$Text
at java.lang.Class.asSubclass(Class.java:3404)
at org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:887)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:1004)
at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:402)
... 9 more
20/02/25 20:01:25 INFO mapreduce.Job: Task Id : attempt_1582682268936_0001_m_000000_1, Status : FAILED
Error: java.io.IOException: Initialization of all the collectors failed. Error in last collector was :class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$Text
at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:414)
at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:698)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassCastException: class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$Text
at java.lang.Class.asSubclass(Class.java:3404)
at org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:887)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:1004)
at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:402)
... 9 more
20/02/25 20:01:30 INFO mapreduce.Job: Task Id : attempt_1582682268936_0001_m_000000_2, Status : FAILED
原因:
应该导入import org.apache.hadoop.io.Text;
但是导入了import com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider.Text;