Windows 下部署調試 Hadoop 時關於 java.io.FileNotFoundException 問題總結


折騰了差不多三天,終於從網上找到了相關的解決方案:

 

問題描述:配置搭建好 Windows + Hadoop + Cygwin 之后,一切運行順利,就是在調試下面的小 sample 的時候總是報如下錯誤:

Samples:

[//下面的實例將已解壓的 conf 目錄拷貝作為輸入,查找並顯示匹配給定正則表達式的條目。輸出寫入到指定的output目錄。
$ mkdir input
$ cp conf/*.xml input
$ bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'
$ cat output/*
]

錯誤信息:

$ bin/hadoop jar hadoop-0.20.2-examples.jar grep /input/*.xml /output 'dfs[a-z.]+'
12/09/11 14:24:01 INFO mapred.FileInputFormat: Total input paths to process : 5
12/09/11 14:24:02 INFO mapred.JobClient: Running job: job_201209111241_0002
12/09/11 14:24:03 INFO mapred.JobClient:  map 0% reduce 0%
12/09/11 14:24:12 INFO mapred.JobClient: Task Id : attempt_201209111241_0002_m_000006_0, Status : FAILED
java.io.FileNotFoundException: File C:/tmp/hadoop-tiky.zheng/mapred/local/taskTracker/jobcache/job_201209111241_0002/attempt_201209111241_0002_m_000006_0/work/tmp does not exist.
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
        at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519)
        at org.apache.hadoop.mapred.Child.main(Child.java:155)

12/09/11 14:24:21 INFO mapred.JobClient: Task Id : attempt_201209111241_0002_m_000006_1, Status : FAILED
java.io.FileNotFoundException: File C:/tmp/hadoop-tiky.zheng/mapred/local/taskTracker/jobcache/job_201209111241_0002/attempt_201209111241_0002_m_000006_1/work/tmp does not exist.
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
        at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519)
        at org.apache.hadoop.mapred.Child.main(Child.java:155)

 

最后找到答案:

需要在mapred-site.xml中添加暫存目錄:

  <name>mapred.child.tmp</name>

  <value>/home/hadoop-0.20.1/temp</value>

加完,搞定!


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM