hadoop job -kill 與 yarn application -kii（作業卡了或作業重復提交或MapReduce任務運行到running job卡住）

 

 　　

　　問題詳情

 

 

 

 

 

 

 

 

 

 解決辦法

[hadoop@master ~]$ hadoop job -kill job_1493782088693_0001
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it.

17/05/03 11:48:25 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/05/03 11:48:25 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/05/03 11:48:29 INFO impl.YarnClientImpl: Killed application application_1493782088693_0001
Killed job job_1493782088693_0001






[hadoop@master ~]$ hadoop job -kill job_1493782088693_0002
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it.

17/05/03 11:48:25 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/05/03 11:48:25 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/05/03 11:48:29 INFO impl.YarnClientImpl: Killed application application_1493782088693_0001
Killed job job_1493782088693_0002






[hadoop@master ~]$ hadoop job -kill job_1493782088693_0003
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it.

17/05/03 11:48:25 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/05/03 11:48:25 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/05/03 11:48:29 INFO impl.YarnClientImpl: Killed application application_1493782088693_0001
Killed job job_1493782088693_0003

 

 

　　有時候上述這樣kill做下來，並不管用，得再來

[hadoop@master ~]$ yarn application -kill application_1493782088693_0001
17/05/03 14:45:23 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/05/03 14:45:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Application application_1493782088693_0001 has already finished 
[hadoop@master ~]$ yarn application -kill application_1493782088693_0002
17/05/03 14:45:59 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/05/03 14:45:59 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Application application_1493782088693_0002 has already finished 
[hadoop@master ~]$ yarn application -kill application_1493782088693_0003
17/05/03 14:46:47 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/05/03 14:46:49 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Application application_1493782088693_0003 has already finished 
[hadoop@master ~]$

 

 

 

 

 

 

[hadoop@master ~]$ hadoop job -list
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it.

17/05/03 14:39:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/05/03 14:39:24 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
Total jobs:0
                  JobId         State         StartTime        UserName           Queue      Priority     UsedContainers     RsvdContainers     UsedMem     RsvdMem     NeededMem       AM info
[hadoop@master ~]$ 

 

 

 

 

 

　　同時，大家要注意，是否是進程的消失？

　　也會是你的slave1 還是 slave2的進程自動消失了。注意 ，這是個很隱蔽的問題。

 　　重新停止集群，再重新啟動集群。

　　如果還是出現這個問題的話，則

MapReduce任務運行到running job卡住

 

 

 

 

 

　　關於這里參數配置的解釋，見

Hadoop YARN配置參數剖析(2)—權限與日志聚集相關參數

 

 

　注意，對master、slave1和slave2都要操作，然后，再[hadoop@master hadoop-2.6.0]$ sbin/stop-all.sh

　 再[hadoop@master hadoop-2.6.0]$ sbin/start-all.sh即可。

 

 

 

 　　成功！