spark 提交任務報錯 Yarn application has already ended! It might have been killed or unable to launch application master

本文轉載自查看原文 2018-08-09 16:54 1111 spark

1.任務是提交在yarn上的，查看 resourceManager頁面

有如下信息

Current usage: 58.4 MB of 1 GB physical memory used; 2.2 GB of 2.1 GB virtual memory used. Killing container.

2.解決方法

yarn-site.xml的中加上

<property>
    <name>yarn.nodemanager.pmem-check-enabled</name>
    <value>false</value>
</property>
<property>
    <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
</property>

解釋

yarn.nodemanager.vmem-pmem-ratio

任務每使用1MB物理內存，最多可使用虛擬內存量，默認是2.1。
yarn.nodemanager.pmem-check-enabled
是否啟動一個線程檢查每個任務正使用的物理內存量，如果任務超出分配值，則直接將其殺掉，默認是true。
yarn.nodemanager.vmem-check-enabled
是否啟動一個線程檢查每個任務正使用的虛擬內存量，如果任務超出分配值，則直接將其殺掉，默認是true

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 this web application instance has been stopped already Android報錯：WindowManager$BadTokenException: Unable to add window -- window has already been added [已解決]Tomcat重新部署項目報錯this web application instance has been stopped already 解決this web application instance has been stopped already tomcat報錯：org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. spark on yarn任務提交緩慢解決 variable ans might not have been initialized 報錯，以及初始化注意點 spark submit 提交任務報錯【Tomcat運行時異常:Illegal access: this web application instance has been stopped already.】 Illegal access: this web application instance has been stopped already .Could not load java.net.BindException._轉