啟動Flink的yarn-session模式出錯


  問題如下:bin/yarn-session.sh -d 報錯

 The program finished with the following exception:

org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn session cluster
    at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:425)
    at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:606)
    at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:860)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
    at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
    at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:860)
Caused by: org.apache.flink.configuration.IllegalConfigurationException: The number of requested virtual cores for application master 1 exceeds the maximum number of virtual cores 0 available in the Yarn Cluster.
    at org.apache.flink.yarn.YarnClusterDescriptor.isReadyForDeployment(YarnClusterDescriptor.java:326)
    at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:534)
    at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:418)
    ... 7 more

  解析:日志中顯示的大概原因就行啟動失敗,資源不夠,但是我啟動的時候並沒有指定任何參數,使用的是默認值,申請的資源大小絕對不會超過可用的資源大小,完全夠用,所以報錯原因跟運行時指定的參數無關,那么我們該如何解決呢?

  解決:之前測試時啟動太多次Filnk的yarn-session模式,所以斷定是臨時文件過多導致,故刪除HDFS中的Flink的臨時文件即可,你的文件存在哪就刪除哪,我存在/flink/yarn/ha目錄下,所以直接刪除即可,可以在flink的conf目錄下的flink-conf.yaml查看HDFS的存儲路徑

  再次啟動,發現啟動成功

bin/yarn-session.sh -d


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM