問題如下:bin/yarn-session.sh -d 報錯
The program finished with the following exception: org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn session cluster at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:425) at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:606) at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:860) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:860) Caused by: org.apache.flink.configuration.IllegalConfigurationException: The number of requested virtual cores for application master 1 exceeds the maximum number of virtual cores 0 available in the Yarn Cluster. at org.apache.flink.yarn.YarnClusterDescriptor.isReadyForDeployment(YarnClusterDescriptor.java:326) at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:534) at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:418) ... 7 more
解析:日志中顯示的大概原因就行啟動失敗,資源不夠,但是我啟動的時候並沒有指定任何參數,使用的是默認值,申請的資源大小絕對不會超過可用的資源大小,完全夠用,所以報錯原因跟運行時指定的參數無關,那么我們該如何解決呢?
解決:之前測試時啟動太多次Filnk的yarn-session模式,所以斷定是臨時文件過多導致,故刪除HDFS中的Flink的臨時文件即可,你的文件存在哪就刪除哪,我存在/flink/yarn/ha目錄下,所以直接刪除即可,可以在flink的conf目錄下的flink-conf.yaml查看HDFS的存儲路徑
再次啟動,發現啟動成功
bin/yarn-session.sh -d