1、問題描述
Status: Failed Vertex's TaskResource is beyond the cluster container capability,Vertex=vertex_1597977573448_0003_1_00 [Map 9], Requested TaskResource=<memory:4096, vCores:1>, Cluster MaxContainerCapability=<memory:2048, vCores:2> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex's TaskResource is beyond the cluster container capability,Vertex=vertex_1597977573448_0003_1_00 [Map 9], Requested TaskResource=<memory:4096, vCores:1>, Cluster MaxContainerCapability=<memory:2048, vCores:2>
2、問題原因:
hive.tez.container.size設置了4096內存,超過了yarn的容器允許的最大內存,yarn的nodemanager.resource.memory-mb設置的過小,需要將調整改值。
或者調整hive.tez.container.size的值小於nodemanager.resource.memory-mb的值。
3、解決方法
將nodemanager.resource.memory-mb調整為6g或者將hive.tez.container.size的值改為3g。