flink1.10 taskmanager內存分配


 

taskmanager.memory.process.size: 4096m

-Xmx1748185876(1667M,flink web界面顯示的java heap size) -Xms1748185876 -XX:MaxDirectMemorySize=510698460(487M) -XX:MaxMetaspaceSize=100663296(96M)

4096-JVM specific memory = total flink memory (M)(當配置taskmanager.memory.flink.size時,這里不需要減,配置的內存除JVM Metaspace和JVM Overhead之外全部作為flink  memory)

4096m * 0.88 = 3590M(flink memory,可以拿taskmanager.memory.network.min/0.1算出來)
taskmanager.memory.framework.off-heap.size=134217728b 128M
taskmanager.memory.framework.heap.size=134217728b 128M
taskmanager.memory.network.max=376480732b 359M
taskmanager.memory.network.min=376480732b 359M 3590*0.1
taskmanager.memory.managed.size=1505922928b 1436M 3590*0.4
taskmanager.memory.task.heap.size=1613968148b 1539M (3590-taskmanager.memory.framework.off-heap.size-taskmanager.memory.framework.heap.size
-taskmanager.memory.network.max-taskmanager.memory.managed.size)=1539
taskmanager.memory.task.off-heap.size=0b

Flink taskmanager JVM heap(T)=taskmanager.memory.task.heap.size+taskmanager.memory.framework.heap.size=1539M +128M=1667M

通過以上可快速得出 total flink memory (M):M=(T+128)*2   #####(1667+128)*2=3590

T-taskmanager.memory.framework.heap.size=taskmanager.memory.task.heap.size=M-taskmanager.memory.framework.off-heap.size-taskmanager.memory.framework.heap.size
-taskmanager.memory.network.max-taskmanager.memory.managed.size

T-128=M-128-128-M*0.1-M*0.4

M=(T+128)*2

另外,當配成taskmanager.memory.flink.size時,內存除了JVM Metaspace和JVM Overhead全作為 total flink memory (M),因此通過配置時的taskmanager.memory.flink.size值可以迅速計算出T=M/2-128    #4096/2-128=1920M

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM