關於used下的內存使用情況分析
1、來由
開發同事反應本機沒有做什么，內存就占用了11G，也不知道是什么進程占用了，為此我展開了used內存占用的分析，上數據圖
# free -m
total used free shared buffers cached
Mem: 32111 11316 20794 0 157 1396
-/+ buffers/cache: 9762 22348
Swap: 8047 64 7983
2、原因分析
針對此問題，我及時查看哪個進程占用
# ps aux | head -1; ps aux| sort -rnk 4 | head -10
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
zabbix 22866 0.0 0.0 78244 1836 ? S Jan18 0:27 /usr/sbin/zabbix_agentd: listener #3 [waiting for connection]
zabbix 22865 0.0 0.0 78244 1856 ? S Jan18 0:27 /usr/sbin/zabbix_agentd: listener #2 [waiting for connection]
zabbix 22864 0.0 0.0 78244 1852 ? S Jan18 0:27 /usr/sbin/zabbix_agentd: listener #1 [waiting for connection]
zabbix 22863 0.0 0.0 78120 1924 ? S Jan18 0:59 /usr/sbin/zabbix_agentd: collector [idle 1 sec]
zabbix 22861 0.0 0.0 78120 768 ? S Jan18 0:00 /usr/sbin/zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
rtkit 2855 0.0 0.0 168452 844 ? SNl 2020 4:55 /usr/libexec/rtkit-daemon
rpcuser 2212 0.0 0.0 23344 740 ? Ss 2020 0:00 rpc.statd
rpc 2092 0.0 0.0 18972 604 ? Ss 2020 0:16 rpcbind
root 99 0.0 0.0 0 0 ? S 2020 0:00 [kintegrityd/9]
但是沒有找到問題點，繼續
查看/proc/meminfo文件時
# cat /proc/meminfo 未啟動java時 啟動java后
MemTotal: 32882392 kB 32882392 kB
MemFree: 11152484 kB 9941460 kB
Buffers: 174520 kB 175588 kB
Cached: 1485864 kB 1508656 kB
SwapCached: 5100 kB 5092 kB
Active: 10906000 kB 12098640 kB
Inactive: 845964 kB 859936 kB
Active(anon): 10088872 kB 11271860 kB
Inactive(anon): 4448 kB 4448 kB
Active(file): 817128 kB 826780 kB
Inactive(file): 841516 kB 855488 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 8241144 kB 8241144 kB
SwapFree: 8174820 kB 8174828 kB
Dirty: 980 kB 1472 kB
Writeback: 0 kB
AnonPages: 10090488 kB 11275588 kB
Mapped: 35960 kB 35964 kB
Shmem: 1600 kB 1600 kB
Slab: 9708836 kB 9709656 kB
SReclaimable: 8982448 kB 8982488 kB
SUnreclaim: 726388 kB 727168 kB
KernelStack: 5592 kB 5888 kB
PageTables: 34344 kB 36660 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 24682340 kB 24682340 kB
Committed_AS: 21549588 kB 21577332 kB
VmallocTotal: 34359738367 kB 34359738367 kB
VmallocUsed: 200220 kB 200220 kB
VmallocChunk: 34341972664 kB 34341972664 kB
HardwareCorrupted: 0 kB
AnonHugePages: 9539584 kB 10670080 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB 2048 kB
DirectMap4k: 10240 kB 10240 kB
DirectMap2M: 33544192 kB 33544192 kB
3、在網絡上找原因
自己分析不出問題點，網絡上有很多人說slab來分配內存，但是我想知道slab中內存包括那些內容，很少的知識點，怎么辦？
我找到一個工具，linux-fincore
下載linux-fincore工具：
#wget https://codeload.github.com/waleedmazhar/linux-ftools/zip/master
安裝步驟如下：
#cd /opt
#unzip -d linux-ftools-master.zip
#cd linux-ftools-master
#./configure --prefix=/usr/local/linux-ftools #請確保安裝gcc依賴包，無錯誤進行下一步
#make
#make install
添加環境變量：
#mv /usr/local/ftools/bin/* /usr/local/bin/
添加命令使用 `# linux-fincore --pages=false --summarize --only-cached * filename size total_pages min_cached page cached_pages cached_size cached_perc -------- ---- ----------- --------------- ------------ ----------- ----------- Could not mmap file: linux-ftools-master: No such device netapp.txt 891,161 218 0 218 892,928 100.00 --- total cached size: 892,928 `
為了更好抓取內存中進程，我這邊添加個腳本+定時任務來完成，提供開發需要的資料，
腳本為：
# cat fincore.sh
#!/bin/bash
. /etc/init.d/functions
PATH=/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/java/jdk1.8.0_131/bin:/usr/java/jdk1.8.0_131/lib:/root/bin
echo $PATH
if [ ! -f /usr/local/bin/linux-fincore ];then
echo "you haven't install linux-fincore."
exit 1
fi
ps -e -o pid,rss|sort -rnk 2 | head -10 | awk '{print $1}' > /tmp/cache.pid
if [ -f /tmp/cache.log ];then
echo "the cache.files is exist,removing now"
rm -f /tmp/cache.log
fi
while read line
do
lsof -p $line 2>/dev/null | awk '{print $9}' >>/tmp/cache.log
done > /tmp/cache.fincore
fi
done
linux-fincore -s `cat /tmp/cache.fincore`
rm -f /tmp/cache.{pid,log,fincore}
測試一下腳本使用
#/bin/bash fincore.sh > 2.log
#查看日志中分析的數據
/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/java/jdk1.8.0_131/bin:/usr/java/jdk1.8.0_131/lib:/root/bin
filename size total_pages min_cached page cached_pages cached_size cached_perc
-------- ---- ----------- --------------- ------------ ----------- -----------
/weaver/jdk1.8.0_131/bin/java 7,734 2 0 1 4,096 50.00
/lib64/libuuid.so.1.3.0 18,936 5 0 5 20,480 100.00
/usr/lib64/libstdc++.so.6.0.13 989,840 242 0 109 446,464 45.04
/usr/lib64/libsapjco3.so 5,407,699 1,321 0 500 2,048,000 37.85
/weaver/ecology/WEB-INF/lib/ehcache-2.8.3.jar 5,637,564 1,377 0 454 1,859,584 32.97
這邊我只保留部分數據，
這時，腳本已啟作用了，現在只需要添加定時任務讓它自動運行並輸出日志給到系統了，方便他們及時查看。
