1. 問題:日志中出現下面的警告:
警告: The web application [ROOT] appears to have started a thread named [HouseKeeper] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
java.lang.Thread.sleep(Native Method)
org.logicalcobwebs.proxool.HouseKeeperThread.run(HouseKeeperThread.java:46)
2. 原因調查:
使用的是 proxool 連接池,查看源碼,分析如下:
在項目某種情況下,ProxoolFacade.shutdown(); 被調用,他會調用:
protected static void shutdown(String finalizer, int delay) {
ConnectionPool[] cps = ConnectionPoolManager.getInstance().getConnectionPools();
for (int i = 0; i < cps.length; i++) {
removeConnectionPool(finalizer, cps[i], delay);
}
// If a shutdown hook was registered then remove it
try {
if (shutdownHook != null) {
ShutdownHook.remove(shutdownHook);
}
} catch (Throwable t) {
if (LOG.isDebugEnabled()) {
LOG.debug("Unanticipated error during removal of ShutdownHook. Ignoring it.", t);
}
}
// Stop threads
PrototyperController.shutdown();
HouseKeeperController.shutdown();
}
這里 首先 關閉連接池,然后在最后面關閉了一個 housekeeper 的 demaon線程:
HouseKeeperController.shutdown();
/**
* Stop all house keeper threads.
*/
protected static void shutdown() {
synchronized(LOCK) {
Iterator i = houseKeeperThreads.iterator();
while (i.hasNext()) {
HouseKeeperThread hkt = (HouseKeeperThread) i.next();
LOG.info("Stopping " + hkt.getName() + " thread");
hkt.cancel();
}
houseKeeperThreads.clear();
}
}
對每一個 housekeeper 線程調用 cancel() 方法,一般只有一個 housekeeper線程:
hkt.cancel();
public class HouseKeeperThread extends Thread {
private static final Log LOG = LogFactory.getLog(HouseKeeperThread.class);
private boolean stop;
public HouseKeeperThread(String name) {
setDaemon(true);
setName(name);
}
public void run() {
while (!stop) {
HouseKeeper hk = HouseKeeperController.getHouseKeeperToRun();
while (hk != null && !stop) {
try {
// if (LOG.isDebugEnabled()) {
// LOG.debug("About to sweep " + hk.getAlias());
// }
hk.sweep();
} catch (ProxoolException e) {
LOG.error("Couldn't sweep " + hk.getAlias(), e);
}
hk = HouseKeeperController.getHouseKeeperToRun();
}
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
LOG.error("Interrupted", e);
}
}
}
protected void cancel() {
stop = true;
}
}
尼瑪 stop 盡然是一個 普通 的 boolean 變量,感覺應該使用 volatile 修飾!
保錯的原因是,housekeeper 線程因為每隔5秒執行一次,而系統 想要 stop 它的時候,它正在 sleep ,所以拋出了異常:
警告: The web application [ROOT] appears to have started a thread named [HouseKeeper] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: java.lang.Thread.sleep(Native Method) org.logicalcobwebs.proxool.HouseKeeperThread.run(HouseKeeperThread.java:46)
但是,因為 設置了 stop = true,所以,當它 sleep 完成之后,也可能會 判斷 stop == true, 也可能判斷 stop == false,因為這里沒有使用 volatile 修飾!
所以最壞的結果,就是 有一個名叫 "HouseKeeper" 的 deamon jvm 線程沒有被成功關閉。
3. 日志顯示,數據庫連接池,確實被正確關閉了:
ContainerBackgroundProcessor[StandardEngine[Catalina]] - Proxool shoutdown [INFO ] org.logicalcobwebs.proxool.ConnectionPool.shutdown(ConnectionPool.java:484):ContainerBackgroundProcessor[StandardEngine[Catalina]] - Shutting down 'autumn' pool immediately [ContainerBackgroundProcessor[StandardEngine[Catalina]]] [DEBUG] org.logicalcobwebs.proxool.ConnectionPool.removeProxyConnection(ConnectionPool.java:441):ContainerBackgroundProcessor[StandardEngine[Catalina]] - 000003 (00/05/00) - #0006 removed because of shutdown. [DEBUG] org.logicalcobwebs.proxool.ConnectionPool.shutdown(ConnectionPool.java:547):ContainerBackgroundProcessor[StandardEngine[Catalina]] - Connection #6 closed [DEBUG] org.logicalcobwebs.proxool.ConnectionPool.removeProxyConnection(ConnectionPool.java:441):ContainerBackgroundProcessor[StandardEngine[Catalina]] - 000003 (00/04/00) - #0005 removed because of shutdown. [DEBUG] org.logicalcobwebs.proxool.ConnectionPool.shutdown(ConnectionPool.java:547):ContainerBackgroundProcessor[StandardEngine[Catalina]] - Connection #5 closed [DEBUG] org.logicalcobwebs.proxool.ConnectionPool.removeProxyConnection(ConnectionPool.java:441):ContainerBackgroundProcessor[StandardEngine[Catalina]] - 000003 (00/03/00) - #0004 removed because of shutdown. [DEBUG] org.logicalcobwebs.proxool.ConnectionPool.shutdown(ConnectionPool.java:547):ContainerBackgroundProcessor[StandardEngine[Catalina]] - Connection #4 closed [DEBUG] org.logicalcobwebs.proxool.ConnectionPool.removeProxyConnection(ConnectionPool.java:441):ContainerBackgroundProcessor[StandardEngine[Catalina]] - 000003 (00/02/00) - #0003 removed because of shutdown. [DEBUG] org.logicalcobwebs.proxool.ConnectionPool.shutdown(ConnectionPool.java:547):ContainerBackgroundProcessor[StandardEngine[Catalina]] - Connection #3 closed [DEBUG] org.logicalcobwebs.proxool.ConnectionPool.removeProxyConnection(ConnectionPool.java:441):ContainerBackgroundProcessor[StandardEngine[Catalina]] - 000003 (00/01/00) - #0002 removed because of shutdown. [DEBUG] org.logicalcobwebs.proxool.ConnectionPool.shutdown(ConnectionPool.java:547):ContainerBackgroundProcessor[StandardEngine[Catalina]] - Connection #2 closed [DEBUG] org.logicalcobwebs.proxool.ConnectionPool.removeProxyConnection(ConnectionPool.java:441):ContainerBackgroundProcessor[StandardEngine[Catalina]] - 000003 (00/00/00) - #0001 removed because of shutdown. [DEBUG] org.logicalcobwebs.proxool.ConnectionPool.shutdown(ConnectionPool.java:547):ContainerBackgroundProcessor[StandardEngine[Catalina]] - Connection #1 closed [INFO ] org.logicalcobwebs.proxool.ConnectionPool.shutdown(ConnectionPool.java:564):ContainerBackgroundProcessor[StandardEngine[Catalina]] - 'autumn' pool has been closed down by ContainerBackgroundProcessor[StandardEngine[Catalina]] in 6 milliseconds. [DEBUG] org.logicalcobwebs.proxool.ShutdownHook.remove(ShutdownHook.java:41):ContainerBackgroundProcessor[StandardEngine[Catalina]] - Removed shutdownHook [INFO ] org.logicalcobwebs.proxool.PrototyperController.shutdown(PrototyperController.java:100):ContainerBackgroundProcessor[StandardEngine[Catalina]] - Stopping Prototyper thread [INFO ] org.logicalcobwebs.proxool.HouseKeeperController.shutdown(HouseKeeperController.java:107):ContainerBackgroundProcessor[StandardEngine[Catalina]] - Stopping HouseKeeper thread
排除了 poxool 導致 數據庫連接泄露的問題。
