最近寫了一個WinFrom程序。此程序偵聽TCP端口,接受消息處理,然后再把處理后的消息,利用線程池通過WebService發送出去(即一進一出)。
在程序編寫完成后,進行壓力測試。用Fiddler提交1萬請求。
ThreadPool.QueueUserWorkItem((o) => { try { APPService.AddLog(o as MDMPortalWCF.LogInfo);//發送此Log時,是提交WebService請求的。 } catch (Exception e) { Console.WriteLine(e.ToString()); } },log);
使用procexp.exe查看,隨着TCP請求的增多,2代GCHeap不斷增大。是明顯的內存泄漏線索。
使用WinDbg分析,GC2明顯大,線程正在忙。發現很多String(LogInfo中的屬性)被System.Threading.ThreadPoolWorkQueue.QueueSegment引用。正是內存高的原因。
使用Fiddler查看,發現出乎意料的行為,1萬個TCP接受完成后,Log卻沒有發送完。而是持續之后很久,一直發送,直到幾分鍾后才發送完成。
此行為表明,ThreadPool傾向於少數線程排隊任務,不傾向於開很多線程迅速完成任務。
當發送日志的任務在ThreadPool的隊列排隊時,LogInfo不會被GC回收,而是被QueueSegment持有。待ThreadPool執行完成后,QueueSegment不再引用任務對象,故內存被回收。
后來想想,ThreadPool.QueueUserWorkItem(WaitCallback callBack, object state)中的state對象,在被執行前,將一直被引用,這是合情合理的。但由於ThreadPool的排隊性質,導致內存釋放也是緩慢的,往往是人們想不到的。在此記錄,以示后人。
2018-03-23
在網上讀到一篇文章(http://www.albahari.com/threading/#_Optimizing_the_Thread_Pool)說的非常詳細。更好的解釋了如上的現象。SetMinThreads是一種高級優化線程池的技術。線程池策略是經濟型的,即盡量節省線程新建,這是為了防止多個短生命期任務造成內存突然膨脹。但當入隊的任務,超過半秒執行時間,線程池才開始每半秒新增一個Thread。當然,這也造成了一些問題,比如多個訪問互聯網的線程,我們希望同時一起訪問。這時需要設置ThreadPool.SetMinThreads(100,100),這條語句,告訴線程池管理器,在前100個線程內,不要等待半秒,任務來了,立即創建線程。原文如下:
Increasing the thread pool’s minimum thread count to x doesn’t actually force x threads to be created right away — threads are created only on demand. Rather, it instructs the pool manager to create up to x threads the instant they are required. The question, then, is why would the thread pool otherwise delay in creating a thread when it’s needed?
The answer is to prevent a brief burst of short-lived activity from causing a full allocation of threads, suddenly swelling an application’s memory footprint. To illustrate, consider a quad-core computer running a client application that enqueues 40 tasks at once. If each task performs a 10 ms calculation, the whole thing will be over in 100 ms, assuming the work is divided among the four cores. Ideally, we’d want the 40 tasks to run on exactly four threads:
- Any less and we’d not be making maximum use of all four cores.
- Any more and we’d be wasting memory and CPU time creating unnecessary threads.
And this is exactly how the thread pool works. Matching the thread count to the core count allows a program to retain a small memory footprint without hurting performance — as long as the threads are efficiently used (which in this case they are).
But now suppose that instead of working for 10 ms, each task queries the Internet, waiting half a second for a response while the local CPU is idle. The pool manager’s thread-economy strategy breaks down; it would now do better to create more threads, so all the Internet queries could happen simultaneously.
Fortunately, the pool manager has a backup plan. If its queue remains stationary for more than half a second, it responds by creating more threads — one every half-second — up to the capacity of the thread pool.
The half-second delay is a two-edged sword. On the one hand, it means that a one-off burst of brief activity doesn’t make a program suddenly consume an extra unnecessary 40 MB (or more) of memory. On the other hand, it can needlessly delay things when a pooled thread blocks, such as when querying a database or calling WebClient.DownloadFile
. For this reason, you can tell the pool manager not to delay in the allocation of the first x threads, by calling SetMinThreads
, for instance:
ThreadPool.SetMinThreads (50, 50);
(The second value indicates how many threads to assign to I/O completion ports, which are used by the APM, described in Chapter 23 of C# 4.0 in a Nutshell.)
The default value is one thread per core.