https://blog.csdn.net/weixin_39528029/article/details/114124727
最近修改同事代碼時遇到一個問題，通過 httpclient 默認配置產生的 httpclient 如果不關閉，會導致連接無法釋放，很快打滿服務器連接(內嵌 Jetty 配置了 25 連接上限)，主動關閉問題解決；后來優化為通過連接池生成 httpclient 后，如果關閉 httpclient 又會導致連接池關閉，后面新的 httpclient 也無法再請求，這里總結遇到的一些問題和疑問。
官網示例中的以下三個 close 分別釋放了什么資源，是否可以省略，以及在什么時機調用，使用連接池時有區別么？
作為 RPC 通信客戶端，如何復用 TCP 連接？
一、資源釋放
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpget = new HttpGet("http://localhost/");
CloseableHttpResponse response = httpclient.execute(httpget);
try {undefined
HttpEntity entity = response.getEntity();
if (entity != null) {undefined
InputStream instream = entity.getContent();
try {undefined
// do something useful
} finally {undefined
instream.close();
}
}
} finally {undefined
response.close();
}
// httpclient.close();
首先需要了解默認配置 createDefault 和使用了 custom 連接池(文章最后的 HttpClientUtil)兩種情況的區別，通過源碼可以看到前者也創建了連接池，最大連接20個，單個 host最大2個，但是區別在於每次創建的 httpclient 都自己維護了自己的連接池，而 custom 連接池時所有 httpclient 共用同一個連接池，這是在 api 使用方面需要注意的地方，要避免每次請求新建連接池、關閉連接池，造成性能問題。
The difference between closing the content stream and closing the response is that the former will attempt to keep the underlying connection alive by consuming the entity content while the latter immediately shuts down and discards the connection.
第一個 close 是讀取 http 正文的數據流，類似的還有響應寫入流，都需要主動關閉，如果是使用 EntityUtils.toString(response.getEntity(), "UTF-8"); 的方式，其內部會進行關閉。如果還有要讀/寫的數據、或不主動關閉，相當於 http 請求事務未處理完成，這時通過其他方式關閉(第二個 close)相當於異常終止，會導致該連接無法被復用，對比下面兩段日志。
第一個 close 未調用時，第二個 close 調用，連接無法被復用，kept alive 0。
o.a.http.impl.execchain.MainClientExec : Connection can be kept alive indefinitely
h.i.c.DefaultManagedHttpClientConnection : http-outgoing-0: Close connection
o.a.http.impl.execchain.MainClientExec : Connection discarded
h.i.c.PoolingHttpClientConnectionManager : Connection released: [id: 0][route: {}->http://127.0.0.1:8080][total kept alive: 0; route allocated: 0 of 2; total allocated: 0 of 20]
第一個 close 正常調用時，第二個 close 調用，連接可以被復用，kept alive 1。
o.a.http.impl.execchain.MainClientExec : Connection can be kept alive indefinitely
h.i.c.PoolingHttpClientConnectionManager : Connection [id: 0][route: {}->http://127.0.0.1:8080] can be kept alive indefinitely
h.i.c.DefaultManagedHttpClientConnection : http-outgoing-0: set socket timeout to 0
h.i.c.PoolingHttpClientConnectionManager : Connection released: [id: 0][route: {}->http://127.0.0.1:8080][total kept alive: 1; route allocated: 1 of 2; total allocated: 1 of 20]
第二個 close 是強行制止和釋放連接到連接池，相當於對第一個 close 的保底操作(上面關閉了這個似乎沒必要了？)，結合上面引用的官方文檔寫到 immediately shuts down and discards the connection，這里如果判斷需要 keep alive 實際也不會關閉 TCP 連接，因為通過 netstat 可以看到，第二段日志后在終端可以繼續觀察到連接：
# netstat -n | grep tcp4 | grep 8080
tcp4 0 0 127.0.0.1.8080 127.0.0.1.51003 ESTABLISHED
tcp4 0 0 127.0.0.1.51003 127.0.0.1.8080 ESTABLISHED
在 SOF 上可以搜到這段話，但是感覺和上面觀察到的並不相符？
The underlying HTTP connection is still held by the response object to allow the response content to be streamed directly from the network socket. In order to ensure correct deallocation of system resources, the user MUST call CloseableHttpResponse#close() from a finally clause. Please note that if response content is not fully consumed the underlying connection cannot be safely re-used and will be shut down and discarded by the connection manager.
第三個 clsoe，也就是 httpclient.close 會徹底關閉連接池，以及其中所有連接，一般情況下，只有在關閉應用時調用以釋放資源(補充：當 httpClientBuilder.setConnectionManagerShared(true) 時，並不會關閉連接池)。
二、連接復用
根據 http 協議 1.1 版本，各個 web 服務器都默認支持 keepalive，因此當 http 請求正常完成后，服務器不會主動關閉 tcp(直到空閑超時或數量達到上限)，使連接會保留一段時間，前面我們也知道 httpclient 在判斷可以 keepalive 后，即使調用了 close 也不會關閉 tcp 連接(可以認為 release 到連接池)。為了管理這些保留的連接，以及方便 api 調用，一般設置一個全局的連接池，並基於該連接池提供 httpclient 實例，這樣就不需要考慮維護 httpclient 實例生命周期，隨用隨取(方便狀態管理？)，此外考慮到 http 的單路性，一個請求響應完成結束后，該連接才可以再次復用，因此連接池的最大連接數決定了並發處理量，該配置也是一種保護機制，超出上限的請求會被阻塞，也可以配合熔斷組件使用，當服務方慢、或不健康時熔斷降級。
最后還有一個問題，觀察到 keepalive 的 tcp 連接過一段時間后會變成如下狀態：
# netstat -n | grep tcp4 | grep 8080
tcp4 0 0 127.0.0.1.8080 127.0.0.1.51866 FIN_WAIT_2
tcp4 0 0 127.0.0.1.51866 127.0.0.1.8080 CLOSE_WAIT
可以看出服務器經過一段時間，認為該連接空閑，因此主動關閉，收到對方響應后進入 FIN_WAIT_2 狀態(等待對方也發起關閉)，而客戶端進入 CLOSE_WAIT 狀態后卻不再發起自己這一方的關閉請求，這時雙方處於半關閉。官方文檔解釋如下：
One of the major shortcomings of the classic blocking I/O model is that the network socket can react to I/O events only when blocked in an I/O operation. When a connection is released back to the manager, it can be kept alive however it is unable to monitor the status of the socket and react to any I/O events. If the connection gets closed on the server side, the client side connection is unable to detect the change in the connection state (and react appropriately by closing the socket on its end).
這需要有定期主動做一些檢測和關閉動作，從這個角度考慮，默認配置產生的 HttpClient 沒有這一功能，不應該用於生產環境，下面這個監控線程可以完成該工作，包含它的完整的 HttpUtil 從文章最后連接獲取。
public static class IdleConnectionMonitorThread extends Thread {undefined
private final HttpClientConnectionManager connMgr;
private volatile boolean shutdown;
public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {undefined
super();
this.connMgr = connMgr;
}
@Override
public void run() {undefined
try {undefined
while (!shutdown) {undefined
synchronized (this) {undefined
wait(30 * 1000);
// Close expired connections
connMgr.closeExpiredConnections();
// Optionally, close connections
// that have been idle longer than 30 sec
connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
}
}
} catch (InterruptedException ex) {undefined
// terminate
}
}
最后展示一個完整的示例，首先多線程發起兩個請求，看到創建兩個連接，30秒之后再發起一個請求，可以復用之前其中一個連接，另一個連接因空閑被關閉，隨后最后等待 2 分鍾后再發起一個請求，由於之前連接已過期失效，重新創建連接。
並發兩個請求
16:54:44.504 [ Thread-4] : Connection request: [route: {}->http://127.0.0.1:8080][total kept alive: 0; route allocated: 0 of 150; total allocated: 0 of 150]
16:54:44.504 [ Thread-5] : Connection request: [route: {}->http://127.0.0.1:8080][total kept alive: 0; route allocated: 0 of 150; total allocated: 0 of 150]
16:54:44.515 [ Thread-5] : Connection leased: [id: 1][route: {}->http://127.0.0.1:8080][total kept alive: 0; route allocated: 2 of 150; total allocated: 2 of 150]
16:54:44.515 [ Thread-4] : Connection leased: [id: 0][route: {}->http://127.0.0.1:8080][total kept alive: 0; route allocated: 2 of 150; total allocated: 2 of 150]
16:54:44.517 [ Thread-5] : Opening connection {}->http://127.0.0.1:8080
16:54:44.517 [ Thread-4] : Opening connection {}->http://127.0.0.1:8080
16:54:44.519 [ Thread-4] : Connecting to /127.0.0.1:8080
16:54:44.519 [ Thread-5] : Connecting to /127.0.0.1:8080
16:54:44.521 [ Thread-5] : Connection established 127.0.0.1:52421127.0.0.1:8080
16:54:44.521 [ Thread-4] : Connection established 127.0.0.1:52420127.0.0.1:8080
....
16:54:49.486 [ main] : [leased: 2; pending: 0; available: 0; max: 150]
16:54:49.630 [ Thread-4] : Connection can be kept alive indefinitely
16:54:49.630 [ Thread-5] : Connection can be kept alive indefinitely
16:54:49.633 [ Thread-4] : Connection [id: 0][route: {}->http://127.0.0.1:8080] can be kept alive indefinitely
16:54:49.633 [ Thread-5] : Connection [id: 1][route: {}->http://127.0.0.1:8080] can be kept alive indefinitely
16:54:49.633 [ Thread-4] : http-outgoing-0: set socket timeout to 0
16:54:49.633 [ Thread-5] : http-outgoing-1: set socket timeout to 0
16:54:49.633 [ Thread-4] : Connection released: [id: 0][route: {}->http://127.0.0.1:8080][total kept alive: 1; route allocated: 2 of 150; total allocated: 2 of 150]
16:54:49.633 [ Thread-5] : Connection released: [id: 1][route: {}->http://127.0.0.1:8080][total kept alive: 2; route allocated: 2 of 150; total allocated: 2 of 150]
16:54:54.488 [ main] : [leased: 0; pending: 0; available: 2; max: 150]
#netstat -n | grep tcp4 | grep 8080
tcp4 0 0 127.0.0.1.8080 127.0.0.1.52421 ESTABLISHED
tcp4 0 0 127.0.0.1.8080 127.0.0.1.52420 ESTABLISHED
tcp4 0 0 127.0.0.1.52421 127.0.0.1.8080 ESTABLISHED
tcp4 0 0 127.0.0.1.52420 127.0.0.1.8080 ESTABLISHED
下一個請求
16:55:14.489 [ Thread-6] : Connection request: [route: {}->http://127.0.0.1:8080][total kept alive: 2; route allocated: 2 of 150; total allocated: 2 of 150]
16:55:14.491 [ Thread-6] : http-outgoing-1 << "[read] I/O error: Read timed out"
16:55:14.491 [ Thread-6] : Connection leased: [id: 1][route: {}->http://127.0.0.1:8080][total kept alive: 1; route allocated: 2 of 150; total allocated: 2 of 150]
16:55:14.491 [ Thread-6] : http-outgoing-1: set socket timeout to 0
16:55:14.492 [ Thread-6] : http-outgoing-1: set socket timeout to 8000
.....
16:55:19.501 [ main] : [leased: 1; pending: 0; available: 1; max: 150]
16:55:19.504 [ Thread-6] : Connection can be kept alive indefinitely
16:55:19.504 [ Thread-6] : Connection [id: 1][route: {}->http://127.0.0.1:8080] can be kept alive indefinitely
16:55:19.505 [ Thread-6] : http-outgoing-1: set socket timeout to 0
16:55:19.505 [ Thread-6] : Connection released: [id: 1][route: {}->http://127.0.0.1:8080][total kept alive: 2; route allocated: 2 of 150; total allocated: 2 of 150]
16:55:24.504 [ main] : [leased: 0; pending: 0; available: 2; max: 150]
#netstat -n | grep tcp4 | grep 8080
tcp4 0 0 127.0.0.1.8080 127.0.0.1.52421 ESTABLISHED
tcp4 0 0 127.0.0.1.8080 127.0.0.1.52420 ESTABLISHED
tcp4 0 0 127.0.0.1.52421 127.0.0.1.8080 ESTABLISHED
tcp4 0 0 127.0.0.1.52420 127.0.0.1.8080 ESTABLISHED
復用了上面的連接，下面是隨后逐步超時的日志。
16:55:39.513 [ main] : [leased: 0; pending: 0; available: 2; max: 150]
16:55:44.491 [ Thread-8] : Closing expired connections
16:55:44.492 [ Thread-8] : Closing connections idle longer than 30 SECONDS
16:55:44.492 [ Thread-8] : http-outgoing-0: Close connection
16:55:44.518 [ main] : [leased: 0; pending: 0; available: 1; max: 150]
....
16:56:09.535 [ main] : [leased: 0; pending: 0; available: 1; max: 150]
16:56:14.499 [ Thread-8] : Closing expired connections
16:56:14.499 [ Thread-8] : Closing connections idle longer than 30 SECONDS
16:56:14.499 [ Thread-8] : http-outgoing-1: Close connection
16:56:14.540 [ main] : [leased: 0; pending: 0; available: 0; max: 150]
分別對應狀態如下，可以看到復用了 52421，隨后 52420 空閑超時被回收，以及最后 52421 也被回收。
#netstat -n | grep tcp4 | grep 8080
tcp4 0 0 127.0.0.1.8080 127.0.0.1.52421 ESTABLISHED
tcp4 0 0 127.0.0.1.52421 127.0.0.1.8080 ESTABLISHED
tcp4 0 0 127.0.0.1.52420 127.0.0.1.8080 TIME_WAIT
...
#netstat -n | grep tcp4 | grep 8080
tcp4 0 0 127.0.0.1.52421 127.0.0.1.8080 TIME_WAIT
最后一個請求后，日志省略，可以看到是新的連接 52443。
netstat -n | grep tcp4 | grep 8080
tcp4 0 0 127.0.0.1.8080 127.0.0.1.52443 ESTABLISHED
tcp4 0 0 127.0.0.1.52443 127.0.0.1.8080 ESTABLISHED