Httpclient 使用和性能測試
上篇，通過簡介和架構圖，我們對HttpClient有了初步的了解。
本篇我們展示HttpClient的簡單使用，同時為了說明httpclient的使用性能，我們將Httpclient的同步和異步模式與apache的Httpclient4作比較。。
1. HttpClient示例代碼
以下基本是官方示例，分別展示了如何使用Get和Post請求。
HttpClient client = HttpClient.newBuilder()
.version(Version.HTTP_1_1) //可以手動指定客戶端的版本，如果不指定，那么默認是Http2
.followRedirects(Redirect.NORMAL) //設置重定向策略
.connectTimeout(Duration.ofSeconds(20)) //連接超時時間
.proxy(ProxySelector.of(new InetSocketAddress("proxy.example.com", 80))) //代理地址設置
.authenticator(Authenticator.getDefault())
//.executor(Executors.newFixedThreadPoolExecutor(8)) //可手動配置線程池
.build();
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("https://foo.com/")) //設置url地址
.GET()
.build();
HttpResponse<String> response = client.send(request, BodyHandlers.ofString()); //同步發送
System.out.println(response.statusCode()); //打印響應狀態碼
System.out.println(response.body());
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("https://foo.com/"))
.timeout(Duration.ofMinutes(2)) //設置連接超時時間
.header("Content-Type", "application/json")
.POST(BodyPublishers.ofFile(Paths.get("file.json"))) //設置請求體來源
.build();
client.sendAsync(request, BodyHandlers.ofString()) //異步發送
.thenApply(HttpResponse::body) //發送結束打印響應體
.thenAccept(System.out::println);
可以看到，應用編寫的代碼相對流暢自然。不過，也有幾個注意點
- Http連接池不支持手動配置，默認是無限復用的
- 重試次數不支持手動配置
- 不指定Http客戶端或請求的版本，會默認使用Http2模式進行連接，受挫后會進行降級
- 請求的同步發送模式（send）實際上會后台另開線程
短短的幾行代碼只是實現了功能，那么，它的性能如何呢？我們把它和業界標桿——Apache 的HttpClient作對比。
2. 服務器測試代碼編寫
為了簡便，使用node.js的http模塊運行一個簡易的服務器。該服務器駐守在8080端口，每收到一個請求，停留500ms后返回響應。
let http = require("http")
let server = http.createServer()
server.addListener("request", (req, res) => {
if (req.url.startsWith("/")) {
//接到任意請求，停留0.5秒后返回
setTimeout(() => {
res.end("haha")
}, 500)
}
}
)
server.listen(8080, () => console.log("啟動成功！"))
使用node運行該js文件，提示已啟動成功
3. JDK httpclient 和apache Httpclient 測試代碼
首先定義公共的測試接口：
public interface Tester {
//測試參數
class TestCommand {
}
/**
* 測試主方法
* @param testCommand 測試參數
*/
void test(TestCommand testCommand) throws Exception;
/**
* 重復測試多次
* @param testName 測試名稱
* @param times 測試次數
* @param testCommand 每次測試的參數
*/
default void testMultipleTimes(String testName, int times, TestCommand testCommand) throws Exception{
long startTime = System.currentTimeMillis();
System.out.printf(" ----- %s開始，共%s次 -----\n", testName, times);
for (int i = 0; i < times; i++) {
long currentStartTime = System.currentTimeMillis();
test(testCommand);
System.out.printf("第%s次測試用時：%sms\n", i + 1, (System.currentTimeMillis() - currentStartTime));
}
long usedTime = System.currentTimeMillis() - startTime;
System.out.printf("%s次測試共用時:%sms,平均用時：%sms\n", times, usedTime, usedTime / times);
}
}
定義測試類，包含三個靜態嵌套類，分別用作JDK httpclient的異步模式、同步模式和apache Httpclient的同步模式
public class HttpClientTester {
/** Http請求的真正測試參數*/
static class HttpTestCommand extends Tester.TestCommand {
/**目的url*/
String url;
/**單次測試請求次數*/
int requestTimes;
/**請求線程數*/
int threadCount;
public HttpTestCommand(String url, int requestTimes, int threadCount) {
this.url = url;
this.requestTimes = requestTimes;
this.threadCount = threadCount;
}
}
static class BlocklyHttpClientTester implements Tester {
@Override
public void test(TestCommand testCommand) throws Exception {
HttpTestCommand httpTestCommand = (HttpTestCommand) testCommand;
testBlockly(httpTestCommand.url, httpTestCommand.requestTimes,httpTestCommand.threadCount);
}
/**
* 使用JDK Httpclient的同步模式進行測試
* @param url 請求的url
* @param times 請求次數
* @param threadCount 開啟的線程數量
* @throws ExecutionException
* @throws InterruptedException
*/
void testBlockly(String url, int times, int threadCount) throws ExecutionException, InterruptedException {
threadCount = threadCount <= 0 ? 1 : threadCount;
ExecutorService executorService = Executors.newFixedThreadPool(threadCount);
HttpClient client = HttpClient.newBuilder().build();
Callable<String> callable1 = () -> {
HttpRequest request = HttpRequest.newBuilder(URI.create(url)).GET().build();
HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
return response.body();
};
List<Future<String>> futureList1 = new ArrayList<>();
for (int i = 0; i < times; i++) {
Future<String> future1 = executorService.submit(callable1);
futureList1.add(future1);
}
for (Future<String> stringFuture : futureList1) {
//阻塞直至所有請求返回
String s = stringFuture.get();
}
executorService.shutdown();
}
}
static class NonBlocklyHttpClientTester implements Tester {
@Override
public void test(TestCommand testCommand) throws Exception {
HttpTestCommand httpTestCommand = (HttpTestCommand) testCommand;
testNonBlockly(httpTestCommand.url, httpTestCommand.requestTimes);
}
/**
* 使用JDK Httpclient的異步模式進行測試
* @param url 請求的url
* @param times 請求次數
* @throws InterruptedException
*/
void testNonBlockly(String url, int times) throws InterruptedException {
//給定16個線程，業務常用 2 * Runtime.getRuntime().availableProcessors()
ExecutorService executor = Executors.newFixedThreadPool(16);
HttpClient client = HttpClient.newBuilder()
.executor(executor)
.build();
//使用倒計時鎖來保證所有請求完成
CountDownLatch countDownLatch = new CountDownLatch(times);
HttpRequest request = HttpRequest.newBuilder(URI.create(url)).GET().build();
while (times-- >= 0) {
client.sendAsync(request, HttpResponse.BodyHandlers.ofString())
.whenComplete((stringHttpResponse, throwable) -> {
if (throwable != null) {
throwable.printStackTrace();
}
if (stringHttpResponse != null) {
stringHttpResponse.body();
}
countDownLatch.countDown();
});
}
//阻塞直至所有請求完成
countDownLatch.await();
executor.shutdown();
}
}
static class ApacheHttpClientTester implements Tester {
@Override
public void test(TestCommand testCommand) throws Exception {
HttpTestCommand httpTestCommand = (HttpTestCommand) testCommand;
testBlocklyWithApacheClient(httpTestCommand.url, httpTestCommand.requestTimes,httpTestCommand.threadCount);
}
/**
* 使用Apache HttpClient進行測試
* @param url 請求的url
* @param times 使用時長
* @param threadCount 開啟的線程數量
* @throws ExecutionException
* @throws InterruptedException
*/
void testBlocklyWithApacheClient(String url, int times, int threadCount) throws ExecutionException, InterruptedException {
threadCount = threadCount <= 0 ? 1 : threadCount;
ExecutorService executorService = Executors.newFixedThreadPool(threadCount);
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
//設置apache Httpclient連接復用無限制，體現其最大性能
connectionManager.setDefaultMaxPerRoute(Integer.MAX_VALUE);
connectionManager.setMaxTotal(Integer.MAX_VALUE);
CloseableHttpClient httpClient = HttpClientBuilder.create().setConnectionManager(connectionManager).build();
Callable<String> callable1 = () -> {
HttpGet httpGet = new HttpGet(url);
CloseableHttpResponse response = httpClient.execute(httpGet);
return EntityUtils.toString(response.getEntity());
};
List<Future<String>> futureList1 = new ArrayList<>();
for (int i = 0; i < times; i++) {
Future<String> future1 = executorService.submit(callable1);
futureList1.add(future1);
}
for (Future<String> stringFuture : futureList1) {
//阻塞直至所有請求返回
String s = stringFuture.get();
}
executorService.shutdown();
}
}
測試的main方法：
public static void main(String[] args) {
try {
//
HttpTestCommand testCommand = new HttpTestCommand("http://localhost:8080", 200, 16);
//每個測試重復3輪，減少誤差
final int testTimes = 3;
new BlocklyHttpClientTester().testMultipleTimes("JDK HttpClient同步模式測試", testTimes, testCommand);
new NonBlocklyHttpClientTester().testMultipleTimes("JDK HttpClient異步模式測試", testTimes, testCommand);
new ApacheHttpClientTester().testMultipleTimes("Apache Httpclient同步模式測試", testTimes, testCommand);
} catch (Exception e) {
e.printStackTrace();
}
}
4. 測試結果
----- JDK HttpClient同步模式測試開始，共3次 -----
第1次測試用時：4414ms
第2次測試用時：3580ms
第3次測試用時：3620ms
3次測試共用時:11620ms,平均用時：3873ms
----- JDK HttpClient異步模式測試開始，共3次 -----
第1次測試用時：568ms
第2次測試用時：595ms
第3次測試用時：579ms
3次測試共用時:1742ms,平均用時：580ms
----- Apache Httpclient同步模式測試開始，共3次 -----
第1次測試用時：3719ms
第2次測試用時：3557ms
第3次測試用時：3574ms
3次測試共用時:10851ms,平均用時：3617ms
可見，Httpclient同步模式與apacheHttpclient同步模式性能接近；異步模式由於充分利用了nio非阻塞的特性，在線程數相同的情況下，效率大幅優於同步模式。
需要注意的是，此處的“同步”“異步”並非I/O模型中的同步，而是指編程方式上的同步/異步。
5. 總結
通過以上示例代碼，可以看出HttpClient具有編寫流暢、性能優良的特點，也有可定制性不足的遺憾。
下一節，我們將深入客戶端的構建和啟動過程，接觸選擇器管理者這一角色，探尋它和Socket通道的交互的交互過程。