原文網址：https://www.cjavapy.com/article/723/
本文主要介紹.NET Core(C#)中，使用HttpClientFactory(IHttpClientFactory)實現簡單爬蟲，執行GET和POST請求的方法，以及相關的示例代碼。
1､.NET Core項目中注冊HttpClientFactory
1) ASP .NET Core項目
在Startup.cs中ConfigureService方法中注冊HttpClient，代碼如下，
services.AddHttpClient();
2) .NET Core項目
.NET Core項目中需要安裝引用Microsoft.Extensions.Http和Microsoft.Extensions.Hosting，注冊代碼如下，
static void Main(string[] args) { var builder = Host.CreateDefaultBuilder(args); builder.ConfigureServices(x => { x.AddHttpClient(); }); Console.ReadKey(); }
2､配置HTTP請求頭Header
1) 注冊HttpClient時配置
services.AddHttpClient();
services.AddHttpClient("cjavapy", c =>
{
c.BaseAddress = new Uri("https://api.cjavapy.com/"); c.DefaultRequestHeaders.Add("content-type","application/json"); c.DefaultRequestHeaders.Add("User-Agent","Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36 MicroMessenger/7.0.9.501 NetType/WIFI MiniProgramEnv/Windows WindowsWechat"); c.DefaultRequestHeaders.Add("Referer",""); });
或者
static void Main(string[] args) { var builder = Host.CreateDefaultBuilder(args); builder.ConfigureServices(x => { x..AddHttpClient("cjavapy", c => { c.BaseAddress = new Uri("https://api.cjavapy.com/"); c.DefaultRequestHeaders.Add("content-type","application/json"); c.DefaultRequestHeaders.Add("User-Agent","Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36 MicroMessenger/7.0.9.501 NetType/WIFI MiniProgramEnv/Windows WindowsWechat"); c.DefaultRequestHeaders.Add("Referer",""); }); }); Console.ReadKey(); }
2) 執行請求之前配置
var httpClient = context.ServiceProvider.GetRequiredService<IHttpClientFactory>().CreateClient();
httpClient.BaseAddress = new Uri("https://api.cjavapy.com/"); httpClient.DefaultRequestHeaders.Add("content-type","application/json"); httpClient.DefaultRequestHeaders.Add("User-Agent","Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36 MicroMessenger/7.0.9.501 NetType/WIFI MiniProgramEnv/Windows WindowsWechat"); httpClient.DefaultRequestHeaders.Add("Referer","");
3､使用HttpClientFactory(IHttpClientFactory)執行GET和POST請求
命名空間：
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Reflection;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
1) GET請求獲取網頁代碼
static void Main(string[] args) { var builder = Host.CreateDefaultBuilder(args); builder.ConfigureServices(x => { x.AddHttpClient(); }); using (var scope = builder.Build().Services.CreateScope()) { var httpClient = scope.ServiceProvider.GetRequiredService<IHttpClientFactory>().CreateClient(); httpClient.BaseAddress = new Uri("https://www.baidu.com"); httpClient.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36 MicroMessenger/7.0.9.501 NetType/WIFI MiniProgramEnv/Windows WindowsWechat"); httpClient.DefaultRequestHeaders.Add("Referer", "https://www.baidu.com"); Console.WriteLine(httpClient.GetAsync("/").Result.Content.ReadAsStringAsync().Result); } Console.ReadKey(); }
2) POST請求提交數據
static void Main(string[] args) { var builder = Host.CreateDefaultBuilder(args); builder.ConfigureServices(x => { x.AddHttpClient(); }); using (var scope = builder.Build().Services.CreateScope()) { var httpClient = scope.ServiceProvider.GetRequiredService<IHttpClientFactory>().CreateClient(); httpClient.BaseAddress = new Uri("apiUrl"); httpClient.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36 MicroMessenger/7.0.9.501 NetType/WIFI MiniProgramEnv/Windows WindowsWechat"); //httpClient.DefaultRequestHeaders.Add("Referer", "https://www.baidu.com"); var result=""; try { //2 minute timeout on wait for response httpClient.Timeout = new TimeSpan(0, 2, 0); //Create an HttpRequestMessage object and pass it into SendAsync() HttpRequestMessage message = new HttpRequestMessage(); message.Headers.Add("Accept", "application/json"); message.Content = new StringContent("{\"user\":\"11\"}", System.Text.Encoding.UTF8, "application/json"); message.Method = HttpMethod.Post; message.RequestUri = new Uri(httpClient.BaseAddress.ToString() + "someapiendpoint"); HttpResponseMessage response = httpClient.SendAsync(message).Result; result = response.Content.ReadAsStringAsync().Result; //deserialize the result into proper object type } catch (Exception ex) { //Log exception } } Console.ReadKey(); }
相關文檔：
ASP.NET Core 2.1 HttpClientFactory使用的3種方法
.NET(C#)使用HttpClient請求JSON數據的示例代碼
.NET Core(C#) RestSharp GET和POST請求、下載大文件及cookie管理
.NET(C#) Fluent HTTP (Flurl Get和Post請求)使用方法及示例代碼