C#抓取網面上的html內容(JS動態生成的無法抓取)


抓取內容的代碼:

 1         /// </summary>
 2         /// <param name="url">路徑URL</param>
 3         /// <param name="path">存儲路徑</param>
 4         /// <returns></returns>
 5         public static string HttpDownloadFile(string url, string path)
 6         {
 7             try
 8             {
 9                 // 設置參數
10                 HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
11 
12                 //發送請求並獲取相應回應數據
13                 HttpWebResponse response = request.GetResponse() as HttpWebResponse;
14                 //直到request.GetResponse()程序才開始向目標網頁發送Post請求
15                 Stream responseStream = response.GetResponseStream();
16 
17                 //可以將生成的流文件直接生成字符串 SourceCode就是生成后的字符串
18                 //StreamReader readStream = new StreamReader(responseStream, Encoding.UTF8);
19                 //string SourceCode = readStream.ReadToEnd();
20 
21                 //創建本地文件寫入流
22                 if (File.Exists(path))
23                 {
24                     File.Delete(path);
25                 }
26                 FileStream fs = File.Create(path);
27                 fs.Close();
28                 
29                 Stream stream = new FileStream(path, FileMode.Create);
30                 byte[] bArr = new byte[1024];
31                 int size = responseStream.Read(bArr, 0, (int)bArr.Length);
32                 while (size > 0)
33                 {
34                     stream.Write(bArr, 0, size);
35                     size = responseStream.Read(bArr, 0, (int)bArr.Length);
36                 }
37                 stream.Close();
38                 responseStream.Close();
39                 return path;
40             }
41             catch (Exception ex)
42             {
43 
44                 throw ex;
45             }
46 
47         }

 

調用方式:

1                 HttpReviceFile.HttpDownloadFile("http://localhost:811/ ", @"D:\Work\Test.xml");

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM