抓取登錄后的數據

    這次是應一個客戶需要，抓取另外一個網站的數據，包括數據提交。這些操作需要在登錄之后完成。技術上沒有什么難點。關鍵都是用fiddler找到參數和url。

記住登錄狀態

    HttpClient能夠記住登錄狀態的，登錄完了可以講Httpclient保存起來。

 private HttpClient _client;
        public HttpClient HttpClient
        {
            get
            {
                if (_client == null)
                {
                    if (Session["Client"]!= null)
                    {
                        _client = Session["Client"] as HttpClient;
                    }
                    else
                    {
                        var handler = new HttpClientHandler
                        {
                            AutomaticDecompression = DecompressionMethods.GZip,
                            UseCookies = true,
                            Proxy =
                                new WebProxy("http://ip:8080/", true, null,
                                    new NetworkCredential("username", "pwd", "domain"))
                        };//代理 
                        _client = new HttpClient(handler);
                        _client.DefaultRequestHeaders.Add("user-agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36");
                        ClientLogin(new ClientLogoModel());
                        Session["Client"] = _client;
                    }
                  
                }

                return _client;
            }
        }

 因為目標網站都是用的json傳的參數。也是用json返回的參數。不是form提交的格式。所以post之前也要將參數轉成json。

 public object ClientLogin(ClientLogoModel logoModel)
        {
            if (logoModel == null)
            {
                logoModel=new ClientLogoModel();

            }
            var data = JsonConvert.SerializeObject(logoModel); ;
            var logoParams = new List<KeyValuePair<string, string>>();
            logoParams.Add(new KeyValuePair<string, string>("data", data));
            var response = _client.PostAsync(new Uri(LogonUrl), new FormUrlEncodedContent(logoParams)).Result;
            var result = response.Content.ReadAsStringAsync().Result;
            return result;
        }

返回數據轉化

從Fiddler左邊獲得Url，右邊TextView上方是參數格式，下方是返回的數據格式。 

每次都要轉換，寫成泛型。

    public T GetTList<T>(object obj, string url)
        {
            var data = JsonConvert.SerializeObject(obj); ;
            var paramList = new List<KeyValuePair<String, String>> { new KeyValuePair<string, string>("data", data) };
            var response = HttpClient.PostAsync(new Uri(url), new FormUrlEncodedContent(paramList)).Result;

            var result = response.Content.ReadAsStringAsync().Result;
            return JsonConvert.DeserializeObject<T>(result);
        }

調用：

    public ActionResult TradePage(TradeQueryParm param)
        {
       var data = GetTList<TradeRequstResult>(obj, tradeListUrl);
            return PartialView(data);
        }

 前端再將參數傳遞過來。

  $.post("/Trade/TradePage", {
                    agentName: agentName, shortName: shortName,
                    startDate: startDate, endDate: endDate, page: cpage
                }, function (data) {
                    $("#mtable").html(data);
  }

HttpClient 上傳圖片：

   private string UploadImage(string fileName,string path)
        {
            FileStream aFile = new FileStream(path, FileMode.Open);
            MultipartFormDataContent form = new MultipartFormDataContent();
            var  content = new StreamContent(aFile);
            content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
            content.Headers.ContentDisposition = new ContentDispositionHeaderValue("form-data")
            {
                Name = "protocolFile",
                FileName = fileName
            };
            form.Add(content);
            var response = HttpClient.PostAsync(imgLoadUrl, form).Result;
            return response.Content.ReadAsStringAsync().Result;
        }