爬取微信公眾號

本文轉載自查看原文 2022-08-03 15:23 651 爬蟲/ 數據開發

技術准備

HttpClient

Java比較常用的發起請求的工具，功能有：

方便的發起get、post等請求
可以設置連接池（類似線程池），使用池化思想降低頻繁創建連接的開銷
可以自己編寫代碼設置多線程爬取
...

代碼模板

 public static void main(String[] args) throws Exception {
        CloseableHttpClient httpClient = HttpClients.createDefault();
        URIBuilder uriBuilder = new URIBuilder("https://mp.weixin.qq.com/cgi-bin/bizlogin");
        HttpPost httpPost = new HttpPost(uriBuilder.build());

        httpPost.addHeader("User-Agent","Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36");

        List<NameValuePair> params = new ArrayList<>();
        params.add(new BasicNameValuePair("action","startlogin"));

        UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(params,"utf8");
        httpPost.setEntity(formEntity);

        CloseableHttpResponse response = null;
        try {
            response = httpClient.execute(httpPost);
            if(response.getStatusLine().getStatusCode() == 200){
                String content = EntityUtils.toString(response.getEntity(),"utf8");
                System.out.println(content);
            }
        }catch (IOException e){
            e.printStackTrace();
        }finally {
            try {
                response.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            try {
                httpClient.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

以上是post請求的代碼模板，get等其他請求也差不太多，將HttpPost換成HttpGet就行了