6.簡單提取小紅書app數據保存txt-2

對頁面信息進行簡單抓取：

需要注意的問題 ：

auth-sign 和 auth 都是有一定的時效性，還有url原url是https這里要改為http請求。

這參數的問題需要通過mitmdump去獲取請求的具體參數並將之取出，不用手動去截獲分析http請求和響應，寫好請求和相應的處理邏輯，通過python實現二次操作。

后期通過appium模擬人為操作去滑動請求刷新界面，得到相應再做處理。

import requests

def main():
    headers = {
    "charset":"utf-8",
    "Accept-Encoding":"gzip",
    "referer":"https://servicewechat.com/wxffc08ac7df482a27/117/page-frame.html",
    "authorization":"5bda7657a4ce660001f7eed8",
    "auth":"eyJoYXNoIjoibWQ0IiwiYWxnIjoiSFMyNTYiLCJ0eXAiOiJKV1QifQ.eyJzaWQiOiI0M2RkNGY2YS01NTk1LTRjNGEtYTkyMi05ODEzNjdiMTlmMTEiLCJleHBpcmUiOjE1NDExMzAyNjJ9.9AC8VBcXiBG48vHa-LLgVEWOnloTdQvNWzYAyvqGnMA",
    "content-type":"application/json",
    "auth-sign":"c475525b214bb5d9ae431ac029cb9b50",
    "User-Agent":"Mozilla/5.0 (Linux; Android 7.1.2; MI 5X Build/N2G47H; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/64.0.3282.137 Mobile Safari/537.36 MicroMessenger/6.7.3.1360(0x26070336) NetType/WIFI Language/zh_CN Process/appbrand2",
    "Host":"www.xiaohongshu.com",
    "Connection":"Keep-Alive",
    }
    # url = "http://www.xiaohongshu.com/sapi/wx_mp_api/sns/v1/homefeed?oid=homefeed.cosmetics_v2&cursor_score=&sid=session.1540996623416187718"
    url = "http://www.xiaohongshu.com/sapi/wx_mp_api/sns/v1/homefeed?oid=homefeed.cosmetics_v2&cursor_score=1541067389.9550&sid=session.1540996623416187718"


    datas = requests.get(url= url, headers=headers ).json()
    data = datas['data']
    # print(data)
    for i in data:
        print(i)
        # print(i['title'])
        # print(i['share_link'])
        title = '標題: ' + i['mini_program_info']['share_title']
        print(title)
        link_url = '鏈接: ' + i['share_link']
        print(link_url)
        b_picture = '封面圖片: '+ i['mini_program_info']['thumb']
        print(b_picture)
        type = '類型: ' + i['type']
        print(type)
        level = '級別: ' + str(i['level'])
        print(level)
        h_picture = '用戶頭像: ' + i['user']['images']
        print(h_picture)
        username = '用戶名: ' + i['user']['nickname']
        print(username)
        user_id = 'userid: ' + i['user']['userid']
        print(user_id)
        zan = '喜歡點心: ' + str(i['likes'])
        print(zan)

        # 以追加的方式及打開一個文件，文件指針放在文件結尾，追加讀寫！
        with open('text', 'a', encoding='utf-8')as f:
            f.write('\n'.join([title,link_url,b_picture,type,level,h_picture,username,user_id,zan]))
            f.write('\n' + '=' * 100 + '\n')
if __name__ == "__main__":
    main()

 

保存本地

 

 

字段信息：

標題: 王者榮耀——貂蟬～仲夏夜之夢 游戲角色貂蟬皮膚印象妝容 主色
鏈接: https://www.xiaohongshu.com/discovery/item/5bc0b2bf910cf646cc1087aa
封面圖片: http://ci.xiaohongshu.com/161f03cb-0cf6-355f-b178-712a928a7720?imageView2/2/w/540/format/jpg
類型: normal
級別: 4
用戶頭像: https://img.xiaohongshu.com/avatar/5bb1047b0fd0590001997f83.jpg@80w_80h_90q_1e_1c_1x.jpg
用戶名: zanleo
userid: 582c5f8982ec393b5ec866ba
喜歡點心: 233
====================================================================================================
標題: 💜仲夏夜之紫妝 | HUDA beauty 沙漠黃昏教程
鏈接: https://www.xiaohongshu.com/discovery/item/5bc9e121672e144fac0d3438
封面圖片: http://ci.xiaohongshu.com/29b82aa1-ad20-355c-9d42-396ddf52e5d6?imageView2/2/w/540/format/jpg
類型: normal
級別: 2
用戶頭像: https://img.xiaohongshu.com/avatar/5a296015d2c8a51d5734be82.jpg@80w_80h_90q_1e_1c_1x.jpg
用戶名: Miya楊奶奶_
userid: 558b9f43a75c956c2accf4cf
喜歡點心: 211
====================================================================================================
標題: 6款熱門平價粉底上臉測評🌟🌟 到底該選哪款？ 平價粉底到
鏈接: https://www.xiaohongshu.com/discovery/item/5bd1ab2a07ef1c2e707bf66c
封面圖片: http://ci.xiaohongshu.com/7d86d4bc-1170-524b-8566-2a7ea4e37843?imageView2/2/w/540/format/jpg
類型: video
級別: 4
用戶頭像: https://img.xiaohongshu.com/avatar/5b20ca00b46c5d4130aad5f9.jpg@80w_80h_90q_1e_1c_1x.jpg
用戶名: 喵格singherC
userid: 5aa65f6411be10488ded22ed
喜歡點心: 945
====================================================================================================
標題: 萬聖節妝容🎃超簡單|3步搞定星空版的zipper face
鏈接: https://www.xiaohongshu.com/discovery/item/5bd6882907ef1c7693a1b241
封面圖片: http://ci.xiaohongshu.com/7ccd594d-e525-502c-b9bb-4c430158af3c?imageView2/2/w/540/format/jpg
類型: normal
級別: 2
用戶頭像: https://img.xiaohongshu.com/avatar/5ab3bc0d14de410bdfa82234.jpg@80w_80h_90q_1e_1c_1x.jpg
用戶名: 達寶Linda
userid: 5a9e51c2e8ac2b2b796a7b58
喜歡點心: 24
====================================================================================================
標題: 傻瓜式眼線畫法✔️簡單三步畫出流暢眼線 最近南南收到很多寶寶
鏈接: https://www.xiaohongshu.com/discovery/item/5bcd2635910cf646df155c0d
封面圖片: http://ci.xiaohongshu.com/9d8408ff-0518-5051-93cc-d0f8e8b46ba8?imageView2/2/w/540/format/jpg
類型: video
級別: 4
用戶頭像: https://img.xiaohongshu.com/avatar/5aba0b9ab46c5d273ecf8e9d.jpg@80w_80h_90q_1e_1c_1x.jpg
用戶名: 一枝南南
userid: 5604f1e6e4b1cf3ec7c681aa
喜歡點心: 379
====================================================================================================
標題: 🌟萬聖節妝容| 受傷小精靈妝 不恐怖 仙仙噠✨ - 萬聖節
鏈接: https://www.xiaohongshu.com/discovery/item/5bd0e707910cf646de1ea5c4
封面圖片: http://ci.xiaohongshu.com/cada288c-f792-5d6c-a9c0-da3ada8f7dc9?imageView2/2/w/540/format/jpg
類型: normal
級別: 2
用戶頭像: https://img.xiaohongshu.com/avatar/5b8a768e9042e3000127ee7c.jpg@80w_80h_90q_1e_1c_1x.jpg
用戶名: 原口元子
userid: 584279446a6a697c18b0fd20
喜歡點心: 305
====================================================================================================
標題: 和我一起過萬聖節🎃暗黑系御姐妝容【視頻教程
鏈接: https://www.xiaohongshu.com/discovery/item/5bd6c20b910cf63164681086
封面圖片: http://ci.xiaohongshu.com/c1e67a16-ba72-5095-a205-5372d5ffc4b2?imageView2/2/w/540/format/jpg
類型: video
級別: 2
用戶頭像: https://img.xiaohongshu.com/avatar/5b9bb566e277db00012cae1c.jpg@80w_80h_90q_1e_1c_1x.jpg
用戶名: 球大王
userid: 55e96f24a75c950acd3358b8
喜歡點心: 119
====================================================================================================
標題: 90%的女生都不知道的鼻影正確畫法
鏈接: https://www.xiaohongshu.com/discovery/item/5bd3ce20672e143bd2c40c98
封面圖片: http://ci.xiaohongshu.com/8e67c937-12d7-5f90-a905-2211fa40c620?imageView2/2/w/540/format/jpg
類型: video
級別: 2
用戶頭像: https://img.xiaohongshu.com/avatar/5b3de03ad2c8a52d01693211.jpg@80w_80h_90q_1e_1c_1x.jpg
用戶名: 愛美妝的雪禾子
userid: 5b3de01911be10724c823add
喜歡點心: 140
====================================================================================================
標題: @趙奕歡Chloe 發了一篇超贊的筆記，快點來看！
鏈接: https://www.xiaohongshu.com/discovery/item/5bc55241910cf646d416c55a
封面圖片: http://ci.xiaohongshu.com/172c29bd-0ba2-5753-bdce-e4f1d98bde67?imageView2/2/w/540/format/jpg
類型: video
級別: 4
用戶頭像: https://img.xiaohongshu.com/avatar/5ac38c99d2c8a5130c1e4f7a.jpg@80w_80h_90q_1e_1c_1x.jpg
用戶名: 趙奕歡Chloe
userid: 5aa12b5311be107df912efb4
喜歡點心: 2557
====================================================================================================
標題: 黃黑皮 塗了也顯白的豆沙色！ 最滋潤的口紅 阿瑪尼唇釉試色
鏈接: https://www.xiaohongshu.com/discovery/item/5bc71ffa910cf646d813008d
封面圖片: http://ci.xiaohongshu.com/c153caab-44b8-5121-9e49-abc2e7334ae9?imageView2/2/w/540/format/jpg
類型: video
級別: 2
用戶頭像: https://img.xiaohongshu.com/avatar/5ba395e0e582ff0001888aac.jpg@80w_80h_90q_1e_1c_1x.jpg
用戶名: 認真少女_顏九
userid: 5a52d211e8ac2b78a241269e
喜歡點心: 4926
====================================================================================================