通常情況下用selenium開發爬蟲時,會使用以下方式添加UA
chrome_options.add_argument('user-agent=' + self.ua)
但是每次更換需要重啟瀏覽器,這會造成資源浪費。好在selenium支持CDP,於是翻了文檔終於找到了解決方法。解決方法如下
driver.execute_cdp_cmd("Emulation.setUserAgentOverride", { "userAgent": "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20100101 Firefox/6.0" })
執行后打開控制台,輸入navigator.userAgent查看,會發現userAgent已經改變,訪問任何頁面都會是目前設置的
同時官方文檔中還有以下參數可以使用:
{ "name": "setUserAgentOverride", "description": "Allows overriding user agent with the given string.", "redirect": "Emulation", "parameters": [ { "name": "userAgent", "description": "User agent to use.", "type": "string" }, { "name": "acceptLanguage", "description": "Browser langugage to emulate.", "optional": true, "type": "string" }, { "name": "platform", "description": "The platform navigator.platform should return.", "optional": true, "type": "string" }, { "name": "userAgentMetadata", "description": "To be sent in Sec-CH-UA-* headers and returned in navigator.userAgentData", "experimental": true, "optional": true, "$ref": "Emulation.UserAgentMetadata" } ] }