簡單爬取京東,不考慮反爬
簡單實現自動化模擬人爬取京東,當然這爬不了淘寶,淘寶有自動化爬取監測
直接上代碼吧
import time from selenium import webdriver from lxml import html etree = html.etree # 創建對象 Browner = webdriver.Chrome() Browner.get('https://www.jd.com/') # 輸入搜索內容 kw = Browner.find_element_by_id("key") kw.send_keys('華為手機') # 點擊 iconfont =Browner.find_element_by_class_name('button') iconfont.click() # 滑動至瀏覽器下端 Browner.execute_script("window.scrollTo(0, document.body.scrollHeight);") time.sleep(4) # 獲取網頁源碼 html = Browner.page_source # 解析獲取數據 h = etree.HTML(html) l = h.xpath('//ul[@class="gl-warp clearfix"]/li') # 循環獲取各個對象的數據 for k in l: price = k.xpath('./div/div[@class="p-price"]//i/text()') name = k.xpath('./div/div[@class="p-name p-name-type-2"]/a/@title') #獲取到的名字和價格,打印出來 print(name,price)
原文:https://blog.csdn.net/weixin_44119390/article/details/90966132