簡單爬取京東,不考慮反爬
簡單實現自動化模擬人爬取京東,當然這爬不了淘寶,淘寶有自動化爬取監測
直接上代碼吧
import time
from selenium import webdriver
from lxml import html
etree = html.etree
# 創建對象
Browner = webdriver.Chrome()
Browner.get('https://www.jd.com/')
# 輸入搜索內容
kw = Browner.find_element_by_id("key")
kw.send_keys('華為手機')
# 點擊
iconfont =Browner.find_element_by_class_name('button')
iconfont.click()
# 滑動至瀏覽器下端
Browner.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(4)
# 獲取網頁源碼
html = Browner.page_source
# 解析獲取數據
h = etree.HTML(html)
l = h.xpath('//ul[@class="gl-warp clearfix"]/li')
# 循環獲取各個對象的數據
for k in l:
price = k.xpath('./div/div[@class="p-price"]//i/text()')
name = k.xpath('./div/div[@class="p-name p-name-type-2"]/a/@title')
#獲取到的名字和價格,打印出來
print(name,price)
原文:https://blog.csdn.net/weixin_44119390/article/details/90966132
