代碼：
# nvshens圖片批量下載爬蟲 from bs4 import BeautifulSoup import requests import time import urllib.request user_agent='Mozilla/4.0 (compatible;MEIE 5.5;windows NT)' headers={'User-Agent':user_agent} # 找到的圖片 pictures=[] # 不斷追逐，直到結束 def crawl(url): print("爬取頁面"+url); try: rsp=requests.get(url,headers=headers) soup= BeautifulSoup(rsp.text,'html.parser',from_encoding='utf-8') nextUrl="none"; for divs in soup.find_all(class_="gallery_wrapper"): # 把找到的圖片放到數組里去 for img in divs.find_all('img'): print(img.get("src")) pictures.append(img.get("src")) #找下一頁 for link in divs.find_all('a',class_='a1'): if link.string=='下一頁' and link.get("href").find('.html')!=-1: nextUrl='https://www.nvshens.com'+link.get("href"); if nextUrl!="none": print("前往下一頁"); crawl(nextUrl) else: print('爬取結束，開始下載...') downloadPics() except Exception as e: print("發生異常。重新爬行")# 不管怎么出現的異常，就讓它一直爬到底 crawl(nextUrl) # 下載圖片到本地 def downloadPics(): for pic in pictures: name=pic.split('/')[-1] rsp=urllib.request.urlopen(pic) img=rsp.read() with open(name,'wb') as f: f.write(img) print('圖片'+pic+'下載完成') # Kickoff crawl('https://www.nvshens.com/g/22210/')
輸出示例：
C:\Users\horn1\Desktop\python\7>python downloadall.py 爬取頁面https://www.nvshens.com/g/22210/ C:\Users\horn1\AppData\Local\Programs\Python\Python36\lib\site-packages\bs4\__init__.py:146: UserWarning: You provided Unicode markup but also provided a value for from_encoding. Your from_encoding will be ignored. warnings.warn("You provided Unicode markup but also provided a value for from_encoding. Your from_encoding will be ignored.") https://img.onvshen.com:85/gallery/23789/22210/s/0.jpg https://img.onvshen.com:85/gallery/23789/22210/s/001.jpg https://img.onvshen.com:85/gallery/23789/22210/s/002.jpg https://img.onvshen.com:85/gallery/23789/22210/s/003.jpg https://img.onvshen.com:85/gallery/23789/22210/s/004.jpg 前往下一頁 爬取頁面https://www.nvshens.com/g/22210/2.html https://img.onvshen.com:85/gallery/23789/22210/s/005.jpg https://img.onvshen.com:85/gallery/23789/22210/s/006.jpg https://img.onvshen.com:85/gallery/23789/22210/s/007.jpg https://img.onvshen.com:85/gallery/23789/22210/s/008.jpg https://img.onvshen.com:85/gallery/23789/22210/s/009.jpg 前往下一頁 爬取頁面https://www.nvshens.com/g/22210/3.html https://img.onvshen.com:85/gallery/23789/22210/s/010.jpg https://img.onvshen.com:85/gallery/23789/22210/s/011.jpg https://img.onvshen.com:85/gallery/23789/22210/s/012.jpg https://img.onvshen.com:85/gallery/23789/22210/s/013.jpg https://img.onvshen.com:85/gallery/23789/22210/s/014.jpg 前往下一頁 爬取頁面https://www.nvshens.com/g/22210/4.html https://img.onvshen.com:85/gallery/23789/22210/s/015.jpg https://img.onvshen.com:85/gallery/23789/22210/s/016.jpg https://img.onvshen.com:85/gallery/23789/22210/s/017.jpg https://img.onvshen.com:85/gallery/23789/22210/s/018.jpg https://img.onvshen.com:85/gallery/23789/22210/s/019.jpg 前往下一頁 爬取頁面https://www.nvshens.com/g/22210/5.html https://img.onvshen.com:85/gallery/23789/22210/s/020.jpg https://img.onvshen.com:85/gallery/23789/22210/s/021.jpg https://img.onvshen.com:85/gallery/23789/22210/s/022.jpg https://img.onvshen.com:85/gallery/23789/22210/s/023.jpg https://img.onvshen.com:85/gallery/23789/22210/s/024.jpg 前往下一頁 爬取頁面https://www.nvshens.com/g/22210/6.html https://img.onvshen.com:85/gallery/23789/22210/s/025.jpg https://img.onvshen.com:85/gallery/23789/22210/s/026.jpg https://img.onvshen.com:85/gallery/23789/22210/s/027.jpg https://img.onvshen.com:85/gallery/23789/22210/s/028.jpg https://img.onvshen.com:85/gallery/23789/22210/s/029.jpg 前往下一頁 爬取頁面https://www.nvshens.com/g/22210/7.html https://img.onvshen.com:85/gallery/23789/22210/s/030.jpg https://img.onvshen.com:85/gallery/23789/22210/s/031.jpg https://img.onvshen.com:85/gallery/23789/22210/s/032.jpg https://img.onvshen.com:85/gallery/23789/22210/s/033.jpg https://img.onvshen.com:85/gallery/23789/22210/s/034.jpg 前往下一頁 爬取頁面https://www.nvshens.com/g/22210/8.html https://img.onvshen.com:85/gallery/23789/22210/s/035.jpg https://img.onvshen.com:85/gallery/23789/22210/s/036.jpg https://img.onvshen.com:85/gallery/23789/22210/s/037.jpg https://img.onvshen.com:85/gallery/23789/22210/s/038.jpg https://img.onvshen.com:85/gallery/23789/22210/s/039.jpg 爬取結束，開始下載... 圖片https://img.onvshen.com:85/gallery/23789/22210/s/0.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/001.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/002.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/003.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/004.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/005.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/006.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/007.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/008.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/009.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/010.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/011.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/012.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/013.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/014.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/015.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/016.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/017.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/018.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/019.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/020.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/021.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/022.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/023.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/024.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/025.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/026.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/027.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/028.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/029.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/030.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/031.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/032.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/033.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/034.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/035.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/036.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/037.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/038.jpg下載完成 圖片https://img.onvshen.com:85/gallery/23789/22210/s/039.jpg下載完成
感覺Python爬蟲是比Nodejs爬蟲省事一些。