在win10下用pyinstaller打包scrapy項目生成exe文件


參照官方文檔:https://doc.scrapy.org/en/latest/topics/practices.html,認真學習文檔才是正解

1.安裝pyinstaller

2.安裝pywin32

3.安裝其他模塊

4.在爬蟲項目里進行相關操作,參照博友:https://blog.csdn.net/la_vie_est_belle/article/details/79017358

  4.1在scrapy.cfg文件同路徑下創建s_spider.py

  4.2寫入相關代碼

# -*- coding: utf-8 -*-
from scrapy.crawler import CrawlerProcess
from scrapy.utils.project import get_project_settings

# 這里是必須引入的
import robotparser

import scrapy.spiderloader
import scrapy.statscollectors
import scrapy.logformatter
import scrapy.dupefilters
import scrapy.squeues

import scrapy.extensions.spiderstate
import scrapy.extensions.corestats
import scrapy.extensions.telnet
import scrapy.extensions.logstats
import scrapy.extensions.memusage
import scrapy.extensions.memdebug
import scrapy.extensions.feedexport
import scrapy.extensions.closespider
import scrapy.extensions.debug
import scrapy.extensions.httpcache
import scrapy.extensions.statsmailer
import scrapy.extensions.throttle

import scrapy.core.scheduler
import scrapy.core.engine
import scrapy.core.scraper
import scrapy.core.spidermw
import scrapy.core.downloader

import scrapy.downloadermiddlewares.stats
import scrapy.downloadermiddlewares.httpcache
import scrapy.downloadermiddlewares.cookies
import scrapy.downloadermiddlewares.useragent
import scrapy.downloadermiddlewares.httpproxy
import scrapy.downloadermiddlewares.ajaxcrawl
import scrapy.downloadermiddlewares.chunked
import scrapy.downloadermiddlewares.decompression
import scrapy.downloadermiddlewares.defaultheaders
import scrapy.downloadermiddlewares.downloadtimeout
import scrapy.downloadermiddlewares.httpauth
import scrapy.downloadermiddlewares.httpcompression
import scrapy.downloadermiddlewares.redirect
import scrapy.downloadermiddlewares.retry
import scrapy.downloadermiddlewares.robotstxt

import scrapy.spidermiddlewares.depth
import scrapy.spidermiddlewares.httperror
import scrapy.spidermiddlewares.offsite
import scrapy.spidermiddlewares.referer
import scrapy.spidermiddlewares.urllength

import scrapy.pipelines

import scrapy.core.downloader.handlers.http
import scrapy.core.downloader.contextfactory

# 自己項目用到的
#import scrapy.pipelines.images  # 用到圖片管道
import openpyxl  # 用到openpyxl庫

process = CrawlerProcess(get_project_settings())

# 'sk' is the name of one of the spiders of the project.
process.crawl('sk')
process.start()  # the script will block here until the crawling is finished

  4.3在s_spider.py目錄下:shift+右鍵,然后點擊‘’在此處打開命令窗口‘’,輸入:pyinstaller crawl.py,生成dist,build(可刪)和crawl.spec(可刪)。

  4.4在s_spider.exe目錄下創建文件夾scrapy,然后到自己安裝的scrapy文件夾中把VERSION和mime.types兩個文件復制到剛才創建的scrapy文件夾中。

  4.5重新打包運行.exe即可,需要爬蟲支持,不能單獨運行。


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM