本文轉載自以下鏈接：https://github.com/Light-City/AutoDownloadWhl
源碼地址：
本次腳本我將它命名為：Whl自助搜索下載器
實現了如下功能：
- 支持檢索
- 支持查找
- 支持下載
- 單、多文件下載
2.如何使用及功能演示
2.1 如何使用
運行腳本需要：
- requests、lxml、os庫
- curl或axel命令
win系統的直接下載win版本的按照網上配置即可！
linux系統與mac系統可用yum 或 apt 或 brew等來進行安裝！
復制源碼地址並下載腳本，運行腳本即可：
python AutoDownloadWhl.py
2.2 功能演示
- 所有包信息展示
- 適應平台whl包
- 多包下載
3.代碼詳解
3.1 封裝
封裝包地址與下載地址：
class AutoDownWhl(): def __init__(self): # 包地址 self.url = 'https://www.lfd.uci.edu/~gohlke/pythonlibs/' # 下載地址 self.base_url = 'https://download.lfd.uci.edu/pythonlibs/r5uhg2lo/' # 模擬瀏覽器 self.headers = { 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36' }
3.2 獲取選擇器與包名
解析源碼與獲取包名：
# 獲取選擇器與包名 def getLib(self): raw_text = requests.get(self.url, headers=self.headers).content seletor = etree.HTML(raw_text) lib_names = seletor.xpath('//ul[@class="pylibs"]//li//strong//text()') return seletor, lib_names
3.3 輸出所有包名
一行15個包名打印：
# 輸出所有包名 def print_AllLib(self, lib_names): print('\n------------------共查找到' + str(len(lib_names)) + '個包------------------\n') for i in range(len(lib_names)): if i and i % 15 == 0: print('\n') print(lib_names[i], end=' ')
3.4 搜索與下載
只摘取了核心代碼，下載采用調用系統命令curl與axel，請在自己的系統上安裝這兩個其中之一下載！
def searchDown(self, downloadMethod='axel'): # 此部分代碼只摘取核心部分 if int(lib_sequence) <= max: detail_name = detail_libs[int(lib_sequence) - 1].strip().replace('‑', '-') download_url = self.base_url + detail_name print('\n\n輸入包名正確,下載地址：' + download_url + '\n') print("---------------------------------------開始下載---------------------------------------\n") if downloadMethod == 'curl': cmd = 'curl -O %s' % download_url else: cmd = 'axel %s' % download_url while True: cmd_res = os.system(cmd) if cmd_res == 0: print('下載成功!\n') break else: print('下載失敗!\n') is_try = input('是否重試下載(y|n)： ') if is_try == 'y' or is_try == 'Y': continue else: break else: print("\n \033[1;31m 輸入序號不符合要求,請重新輸入！ \033[0m \n")
3.5 調用
采用axel下載：
dw = AutoDownWhl()
dw.searchDown(downloadMethod='axel')
實際運行效果顯示：
因為下載使用的windows系統版本的axel命令無法訪問https，
(使用axel命令下載報錯，錯誤原因是Unable to connect to server download.lfd.uci.edu:80。使用瀏覽器直接訪問download.lfd.uci.edu報錯，但是通過查看包的下載地址，地址開頭是https，使用https://www.lfd.uci.edu/就能訪問該網站，而http://www.lfd.uci.edu/卻無法訪問。)
所以使用curl命令，修改最后一行：dw.searchDown(downloadMethod='curl') (前提是需要把curl命令添加到系統環境變量里)
執行python AutoDownloadWhl.py
首先會列出當前搜索到的所有whl包，接下來輸入下載包的名字(英文逗號分割)，然后會按照順序，挨個查找下載包，根據python版本的不同，輸入需要下載的包序號，會出現該包的下載地址。
然后curl就開始下載包了，下載的包會保存到當前目錄。
或者只獲取包的下載地址，使用迅雷下載
完整的下載演示如下所示：
C:\Users\sandu\Anaconda3\envs\Network\python.exe D:/scrapyseleniumtest/AutoDownloadWhl.py
---------------------------------------歡迎使用Whl自助搜索下載器---------------------------------------
------------------共查找到476個包------------------
AD3 Aggdraw Aiohttp Akima APSW Arctic Aspell-python Assimulo Astropy Atom AutoPy Babel Backports.lzma Basemap Bazaar
Bcolz Bigfloat Bintrees Bio_formats Biopython Bitarray Blaze Blist Blosc Blz Boost.Python Bottleneck Brotli brotlipy Bsddb3
Bsdiff4 BTrees Cairocffi Cantera Carray Cartopy Casuarius cChardet Cdecimal CellCognition CellProfiler Centrosome ceODBC Cf-units Cffi
Cftime CGAL-bindings Cgkit Chaco Chebyfit Cheetah Chompack CLD Cmapfile cmarkgfm COBRA Coverage Crc16 Crcmod Cupy
Curses CVXcanon CVXOPT CVXPY cx_Freeze cx_Oracle Cyassimp Cyordereddict Cyrasterize Cython Cytoolz Czifile Daal4py Datrie Debug-information-files
Delny Dipy Discretize Dnacurve Dulwich DyND ECOS Enable Enaml Entropy ETS FabIO Fann2 Fast-histogram Fastcache
Fastparquet Fastrlock fastText fastTSNE Faulthandler Fcsfiles Fdint Ffnet Fiona Fisher Fisx FLANN Fmkr Fpzip FreeImageDLL
FreeSASA Friture GDAL Genshi Gensim Gevent Glumpy GMPY GPy GR Grako Greenlet gRPCio GuiQwt Gvar
H5py HDDM Heatmap Hmmlearn HoloPy HTSeq HyperSpy Ilastik Imagecodecs ImageD11 Iminuit Imread Imreg Indexed_gzip Intbitset
IOCBio Iris Javabridge JCC JPype Jsonlib Jupyter Kapteyn Kivy Kiwisolver Kwant La Lazy_object_proxy Lazyflow Lfdfiles
LibLAS LIBLINEAR Libpython libSBML LIBSVM Libtfr Libxml-python Lightning Line_profiler Llist Llvmlite LLVMPy Logbook Louvain-igraph lp_solve
Lru_dict Lsqfit Lucam Lxml Lz4 Mahotas Marisa-trie MarkupSafe Mathutils Matplotlib Mayavi Menpo Mercurial MeshPy Milk
Minepy Mistune MKL-service MKL_fft MKL_random Mlpy mmLib MMTK Mod_wsgi ModernGL Molmass Mpi4py Mplcairo Msgpack Multidict
MultiNEAT Multiprocess mxBase MySQL-python Mysqlclient Natgrid NDimage NetCDF4 Netifaces Netpbmfile NIPY NiTime NLopt nMOLDYN Noise
Numba Numcodecs Numexpr NumPy Numpy-quaternion numpy-stl ObsPy ODE Oiffile Open3D OpenBabel OpenCV OpenEXR OpenImageIO OpenPIV
openTSNE Orange Osqp Oursql Pandas Param Peewee Pendulum Persistent Pgmagick Pillow Pillow-SIMD Pip Planar Pocketsphinx
Polygon Polylearn Polymode PostgresAdapter Protobuf Psf Psutil Psycopg PuLP Py-earth Py-fcm Py-lmdb Py2exe Py_gd PyAlembic
PyAMF PyAMG PyArrow Pyasn PyAudio PyBluez PyBox2D Pycairo Pycares PyCifRW PyCLD2 PyCluster PyCogent PyCorrFit Pycosat
PyCryptoSat PyCUDA PycURL Pycvodes PyDbg PyDDE PyEDA PyEMD PyEphem Pyexiv2 PyFFTW PyFITS pyFLTK PyFlux pyFM
PyFMI Pygame Pygit2 PyGraphviz PyGreSQL PyGTK PyHDF PyHook Pyhull PyICU PyISAPIe PyJNIus Pykinsol PyKSVD Pyldap
PyLibDeconv PyLibTiff PyLZMA Pymatgen PyMC PyMca PyMCMC PyMCubes PyMedia PyMetis PyMinuit PyMix PyMOL Pymongo Pymssql
Pymunk Pymutt PyMVPA PyODBC Pyodeint PyOpenCL PyOpenGL Pypmc Pyproj PyQPBO PyQt4 PyQwt Pyropes Pyrsistent pyRXP
PySFML PySide pySLALIB PySparse Pyspharm Pysqlite PyStemmer PyStruct Pyswisseph PyTables Python-cjson Python-geohash Python-igraph Python-ldap Python-Levenshtein
Python-lzf Python-LZO Python-snappy Python-sundials PythonMagick PythonNET Pytiff PyTorch PyTST PyUSB-FTDI PyViennaCL PyVRML97 PyWavelets PyWCS PyWin32
PyWinHook Pywinpty PyX PyXML PyYAML PyZMQ Pyzopfli Qdafile Qimage2ndarray Qt_graph_helpers QuantLib QuickFIX QuTiP Rapidjson Rasterio
Re2 Recordclass Regex ReportLab Rpy2 Rtmidi-python Rtree Ruamel.yaml Salientdetect Sasl Scandir ScientificPython Scikit-cycling Scikit-fmm Scikit-image
Scikit-learn Scikit-misc Scikit-umfpack Scikits.ann Scikits.audiolab Scikits.delaunay Scikits.hydroclimpy Scikits.odes Scikits.samplerate Scikits.scattpy Scikits.timeseries Scikits.vectorplot Scimath SciPy Scipy-cluster
SCS Sdtfile SendKeys Seqlearn Setproctitle SfePy Shapely SilverCity Sima Simpleaudio SimpleITK Simplejson SimpleParse Slycot Sounddevice
spaCy Sparsesvd Spectrum Spglib SQLAlchemy Statsmodels STEPS Stratify Swiglpk Sympy TA-Lib TatSu TensorFlow Thrift ThriftPy
Tifffile Tinyarray Tomopy Tornado Traits Transformations Triangle Trollius TWAINmodule Twisted typed_ast Uciwebauth UDUNITS Ujson Umysql
Veusz VideoCapture Vidsrc VIGRA ViTables VPython VTK Winrandom Wordcloud Wrapt Wrf_python wxPython x86cpu Xgboost Xxhash
XYlib-py Yappi Yarl Yt Zfec Zipline Zodbpickle Zope.interface Zs Zstd Misc
請輸入下載包的名字(英文逗號分割): SQLAlchemy
---------查找SQLAlchemy成功,正在輸出包的詳細信息---------
(1)SQLAlchemy‑1.2.18‑cp27‑cp27m‑win32.whl
(2)SQLAlchemy‑1.2.18‑cp27‑cp27m‑win_amd64.whl
(3)SQLAlchemy‑1.2.18‑cp35‑cp35m‑win32.whl
(4)SQLAlchemy‑1.2.18‑cp35‑cp35m‑win_amd64.whl
(5)SQLAlchemy‑1.2.18‑cp36‑cp36m‑win32.whl
(6)SQLAlchemy‑1.2.18‑cp36‑cp36m‑win_amd64.whl
(7)SQLAlchemy‑1.2.18‑cp37‑cp37m‑win32.whl
(8)SQLAlchemy‑1.2.18‑cp37‑cp37m‑win_amd64.whl
請輸入下載的包名序號： 8
輸入包名正確,下載地址：https://download.lfd.uci.edu/pythonlibs/r5uhg2lo/SQLAlchemy-1.2.18-cp37-cp37m-win_amd64.whl
---------------------------------------開始下載---------------------------------------
curl -O https://download.lfd.uci.edu/pythonlibs/r5uhg2lo/SQLAlchemy-1.2.18-cp37-cp37m-win_amd64.whl
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1087k 100 1087k 0 0 39631 0 0:00:28 0:00:28 --:--:-- 45299
下載成功!
是否繼續下載其他庫(y|n)： n
---------------------------------------感謝使用---------------------------------------
修改后使用curl命令下載的代碼地址：
https://files.cnblogs.com/files/sanduzxcvbnm/AutoDownloadWhl-master.7z