使用curl斷點續傳下載文件

本文轉載自查看原文 2021-07-11 00:07 1226 curl/ tools

辦公網絡網速不是很好，使用Chrome下載一些軟件時不時會中斷，惡心的是Chrome居然不支持斷點續傳下載(為什么chrome的下載不支持斷點續傳呢?)，迅雷自然是不能裝的，那怎么辦？還好我有大名鼎鼎的curl，看官網的介紹，是不是有點6到沒朋友👍~~

Supports...

DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, HTTP/2, HTTP/3, cookies, user+password authentication (Basic, Plain, Digest, CRAM-MD5, SCRAM-SHA, NTLM, Negotiate and Kerberos), file transfer resume, proxy tunneling and more.

What's curl used for?

curl is used in command lines or scripts to transfer data. curl is also used in cars, television sets, routers, printers, audio equipment, mobile phones, tablets, settop boxes, media players and is the Internet transfer engine for thousands of software applications in over ten billion installations.

curl is used daily by virtually every Internet-using human on the globe.

而且 Windows 10 已經內置支持curl(Tar and Curl Come to Windows)，通過 PowerShell/CMD 可以直接使用，一口氣上六樓，很方便😎！

C:\>curl
curl: try 'curl --help' for more information

C:\>curl -V -v
curl 7.55.1 (Windows) libcurl/7.55.1 WinSSL
Release-Date: 2017-11-14, security patched: 2019-11-05
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL

在命令行語法中，某些字符與格式有着特殊的意義與含義，一般的約定如下

<>：表示必選, 即：該參數、值或信息必傳。
[]：表示可選，其內的參數、值或信息可傳可不傳。
… : 由三個句點組成的省略號表示"等等", 表示前述元素(參數、值或信息)可以重復多次。
{}：表示枚舉，所傳值必須在{}中給定的范圍內。
|：管道符號(豎線)表示"或者"，表示在一個集合范圍中的一個選項。
/：同上，表示在一個集合范圍中的一個選項。
斜體: 表示要提供的信息，要用該實際值來替換的選項或參數，該約定不常見。

再結合簡單說下curl的基本使用，即curl [options...] <url>

通過curl --help列出所有命令(原列表較長，已大幅刪減，若安裝最新版則會更多，下面只列舉了最常用的)
其中options(選項)由--加小寫英文單詞組成，如--help，也有少量縮寫，如--ipv4
若英文單字超過兩個，則用-連接，如--progress-bar
有的選項可以有參數，help文檔里以<param>的形式展示，如--data <data>, --user <user:password>
部分常用選項有簡寫，由-加一個字符組成，如-d, -4, -#, -:等，需要注意的是，單個字母是區分大小寫的，如-H(--head)和-h(--help)表示不同的含義
選項可以是1個到多個不等(最簡單的curl <url>是curl --url <url>的縮寫，所以至少有一個選項)

C:\>curl --help
Usage: curl [options...] <url>
 -a, --append        Append to target file when uploading
     --basic         Use HTTP Basic Authentication
 -E, --cert <certificate[:password]> Client certificate file and password
 -C, --continue-at <offset> Resumed transfer offset
 -b, --cookie <data> Send cookies from string/file
 -c, --cookie-jar <filename> Write cookies to <filename> after operation
     --create-dirs   Create necessary local directory hierarchy
 -d, --data <data>   HTTP POST data
     --data-ascii <data> HTTP POST ASCII data
     --data-binary <data> HTTP POST binary data
     --data-raw <data> HTTP POST data, '@' allowed
     --data-urlencode <data> HTTP POST data url encoded
 -f, --fail          Fail silently (no output at all) on HTTP errors
     --fail-early    Fail on first transfer error, do not continue
 -F, --form <name=content> Specify HTTP multipart POST data
     --form-string <name=string> Specify HTTP multipart POST data
 -G, --get           Put the post data in the URL and use GET
 -I, --head          Show document info only
 -H, --header <header/@file> Pass custom header(s) to server
 -h, --help          This help text
 -k, --insecure      Allow insecure server connections when using SSL
 -i, --include       Include protocol response headers in the output
 -4, --ipv4          Resolve names to IPv4 addresses
 -6, --ipv6          Resolve names to IPv6 addresses
 -L, --location      Follow redirects
 -:, --next          Make next URL use its separate set of options
 -N, --no-buffer     Disable buffering of the output stream
     --oauth2-bearer <token> OAuth 2 Bearer Token
 -o, --output <file> Write to file instead of stdout
 -#, --progress-bar  Display transfer progress as a bar
 -x, --proxy [protocol://]host[:port] Use this proxy
 -U, --proxy-user <user:password> Proxy user and password
 -O, --remote-name   Write output to a file named as the remote file
     --remote-name-all Use the remote file name for all URLs
     --retry <num>   Retry request if transient problems occur
     --retry-connrefused Retry on connection refused (use with --retry)
     --retry-delay <seconds> Wait time between retries
     --retry-max-time <seconds> Retry only within this period
 -S, --show-error    Show error even when -s is used
 -s, --silent        Silent mode
 -Y, --speed-limit <speed> Stop transfers slower than this
 -y, --speed-time <seconds> Trigger 'speed-limit' abort after this time
 -t, --telnet-option <opt=val> Set telnet option
     --trace <file>  Write a debug trace to FILE
     --trace-ascii <file> Like --trace, but without hex output
     --trace-time    Add time stamps to trace/verbose output
 -T, --upload-file <file> Transfer local FILE to destination
     --url <url>     URL to work with
 -B, --use-ascii     Use ASCII/text transfer
 -u, --user <user:password> Server user and password
 -A, --user-agent <name> Send User-Agent <name> to server
 -v, --verbose       Make the operation more talkative
 -V, --version       Show version number and quit

言歸正傳，下面我們使用酷酷的curl下載一些文件(注：以下演示均在高速網絡⚡下進行)

最簡單的使用curl --remote-name(-O) URL 或者 curl --output(-o) fileName URL，以下載最新64位windows版本Curl為例

C:\>mkdir Software

C:\>cd Software

C:\Software>curl -O https://curl.se/windows/dl-7.77.0_2/curl-7.77.0_2-win64-mingw.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 100 5154k  100 5154k    0     0  5154k      0  0:00:01 --:--:--  0:00:01 6467k

使用-continue-at(-C)斷點續傳下載，這次用PowerShell下載一個大文件(SSMS)，關於該選項的使用，參見以下官方用戶手冊注解

-C, --continue-at

Continue/Resume a previous file transfer at the given offset. The given offset is the exact number of bytes that will be skipped, counting from the beginning of the source file before it is transferred to the destination. If used with uploads, the FTP server command SIZE will not be used by curl.

Use "-C -" to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out.

If this option is used several times, the last one will be used.

在Powershell中使用需要帶上后綴名,即curl.exe，不然不能正確識別

PS C:\Software> curl -C - -O https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
Invoke-WebRequest : 無法處理參數，因為參數名稱“C”具有二義性。可能的匹配項包括:  -Credential -CertificateThumbprint -Certificate -ContentType。
所在位置 行:1 字符: 6
+ curl -C - -O https://download.microsoft.com/download/4/6/8/4681f3b2-f ...
+      ~~
    + CategoryInfo          : InvalidArgument: (:) [Invoke-WebRequest]，ParameterBindingException
    + FullyQualifiedErrorId : AmbiguousParameter,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

PS C:\Software> curl --continue-at - -O https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
Invoke-WebRequest : 找不到接受實際參數“-”的位置形式參數。
所在位置 行:1 字符: 1
+ curl --continue-at - -O https://download.microsoft.com/download/4/6/8 ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Invoke-WebRequest]，ParameterBindingException
    + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

通過關閉無線/有線網絡，模擬網絡不穩定的情況

PS C:\Software> curl.exe -C - -O https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 16  565M   16 94.7M    0     0  4852k      0  0:01:59  0:00:20  0:01:39     0
curl: (56) Send failure: Connection was reset

PS C:\Software> curl.exe -C - -O https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
** Resuming transfer from byte position 172521856
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  9  470M    9 46.9M    0     0  1502k      0  0:05:20  0:00:32  0:04:48 3720k
curl: (56) Send failure: Connection was reset

可以使用--retry <num>選項在下載失敗后自動重試(此時網絡沒連接，在觀察到重試現象后，打開網絡連接)，另外：請動手嘗試前文curl --help列舉的其他retry選項，這里不再逐一舉例

PS C:\Software> curl.exe -C - -O --retry 10 https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
** Resuming transfer from byte position 221771295
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 1 seconds. 10 retries left.
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 2 seconds. 9 retries left.
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 4 seconds. 8 retries left.
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 8 seconds. 7 retries left.
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: Transient problem: timeout Will retry in 16 seconds. 6 retries left.
100  423M  100  423M    0     0  4667k      0  0:01:33  0:01:33 --:--:-- 7028k
PS C:\Software>

下載完成后使用斷點續傳及重試選項，不會覆蓋已下載的文件；反之，則不成立

PS C:\Software> curl.exe -C - -O https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
** Resuming transfer from byte position 666228072
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0   237    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (33) HTTP server doesn''t seem to support byte ranges. Cannot resume.

PS C:\Software> curl.exe -C - -O --retry 3 https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
** Resuming transfer from byte position 666228072
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0   237    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
Warning: Transient problem: FTP error Will retry in 1 seconds. 3 retries left.
  0   237    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
Warning: Transient problem: FTP error Will retry in 2 seconds. 2 retries left.
  0   237    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
Warning: Transient problem: FTP error Will retry in 4 seconds. 1 retries left.
  0   237    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (33) HTTP server doesn''t seem to support byte ranges. Cannot resume.

PS C:\Software> curl.exe -O https://download.microsoft.com/download/4/6/8/4681f3b2-f327-4d3d-8617-264b20685be0/SSMS-Setup-ENU.exe
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  635M  100  635M    0     0  16.2M      0  0:00:39  0:00:39 --:--:-- 24.2M

對於有些資源，網速不好的時候，甚至跳轉到具體下載鏈接都很困難，以下載最新的 windows 64 位版本的 PostgreSQL 13.X為例，官網找到下載資源列表，頁面跳轉后一直不開始下載，點擊下載鏈接也沒反應。。。

別捉急，curl一樣可以幫到你。鼠標右鍵復制下載鏈接，使用--head(-I) 或者 --include(-i)選項(也可使用--verbose(-v)選項，這樣，你可以看到所謂的TCP三次握手是怎樣發生的😂)，具體下載鏈接在Location首部字段里，然后使用上面提到的斷點續傳下載選項，DONE✌！

C:\>curl --head https://sbp.enterprisedb.com/getfile.jsp?fileid=1257713
HTTP/1.1 302 Found
Content-Type: text/html; charset=utf-8
Content-Length: 0
Connection: keep-alive
Server: nginx/1.16.1
Date: Sat, 10 Jul 2021 08:50:09 GMT
Location: https://get.enterprisedb.com/postgresql/postgresql-13.3-2-windows-x64.exe
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
X-Cache: Miss from cloudfront
Via: 1.1 a43db2746d5ea9543e11897b6654f9b6.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: HKG62-C1
X-Amz-Cf-Id: KuxMKejZWH-BMtFbzdKMCXsJE1ctLNMoSIcSYHLIkWQmbFnbexDsNg==

C:\>curl -v https://sbp.enterprisedb.com/getfile.jsp?fileid=1257713
*   Trying 13.32.53.65...
* TCP_NODELAY set
* Connected to sbp.enterprisedb.com (13.32.53.65) port 443 (#0)
* schannel: SSL/TLS connection with sbp.enterprisedb.com port 443 (step 1/3)
* schannel: checking server certificate revocation
* schannel: sending initial handshake data: sending 191 bytes...
* schannel: sent initial handshake data: sent 191 bytes
* schannel: SSL/TLS connection with sbp.enterprisedb.com port 443 (step 2/3)
* schannel: failed to receive handshake, need more data
* schannel: SSL/TLS connection with sbp.enterprisedb.com port 443 (step 2/3)
* schannel: encrypted data got 4096
* schannel: encrypted data buffer: offset 4096 length 4096
* schannel: encrypted data length: 3996
* schannel: encrypted data buffer: offset 3996 length 4096
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with sbp.enterprisedb.com port 443 (step 2/3)
* schannel: encrypted data got 1024
* schannel: encrypted data buffer: offset 5020 length 5020
* schannel: encrypted data length: 164
* schannel: encrypted data buffer: offset 164 length 5020
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with sbp.enterprisedb.com port 443 (step 2/3)
* schannel: encrypted data got 150
* schannel: encrypted data buffer: offset 314 length 5020
* schannel: sending next handshake data: sending 93 bytes...
* schannel: SSL/TLS connection with sbp.enterprisedb.com port 443 (step 2/3)
* schannel: encrypted data got 170
* schannel: encrypted data buffer: offset 170 length 5020
* schannel: SSL/TLS handshake complete
* schannel: SSL/TLS connection with sbp.enterprisedb.com port 443 (step 3/3)
* schannel: stored credential handle in session cache
...

寫在最后，准備這篇隨筆的時候，翻閱了不少資料，發現有兩個資源是很不錯的, 一個是官網的教程, 另一個是官網提到的Everything curl，Everything curl另有開源翻譯版, 找到下載章節，一個感覺就是：~~我有必要寫這篇博客嗎？寫個寂寞🤔~~。。。最后，附上原文地址, 原創不易，且行且珍惜🤣~~

更新: 另有中文版圖書《cURL必知必會》, 在微信讀書可免費閱讀。

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 python實現斷點續傳下載文件 C#斷點續傳下載文件 asp.net mvc5 下載文件及斷點續傳下載 [c#] 斷點續傳下載文件[帶進度條類似迅雷] AFNetworking 下載文件斷點續傳操作 rsync實現linux下載文件斷點續傳 qt通過http請求下載文件（支持斷點續傳） C# Ftp方式下載文件（無用戶認證方式,支持斷點續傳） php實現大文件斷點續傳下載實例 iOS開發之網絡編程--使用NSURLConnection實現大文件斷點續傳下載