瀏覽器緩存與nginx緩存
瀏覽器緩存
優點：使用有效緩存時，沒有網絡消耗，速度快；即使有網絡消耗，但對失效緩存使用304響應做到網絡消耗最小化
缺點：僅提升一個用戶的體驗
nginx 緩存
優點：提升所有用戶體驗，相比瀏覽器緩存，有效降低上游服務的負載，通過304響應減少nginx與上游服務間的流量消耗
缺點：用戶仍然保持網絡消耗
同時使用瀏覽器與nginx緩存
Etag 頭部
ETagHTTP 響應頭是資源的特定版本的標識符。可以是緩存更高效，並節省帶寬，因為如果內容沒有改變。web服務不需要發送完整的響應。如果內容發生變化，使用ETag有助於防止資源的同時更新互換覆蓋；如果給定URL中的資源更改,則一定要生成新的Etag值。因此Etge類似於指紋，也可能被某些服務器用於追蹤。比較etags能快速確定此資源是否變化，但也可能被追蹤服務器永久保存
W/可選
‘W/’（大小寫敏感）表示使用弱驗證器，弱驗證器很容易生成不利於比較，強驗證器是比較的理想選擇，但很難生成，相同資源的兩個弱Etag值可能語義相同，但不是每個字節都相同
etag 指令
Syntax: etag on | off; Default: etag on; Context: http, server, location
生成規則
ngx_sprintf(etag->value.data, "\"%xT-%xO\"", r->headers_out.last_modified_time, r->headers_out.content_length_n)
If-None-Match
If-Modified-Since 頭部
not_modified過濾模塊
expires 指令
Syntax: expires [modified] time; expires epoch | max | off; #指的一個絕對時間，表示緩存在這段時間內一直有效 Default: expires off; Context: http, server, location, if in location
配置
[root@python vhast]# cat cache.conf
server {
server_name cache.com;
error_log logs/cacgeee.log;
access_log logs/cache.log main;
root html/;
location /{
etag on; 啟用
expires 1h;緩存1小時
#expires -1h;
#expires @20h30m;
#if_modified_since off;
#proxy_cache two;
#proxy_cache_valid 100 10m;
#add_header X-Cache-Status $upstream_cache_status;
#proxy_cache_use_stale error timeout updating;
#proxy_cache_revalidate on;
#proxy_cache_background_update on;
#proxy_hide_header Set-Cookie;
#proxy_ignore_headers Set-Cookie;
#proxy_force_ranges on;
#proxy_pass http://127.0.0.1:8012;
}
}
測試
[root@python vhast]# date -u 2019年 07月 17日 星期三 16:12:16 UTC [root@python vhast]# curl cache.com/ -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 16:12:19 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT Connection: keep-alive ETag: "5d262d06-264" Expires: Wed, 17 Jul 2019 17:12:19 GMT Cache-Control: max-age=3600 Accept-Ranges: bytes
配置
[root@python vhast]# cat cache.conf
server {
server_name cache.com;
error_log logs/cacgeee.log;
access_log logs/cache.log main;
root html/;
location /{
etag on;
#expires 1h;
expires -1h; #設置前一小時失效
#expires @20h30m;
#if_modified_since off;
#proxy_cache two;
#proxy_cache_valid 100 10m;
#add_header X-Cache-Status $upstream_cache_status;
#proxy_cache_use_stale error timeout updating;
#proxy_cache_revalidate on;
#proxy_cache_background_update on;
#proxy_hide_header Set-Cookie;
#proxy_ignore_headers Set-Cookie;
#proxy_force_ranges on;
#proxy_pass http://127.0.0.1:8012;
}
}
測試
[root@python vhast]# date -u 2019年 07月 17日 星期三 16:16:37 UTC [root@python vhast]# curl cache.com/ -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 16:16:48 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT Connection: keep-alive ETag: "5d262d06-264" Expires: Wed, 17 Jul 2019 15:16:48 GMT Cache-Control: no-cache #緩存失效 Accept-Ranges: bytes
配置指的瀏覽器緩存到指定的時間
[root@python vhast]# cat cache.conf
server {
server_name cache.com;
error_log logs/cacgeee.log;
access_log logs/cache.log main;
root html/;
location /{
etag on;
#expires 1h;
#expires -1h;
expires @20h30m; 緩存到最近的20點半
#if_modified_since off;
#proxy_cache two;
#proxy_cache_valid 100 10m;
#add_header X-Cache-Status $upstream_cache_status;
#proxy_cache_use_stale error timeout updating;
#proxy_cache_revalidate on;
#proxy_cache_background_update on;
#proxy_hide_header Set-Cookie;
#proxy_ignore_headers Set-Cookie;
#proxy_force_ranges on;
#proxy_pass http://127.0.0.1:8012;
}
}
測試
[root@python vhast]# date -u 2019年 07月 17日 星期三 16:18:54 UTC [root@python vhast]# curl cache.com/ -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 16:18:57 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT Connection: keep-alive ETag: "5d262d06-264" Expires: Thu, 18 Jul 2019 12:30:00 GMT Cache-Control: max-age=72663 Accept-Ranges: bytes
not_modified 過濾模塊
not_modified 指令
Syntax: if_modified_since off | exact | before; Default: if_modified_since exact; Context: http, server, location
測試
[root@python vhast]# curl cache.com/ -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 16:47:10 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT Connection: keep-alive ETag: "5d262d06-264" Expires: Thu, 18 Jul 2019 12:30:00 GMT Cache-Control: max-age=70970 Accept-Ranges: bytes [root@python vhast]# curl -H 'If_Modified_Since: Wed, 10 Jul 2019 18:23:02 GMT' -H 'If-None-Match: "5d262d06-264"' cache.com/ -I HTTP/1.1 304 Not Modified#命中緩存 Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 16:47:22 GMT Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT Connection: keep-alive ETag: "5d262d06-264" Expires: Thu, 18 Jul 2019 12:30:00 GMT Cache-Control: max-age=70958 [root@python vhast]# curl -H 'If_Modified_Since: Wed, 10 Jul 2019 18:23:02 GMT' -H 'If-None-Match: "5d262d06-267"' cache.com/ -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 16:48:12 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT Connection: keep-alive ETag: "5d262d06-264" Expires: Thu, 18 Jul 2019 12:30:00 GMT Cache-Control: max-age=70908 Accept-Ranges: bytes
If-Match
If-Unmodified-Since
nginx 緩存：定義存放緩存的載體
Syntax: proxy_cache zone | off; Default: proxy_cache off; Context: http, server, location Syntax: proxy_cache_path path [levels=levels] [use_temp_path=on|off] keys_zone=name:size [inactive=time] [max_size=size] [manager_files=number] [manager_sleep=time] [manager_threshold=time] [loader_files=number] [loader_sleep=time] [loader_threshold=time] [purger=on|off] [purger_files=number] [purger_sleep=time] [purger_threshold=time]; Default: — Context: http
proxy_cache_path 指令
緩存關鍵字
Syntax: proxy_cache_key string; Default: proxy_cache_key $scheme$proxy_host$request_uri; Context: http, server, location
緩存什么樣的響應
Syntax: proxy_cache_valid [code ...] time; Default: — Context: http, server, location
那些內容不使用緩存
參數為真時響應不存入緩存
Syntax: proxy_no_cache string ...; Default: — Context: http, server, location
參數為真時，不使用緩存內容
Syntax: proxy_cache_bypass string ...; Default: — Context: http, server, location
變更HEAD方法
Syntax: proxy_cache_convert_head on | off; Default: proxy_cache_convert_head on; Context: http, server, location
upstream_cache_status變量
upstream_cache_status
MISS：未命中緩存； HIT：命中緩存； EXPIRED：緩存已過期；STALE：命中陳舊的緩存；UPDATING:內容陳舊，正在更新；REVALIDATED：nginx驗證陳舊的內容依然有效 ；BYPASS：內容是從原始服務器獲得
緩存流程
對那個method方法使用緩存返回響應
Syntax: proxy_cache_methods GET | HEAD | POST ...; Default: proxy_cache_methods GET HEAD; Context: http, server, location
配置
server {
listen 8012;
default_type text/plain;
root html;
location /{
#add_header X-Accel-Limit-Rate 10;
}
location /test {
return 200 '8012 server response.
uri: $uri
method: $request_method
requset: $request
http_name: $http_name
\n';
}
}
[root@python vhast]# cat cache.conf
proxy_cache_path /data/web/cache levels=2:2 keys_zone=two:10m loader_threshold=300 loader_files=200 max_size=200m inactive=1m;
server {
server_name cache.com;
error_log logs/cacgeee.log;
access_log logs/cache.log main;
root html/;
location /{
etag on;
#expires 1h;
#expires -1h;
#expires @20h30m;
#if_modified_since off;
proxy_cache two;
proxy_cache_valid 200 1m;
add_header X-Cache-Status $upstream_cache_status;
#proxy_cache_use_stale error timeout updating;
#proxy_cache_revalidate on;
#proxy_cache_background_update on;
#proxy_hide_header Set-Cookie;
#proxy_ignore_headers Set-Cookie;
#proxy_force_ranges on;
proxy_pass http://127.0.0.1:8012;
}
}
測試
[root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:23:40 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: MISS 第一次 Accept-Ranges: bytes [root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:23:44 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: HIT 第二次 Accept-Ranges: bytes
X-Accel-Expires頭部
配置
server {
listen 8012;
default_type text/plain;
#client_body_in_single_buffer on;
#add_header Cache_Control 'max-age=3,stale-while-revalidate=3';
#add_header Vary *;
add_header X-Accel-Expires 3; #定義代理服務器緩存為3秒
root html;
location /{
#add_header X-Accel-Limit-Rate 10;
}
location /test {
return 200 '8012 server response.
uri: $uri
method: $request_method
requset: $request
http_name: $http_name
\n';
}
}
測試
[root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:35:20 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: EXPIRED Accept-Ranges: bytes [root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:35:21 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: HIT Accept-Ranges: bytes [root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:35:22 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: HIT Accept-Ranges: bytes [root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:35:23 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: HIT Accept-Ranges: bytes [root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:35:25 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: EXPIRED Accept-Ranges: bytes
vary 頭部
配置
server {
listen 8012;
default_type text/plain;
#client_body_in_single_buffer on;
#add_header Cache_Control 'max-age=3,stale-while-revalidate=3';
add_header Vary *; #配置代理不緩存
add_header X-Accel-Expires 3; #定義上游服務器緩存為3
root html;
location /{
#add_header X-Accel-Limit-Rate 10;
}
location /test {
return 200 '8012 server response.
uri: $uri
method: $request_method
requset: $request
http_name: $http_name
\n';
}
}
測試
[root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:38:23 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" Vary: * Accept-Ranges: bytes X-Cache-Status: MISS [root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:38:23 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" Vary: * Accept-Ranges: bytes X-Cache-Status: MISS
Set-Cookie頭部
Set-Cookie: <cookie-name>=<cookie-value> Set-Cookie: <cookie-name>=<cookie-value>; Expires=<date> Set-Cookie: <cookie-name>=<cookie-value>; Max-Age=<non-zero-digit> Set-Cookie: <cookie-name>=<cookie-value>; Domain=<domain-value> Set-Cookie: <cookie-name>=<cookie-value>; Path=<path-value> Set-Cookie: <cookie-name>=<cookie-value>; Secure Set-Cookie: <cookie-name>=<cookie-value>; HttpOnly Set-Cookie: <cookie-name>=<cookie-value>; SameSite=Strict Set-Cookie: <cookie-name>=<cookie-value>; SameSite=Lax Set-Cookie: <cookie-name>=<cookie-value>; Domain=<domain-value>; Secure; HttpOnly
緩存流程：接收上游的響應
合並回源請求-減少峰值流量下的壓力
Syntax: proxy_cache_lock on | off; Default: proxy_cache_lock off; Context: http, server, location 同一時間，僅第一個請求發往上游，其他請求等待第一個響應返回或者超時候，使用緩存響應客戶端 Syntax: proxy_cache_lock_timeout time; Default: proxy_cache_lock_timeout 5s; Context: http, server, location 等待第一個請求返回響應的最大時間，到達后直接向上游發送請求，但不緩存響應 Syntax: proxy_cache_lock_age time; Default: proxy_cache_lock_age 5s; Context: http, server, location 上一個請求返回響應的超時時間，到達后再放行一個請求發送向上游
減少回源請求試圖stale陳舊的緩存
Syntax: proxy_cache_use_stale error | timeout | invalid_header | updating | http_500 | http_502 | http_503 | http_504 | http_403 | http_404 | http_429 | off ...; Default: proxy_cache_use_stale off; Context: http, server, location Syntax: proxy_cache_background_update on | off; Default: proxy_cache_background_update off; Context: http, server, location
proxy_cache_use_stale指定陳舊緩存的用法
緩存有問題時的響應
Syntax: proxy_cache_background_update on | off; Default: proxy_cache_background_update off; Context: http, server, location
當使用proxy_cache_use_stale允許使用過期響應時，將同步生成一個子請求，通過訪問上游服務更新過期的緩存
Syntax: proxy_cache_revalidate on | off; Default: proxy_cache_revalidate off; Context: http, server, location
更新緩存時，使用If-Modified-Since和If-None-Match作為請求頭部，預期內容未發生變更時通過304來減少傳輸的內容
及時清除緩存
模塊第三方模塊ngx_cache_purge 地址 https://github.com/FRiCKLE/ngx_cache_purge
使用--add-module=指令編譯添加到nginx中
功能：接收到指定HTTP請求后立刻清除緩存
•syntax: proxy_cache_purge on|off|<method> [from all|<ip> [.. <ip>]] •default: none •context: http, server, location •syntax: proxy_cache_purge zone_name key •default: none •context: location
編譯進第三方模塊
git clone https://github.com/FRiCKLE/ngx_cache_purge.git
cd nginx-1.15.9/
./configure --prefix=/data/web --sbin-path=/usr/bin --user=nginx --group=nginx --with-http_stub_status_module --with-http_auth_request_module --with-http_sub_module --add-module=/root/nginx-http-concat --with-http_addition_module --with-http_secure_link_module --with-http_geoip_module --with-http_ssl_module --add-module=/root/ngx_cache_purge
make
mv /usr/bin/nginx{,.01.18.15.41}
cp objs/nginx /usr/bin/
配置
[root@python vhast]# cat cache.conf
proxy_cache_path /data/web/cache levels=2:2 keys_zone=two:10m loader_threshold=300 loader_files=200 max_size=200m inactive=1m;
server {
server_name cache.com;
error_log logs/cacgeee.log;
access_log logs/cache.log main;
root html/;
location ~/purge(/.*) {
proxy_cache_purge two $scheme$1; 匹配上的時候，就清空這個緩存
}
location /{
etag on;
#expires 1h;
#expires -1h;
#expires @20h30m;
#if_modified_since off;
proxy_cache two;
proxy_cache_valid 200 1m;
add_header X-Cache-Status $upstream_cache_status;
#proxy_cache_use_stale error timeout updating;
#proxy_cache_revalidate on;
#proxy_cache_background_update on;
#proxy_hide_header Set-Cookie;
#proxy_ignore_headers Set-Cookie;
#proxy_force_ranges on;
proxy_cache_key $scheme$uri;
proxy_pass http://127.0.0.1:8012;
}
}
測試
[root@python vhast]# curl cache.com/index.html -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 20:16:30 GMT Content-Type: text/html Content-Length: 612 Connection: keep-alive Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT ETag: "5d262d06-264" X-Cache-Status: MISS #未命中 Accept-Ranges: bytes [root@python vhast]# curl cache.com/index.html -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 20:16:31 GMT Content-Type: text/html Content-Length: 612 Connection: keep-alive Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT ETag: "5d262d06-264" X-Cache-Status: HIT #命中 Accept-Ranges: bytes [root@python vhast]# curl cache.com/index.html -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 20:16:35 GMT Content-Type: text/html Content-Length: 612 Connection: keep-alive Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT ETag: "5d262d06-264" X-Cache-Status: HIT #命中 Accept-Ranges: bytes [root@python vhast]# curl cache.com/purge/index.html -I #清除緩存 HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 20:16:40 GMT Content-Type: text/html Content-Length: 270 Connection: keep-alive [root@python vhast]# curl cache.com/index.html -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 20:16:42 GMT Content-Type: text/html Content-Length: 612 Connection: keep-alive Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT ETag: "5d262d06-264" X-Cache-Status: MISS #未命中 Accept-Ranges: bytes
七層反向代理對照：構造請求內容
七層反向代理對照：建立連接並發送請求
七層反向代理：接收上游響應
轉發響應
七層代理SSL
緩存類的指令
七層反代獨有配置
