Nginx反向代理功能-緩存功能
作者：尹正傑
版權聲明：原創作品，謝絕轉載！否則將追究法律責任。
一.未使用緩存時對nginx服務器做壓力測試並記錄結果
1>.自行配置nginx的反向代理
博主推薦閱讀: https://www.cnblogs.com/yinzhengjie/p/12099808.html
試驗架構說明:
node101.yinzhengjie.org.cn:
Nginx反向代理服務器
node108.yinzhengjie.org.cn:
Apache httpd web服務器
node105.yinzhengjie.org.cn:
ab工具壓力測試服務器
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/nginx/conf/nginx.conf worker_processes 4; worker_cpu_affinity 00000001 00000010 00000100 00001000; events { worker_connections 100000; use epoll; accept_mutex on; multi_accept on; } http { include mime.types; default_type text/html; server_tokens off; charset utf-8; log_format my_access_json '{"@timestamp":"$time_iso8601",' '"host":"$server_addr",' '"clientip":"$remote_addr",' '"size":$body_bytes_sent,' '"responsetime":$request_ti me,' '"upstreamtime":"$upstream_response_time",' '"upstreamhost":"$upstream_addr",' '"http_host":"$host",' '"uri":"$uri",' '"domain":"$host",' '"xff":"$http_x_forwarded_for",' '"referer":"$http_referer",' '"tcp_xff":"$proxy_protocol_addr",' '"http_user_agent":"$http_user_agent",' '"status":"$status"}'; access_log logs/access_json.log my_access_json; ssl_certificate /yinzhengjie/softwares/nginx/certs/www.yinzhengjie.org.cn.crt; ssl_certificate_key /yinzhengjie/softwares/nginx/certs/www.yinzhengjie.org.cn.key; ssl_session_cache shared:sslcache:20m; ssl_session_timeout 10m; include /yinzhengjie/softwares/nginx/conf.d/*.conf; } [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/nginx/conf.d/node101_yinzhengjie_org.cn.conf server { listen 80; listen 443 ssl; server_name node101.yinzhengjie.org.cn; access_log /yinzhengjie/softwares/nginx/logs/node101_yinzhengjie_org_cn_access.log my_access_json; error_log /yinzhengjie/softwares/nginx/logs/node101_yinzhengjie_org_cn_error.log; location / { root /yinzhengjie/data/web/nginx/static/cn; index index.html; #定義有效的請求referer,用空格隔開即可 valid_referers none blocked server_names *.baidu.com example.* ~\.google\.; #如果沒有在上面的有效鏈接定義那么均屬於無效請求referer if ($invalid_referer) { return 403; } #如果是一些常見的壓測試工具,咱們直接進給他拒絕訪問 if ($http_user_agent ~ "ApacheBench|WebBench|TurnitinBot|Sougou web spider|Grid Server"){ return 403; } } location = /favicon.ico { root /yinzhengjie/data/web/nginx/images/jd; } location /app01 { #proxy_pass指令用來設置將客戶端請求轉發給的后端服務器的主機，可以是主機名、IP地址：端口的方式，也可以代理到預先設置的主機群組，需要模塊gx_http_upstream_module支持。 #帶斜線，等於訪問后端服務器的http://172.30.108:80/index.html內容返回給客戶端。 proxy_pass http://172.30.1.108/; #proxy_connect_timeout time;配置nginx服務器與后端服務器嘗試建立連接的超時時間，默認為60秒，用法如下： proxy_connect_timeout 60s; } location /static { #不帶斜線將訪問的/static,等於訪問后端服務器 http://172.30.1.108:80/static/index.html，即后端服務器配置的站點根目錄要有/static目錄才可以被訪問。 proxy_pass http://172.30.1.108; } location /image { proxy_pass http://172.30.1.108; #proxy_hide_header指令用於nginx作為反向代理的時候，在返回給客戶端http響應的時候，用於隱藏后端服務器特定的響應首部， #默認nginx在響應報文中不傳遞后端服務器的首部字段Dte, Server, XPad,X-Accel等，可以設置在http,server,location塊. #隱藏掉ETag的文本值,CDN會根據ETag的值是否發生變化而決定該文件內容是否發生變化，一旦發生變化CDN會重新抓取該數據並緩存,此處我故意隱藏該值。 proxy_hide_header ETag; } location /dynamic { proxy_pass http://172.30.1.108; #proxy_set_header可以更改或添加客戶端的請求頭部信息內容並轉發至后端服務器，比如在后端服務器想要獲取客戶端的真實IP的時候，就要更改每一個報文的頭部。 #添加HOST到報文頭部，如果客戶端為NAT上網那么其值為客戶端的共用的公網IP地址。 proxy_set_header yinzhengjie_nginx_ip_forwarded $proxy_add_x_forwarded_for; } } [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node108.yinzhengjie.org.cn ~]# egrep -v "^ *#|^$" /etc/httpd/conf/httpd.conf ServerRoot "/etc/httpd" Listen 80 Include conf.modules.d/*.conf User apache Group apache ServerAdmin root@localhost <Directory /> AllowOverride none Require all denied </Directory> DocumentRoot "/var/www/html" <Directory "/var/www"> AllowOverride None Require all granted </Directory> <Directory "/var/www/html"> Options Indexes FollowSymLinks AllowOverride None Require all granted </Directory> <IfModule dir_module> DirectoryIndex index.html </IfModule> <Files ".ht*"> Require all denied </Files> ErrorLog "logs/error_log" LogLevel warn <IfModule log_config_module> LogFormat "\"%{yinzhengjie_nginx_ip_forwarded}i\" %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined LogFormat "%h %l %u %t \"%r\" %>s %b" common <IfModule logio_module> LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio </IfModule> CustomLog "logs/access_log" combined </IfModule> <IfModule alias_module> ScriptAlias /cgi-bin/ "/var/www/cgi-bin/" </IfModule> <Directory "/var/www/cgi-bin"> AllowOverride None Options None Require all granted </Directory> <IfModule mime_module> TypesConfig /etc/mime.types AddType application/x-compress .Z AddType application/x-gzip .gz .tgz AddType text/html .shtml AddOutputFilter INCLUDES .shtml </IfModule> AddDefaultCharset UTF-8 <IfModule mime_magic_module> MIMEMagicFile conf/magic </IfModule> EnableSendfile on IncludeOptional conf.d/*.conf [root@node108.yinzhengjie.org.cn ~]# [root@node108.yinzhengjie.org.cn ~]#
2>.准備測試數據
[root@node108.yinzhengjie.org.cn ~]# ll -hR /var/www/html/ /var/www/html/: total 4.0K drwxr-xr-x 2 root root 24 Dec 26 14:19 dynamic drwxr-xr-x 2 root root 24 Dec 26 14:18 image -rw-r--r-- 1 root root 37 Dec 26 14:04 index.html drwxr-xr-x 2 root root 42 Dec 26 17:00 static /var/www/html/dynamic: total 4.0K -rw-r--r-- 1 root root 40 Dec 26 14:19 index.html /var/www/html/image: total 4.0K -rw-r--r-- 1 root root 38 Dec 26 14:18 index.html /var/www/html/static: total 348K -rw-r--r-- 1 root root 341K Dec 26 17:00 access.log -rw-r--r-- 1 root root 39 Dec 26 14:18 index.html [root@node108.yinzhengjie.org.cn ~]# [root@node108.yinzhengjie.org.cn ~]#
3>.在node105.yinzhengjie.org.cn使用ab命令對nginx進行壓力測試
[root@node105.yinzhengjie.org.cn ~]# yum -y install httpd-tools Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.bit.edu.cn * extras: mirrors.huaweicloud.com * updates: mirror.bit.edu.cn base | 3.6 kB 00:00:00 extras | 2.9 kB 00:00:00 updates | 2.9 kB 00:00:00 updates/7/x86_64/primary_db | 5.9 MB 00:00:01 Resolving Dependencies --> Running transaction check ---> Package httpd-tools.x86_64 0:2.4.6-90.el7.centos will be installed --> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-tools-2.4.6-90.el7.centos.x86_64 --> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-tools-2.4.6-90.el7.centos.x86_64 --> Running transaction check ---> Package apr.x86_64 0:1.4.8-5.el7 will be installed ---> Package apr-util.x86_64 0:1.5.2-6.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================ Package Arch Version Repository Size ============================================================================================================================================================================ Installing: httpd-tools x86_64 2.4.6-90.el7.centos base 91 k Installing for dependencies: apr x86_64 1.4.8-5.el7 base 103 k apr-util x86_64 1.5.2-6.el7 base 92 k Transaction Summary ============================================================================================================================================================================ Install 1 Package (+2 Dependent packages) Total download size: 286 k Installed size: 584 k Downloading packages: (1/3): apr-util-1.5.2-6.el7.x86_64.rpm | 92 kB 00:00:00 (2/3): httpd-tools-2.4.6-90.el7.centos.x86_64.rpm | 91 kB 00:00:00 (3/3): apr-1.4.8-5.el7.x86_64.rpm | 103 kB 00:00:05 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 48 kB/s | 286 kB 00:00:05 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : apr-1.4.8-5.el7.x86_64 1/3 Installing : apr-util-1.5.2-6.el7.x86_64 2/3 Installing : httpd-tools-2.4.6-90.el7.centos.x86_64 3/3 Verifying : apr-1.4.8-5.el7.x86_64 1/3 Verifying : httpd-tools-2.4.6-90.el7.centos.x86_64 2/3 Verifying : apr-util-1.5.2-6.el7.x86_64 3/3 Installed: httpd-tools.x86_64 0:2.4.6-90.el7.centos Dependency Installed: apr.x86_64 0:1.4.8-5.el7 apr-util.x86_64 0:1.5.2-6.el7 Complete! [root@node105.yinzhengjie.org.cn ~]#
[root@node105.yinzhengjie.org.cn ~]# ab -n100000 -c 2000 http://node101.yinzhengjie.org.cn/static/access.log This is ApacheBench, Version 2.3 <$Revision: 1430300 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking node101.yinzhengjie.org.cn (be patient) Completed 10000 requests Completed 20000 requests Completed 30000 requests Completed 40000 requests Completed 50000 requests Completed 60000 requests Completed 70000 requests Completed 80000 requests Completed 90000 requests Completed 100000 requests Finished 100000 requests Server Software: Tengine Server Hostname: node101.yinzhengjie.org.cn Server Port: 80 Document Path: /static/access.log Document Length: 348797 bytes Concurrency Level: 2000 Time taken for tests: 242.545 seconds Complete requests: 100000 Failed requests: 848 (Connect: 0, Receive: 0, Length: 848, Exceptions: 0) Write errors: 0 Non-2xx responses: 767 Total transferred: 34617054848 bytes HTML transferred: 34591725127 bytes Requests per second: 412.30 [#/sec] (mean) Time per request: 4850.895 [ms] (mean) Time per request: 2.425 [ms] (mean, across all concurrent requests) Transfer rate: 139379.31 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 357 828.0 82 15103 Processing: 84 3767 6267.4 2446 138604 Waiting: 1 2235 5548.2 750 116412 Total: 86 4124 6317.0 2762 138716 Percentage of the requests served within a certain time (ms) 50% 2762 66% 3426 75% 3967 80% 4433 90% 6242 95% 10528 98% 19385 99% 32721 100% 138716 (longest request) [root@node105.yinzhengjie.org.cn ~]#
[root@node105.yinzhengjie.org.cn ~]# ab -n100000 -c 2000 http://node101.yinzhengjie.org.cn/static/access.log This is ApacheBench, Version 2.3 <$Revision: 1430300 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking node101.yinzhengjie.org.cn (be patient) Completed 10000 requests Completed 20000 requests Completed 30000 requests Completed 40000 requests Completed 50000 requests Completed 60000 requests Completed 70000 requests Completed 80000 requests Completed 90000 requests Completed 100000 requests Finished 100000 requests Server Software: Tengine Server Hostname: node101.yinzhengjie.org.cn Server Port: 80 Document Path: /static/access.log Document Length: 348797 bytes Concurrency Level: 2000 Time taken for tests: 231.811 seconds Complete requests: 100000 Failed requests: 401 (Connect: 0, Receive: 0, Length: 401, Exceptions: 0) Write errors: 0 Non-2xx responses: 394 Total transferred: 34766790357 bytes HTML transferred: 34741426365 bytes Requests per second: 431.39 [#/sec] (mean) Time per request: 4636.213 [ms] (mean) Time per request: 2.318 [ms] (mean, across all concurrent requests) Transfer rate: 146464.11 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 452 1107.2 97 15208 Processing: 93 3582 5036.2 2618 114911 Waiting: 1 1903 4579.4 857 73779 Total: 109 4033 5131.7 3010 114972 Percentage of the requests served within a certain time (ms) 50% 3010 66% 3525 75% 3996 80% 4442 90% 6044 95% 9172 98% 17817 99% 30033 100% 114972 (longest request) [root@node105.yinzhengjie.org.cn ~]#
[root@node105.yinzhengjie.org.cn ~]# ab -n100000 -c 2000 http://node101.yinzhengjie.org.cn/static/access.log This is ApacheBench, Version 2.3 <$Revision: 1430300 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking node101.yinzhengjie.org.cn (be patient) Completed 10000 requests Completed 20000 requests Completed 30000 requests Completed 40000 requests Completed 50000 requests Completed 60000 requests Completed 70000 requests Completed 80000 requests Completed 90000 requests Completed 100000 requests Finished 100000 requests Server Software: Tengine Server Hostname: node101.yinzhengjie.org.cn Server Port: 80 Document Path: /static/access.log Document Length: 348797 bytes Concurrency Level: 2000 Time taken for tests: 225.578 seconds Complete requests: 100000 Failed requests: 634 (Connect: 0, Receive: 0, Length: 634, Exceptions: 0) Write errors: 0 Non-2xx responses: 631 Total transferred: 34684484292 bytes HTML transferred: 34659141229 bytes Requests per second: 443.31 [#/sec] (mean) Time per request: 4511.555 [ms] (mean) Time per request: 2.256 [ms] (mean, across all concurrent requests) Transfer rate: 150154.73 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 441 1380.2 82 31163 Processing: 58 3554 5717.6 2413 120031 Waiting: 1 2131 5636.6 893 93156 Total: 62 3995 5863.4 2708 120125 Percentage of the requests served within a certain time (ms) 50% 2708 66% 3280 75% 3809 80% 4309 90% 6109 95% 9598 98% 19167 99% 33881 100% 120125 (longest request) [root@node105.yinzhengjie.org.cn ~]#
[root@node105.yinzhengjie.org.cn ~]# ab -n100000 -c 2000 http://node101.yinzhengjie.org.cn/static/access.log This is ApacheBench, Version 2.3 <$Revision: 1430300 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking node101.yinzhengjie.org.cn (be patient) Completed 10000 requests Completed 20000 requests Completed 30000 requests Completed 40000 requests Completed 50000 requests Completed 60000 requests Completed 70000 requests Completed 80000 requests Completed 90000 requests Completed 100000 requests Finished 100000 requests Server Software: Tengine Server Hostname: node101.yinzhengjie.org.cn Server Port: 80 Document Path: /static/access.log Document Length: 348797 bytes Concurrency Level: 2000 Time taken for tests: 230.977 seconds Complete requests: 100000 Failed requests: 832 (Connect: 0, Receive: 0, Length: 832, Exceptions: 0) Write errors: 0 Non-2xx responses: 829 Total transferred: 34615832624 bytes HTML transferred: 34590507457 bytes Requests per second: 432.94 [#/sec] (mean) Time per request: 4619.532 [ms] (mean) Time per request: 2.310 [ms] (mean, across all concurrent requests) Transfer rate: 146354.77 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 310 611.3 79 15198 Processing: 95 3782 6025.0 2451 84089 Waiting: 2 2246 5877.4 976 67208 Total: 103 4092 6052.2 2701 84949 Percentage of the requests served within a certain time (ms) 50% 2701 66% 3277 75% 3786 80% 4325 90% 6014 95% 9715 98% 19764 99% 35307 100% 84949 (longest request) [root@node105.yinzhengjie.org.cn ~]#
[root@node105.yinzhengjie.org.cn ~]# ab -n100000 -c 2000 http://node101.yinzhengjie.org.cn/static/access.log This is ApacheBench, Version 2.3 <$Revision: 1430300 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking node101.yinzhengjie.org.cn (be patient) Completed 10000 requests Completed 20000 requests Completed 30000 requests Completed 40000 requests Completed 50000 requests Completed 60000 requests Completed 70000 requests Completed 80000 requests Completed 90000 requests Completed 100000 requests Finished 100000 requests Server Software: Tengine Server Hostname: node101.yinzhengjie.org.cn Server Port: 80 Document Path: /static/access.log Document Length: 348797 bytes Concurrency Level: 2000 Time taken for tests: 258.968 seconds Complete requests: 100000 Failed requests: 1658 (Connect: 0, Receive: 0, Length: 1658, Exceptions: 0) Write errors: 0 Non-2xx responses: 1624 Total transferred: 34329476704 bytes HTML transferred: 34304223362 bytes Requests per second: 386.15 [#/sec] (mean) Time per request: 5179.360 [ms] (mean) Time per request: 2.590 [ms] (mean, across all concurrent requests) Transfer rate: 129455.68 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 457 1155.9 79 15073 Processing: 4 3902 7964.1 2252 112143 Waiting: 1 2540 7596.7 651 89124 Total: 4 4359 8038.9 2568 112190 Percentage of the requests served within a certain time (ms) 50% 2568 66% 3137 75% 3691 80% 4207 90% 6468 95% 10455 98% 33939 99% 53567 100% 112190 (longest request) [root@node105.yinzhengjie.org.cn ~]#
未配置Nginx緩存時，對Nginx壓力測試我們以每秒完成請求數(Requests per second)作為參照點: 第一次測試Requests per second值: 412.30 [#/sec] (mean) 第二次測試Requests per second值: 431.39 [#/sec] (mean) 第三次測試Requests per second值: 443.31 [#/sec] (mean) 第四次測試Requests per second值: 432.94 [#/sec] (mean) 第五次測試Requests per second值: 386.15 [#/sec] (mean) 去掉一個最高值(443.31)和一個最低值(386.15)，算得平均數為:"425 [#/sec] (mean)" 溫馨提示: 如下圖所示，測試時可能本地的CPU使用率會飆高，屬於正常現象。
二.nginx配置緩存后再一次做壓力測試並記錄結果
1>.在nginx的主配置文件中定義可用於proxy功能的緩存。
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/nginx/conf/nginx.conf worker_processes 4; worker_cpu_affinity 00000001 00000010 00000100 00001000; events { worker_connections 100000; use epoll; accept_mutex on; multi_accept on; } http { include mime.types; default_type text/html; server_tokens off; charset utf-8; log_format my_access_json '{"@timestamp":"$time_iso8601",' '"host":"$server_addr",' '"clientip":"$remote_addr",' '"size":$body_bytes_sent,' '"responsetime":$request_ti me,' '"upstreamtime":"$upstream_response_time",' '"upstreamhost":"$upstream_addr",' '"http_host":"$host",' '"uri":"$uri",' '"domain":"$host",' '"xff":"$http_x_forwarded_for",' '"referer":"$http_referer",' '"tcp_xff":"$proxy_protocol_addr",' '"http_user_agent":"$http_user_agent",' '"status":"$status"}'; access_log logs/access_json.log my_access_json; #配置Nginx反向代理的緩存 proxy_cache_path /yinzhengjie/data/web/nginx/proxycache levels=1:2:2 keys_zone=proxycache:512m inactive=10m max_size=1g; ssl_certificate /yinzhengjie/softwares/nginx/certs/www.yinzhengjie.org.cn.crt; ssl_certificate_key /yinzhengjie/softwares/nginx/certs/www.yinzhengjie.org.cn.key; ssl_session_cache shared:sslcache:20m; ssl_session_timeout 10m; include /yinzhengjie/softwares/nginx/conf.d/*.conf; } [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# nginx -t nginx: the configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf syntax is ok nginx: configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf test is successful [root@node101.yinzhengjie.org.cn ~]#
2>.在子配置文件中調用緩存功能
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/nginx/conf.d/node101_yinzhengjie_org.cn.conf server { listen 80; listen 443 ssl; server_name node101.yinzhengjie.org.cn; access_log /yinzhengjie/softwares/nginx/logs/node101_yinzhengjie_org_cn_access.log my_access_json; error_log /yinzhengjie/softwares/nginx/logs/node101_yinzhengjie_org_cn_error.log; location / { root /yinzhengjie/data/web/nginx/static/cn; index index.html; #定義有效的請求referer,用空格隔開即可 valid_referers none blocked server_names *.baidu.com example.* ~\.google\.; #如果沒有在上面的有效鏈接定義那么均屬於無效請求referer if ($invalid_referer) { return 403; } #如果是一些常見的壓測試工具,咱們直接進給他拒絕訪問 if ($http_user_agent ~ "ApacheBench|WebBench|TurnitinBot|Sougou web spider|Grid Server"){ return 403; } } location = /favicon.ico { root /yinzhengjie/data/web/nginx/images/jd; } location /app01 { proxy_pass http://172.30.1.108/; proxy_connect_timeout 60s; } location /static { proxy_pass http://172.30.1.108; #指明調用的緩存區,這個名稱在Nginx的主配置文件中有定義 proxy_cache proxycache; #緩存中用於"鍵"的內容 proxy_cache_key $request_uri; #定義對特定響應碼的響應內容的緩存時長 proxy_cache_valid 200 302 301 10m; proxy_cache_valid any 5m; } location /image { proxy_pass http://172.30.1.108; proxy_hide_header ETag; proxy_set_header yinzhengjie_nginx_ip_forwarded $proxy_add_x_forwarded_for; } location /dynamic { proxy_pass http://172.30.1.108; proxy_set_header yinzhengjie_nginx_ip_forwarded $proxy_add_x_forwarded_for; } } [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# nginx -t nginx: the configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf syntax is ok nginx: configuration file /yinzhengjie/softwares/nginx/conf/nginx.conf test is successful [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/web/nginx/proxycache/ #很明顯，緩存目錄配置成功啦,該目錄不需要咱們創建，而是在使用語法檢測時自動創建的。 total 0 [root@node101.yinzhengjie.org.cn ~]#
3>.重新加載nginx的配置文件
[root@node101.yinzhengjie.org.cn ~]# ps -ef | grep nginx | grep -v grep root 21509 1 0 16:34 ? 00:00:00 nginx: master process nginx nginx 21759 21509 0 17:33 ? 00:00:00 nginx: worker process nginx 21760 21509 0 17:33 ? 00:00:00 nginx: worker process nginx 21761 21509 0 17:33 ? 00:00:00 nginx: worker process nginx 21762 21509 0 17:33 ? 00:00:00 nginx: worker process nginx 21763 21509 0 17:33 ? 00:00:00 nginx: cache manager process nginx 21764 21509 0 17:33 ? 00:00:00 nginx: cache loader process [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# nginx -s reload [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ps -ef | grep nginx | grep -v grep root 21509 1 0 16:34 ? 00:00:00 nginx: master process nginx nginx 21782 21509 0 17:37 ? 00:00:00 nginx: worker process nginx 21783 21509 0 17:37 ? 00:00:00 nginx: worker process nginx 21784 21509 0 17:37 ? 00:00:00 nginx: worker process nginx 21785 21509 0 17:37 ? 00:00:00 nginx: worker process nginx 21786 21509 0 17:37 ? 00:00:00 nginx: cache manager process [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
4>.在node105.yinzhengjie.org.cn使用ab命令對nginx進行壓力測試
[root@node105.yinzhengjie.org.cn ~]# ab -n100000 -c 2000 http://node101.yinzhengjie.org.cn/static/access.log This is ApacheBench, Version 2.3 <$Revision: 1430300 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking node101.yinzhengjie.org.cn (be patient) Completed 10000 requests Completed 20000 requests Completed 30000 requests Completed 40000 requests Completed 50000 requests Completed 60000 requests Completed 70000 requests Completed 80000 requests Completed 90000 requests Completed 100000 requests Finished 100000 requests Server Software: Tengine Server Hostname: node101.yinzhengjie.org.cn Server Port: 80 Document Path: /static/access.log Document Length: 348797 bytes Concurrency Level: 2000 Time taken for tests: 175.025 seconds Complete requests: 100000 Failed requests: 0 Write errors: 0 Total transferred: 34905100000 bytes HTML transferred: 34879700000 bytes Requests per second: 571.35 [#/sec] (mean) Time per request: 3500.501 [ms] (mean) Time per request: 1.750 [ms] (mean, across all concurrent requests) Transfer rate: 194755.07 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 490 851.9 165 7269 Processing: 117 2997 698.7 3160 6984 Waiting: 1 205 228.3 123 2027 Total: 118 3487 1032.6 3428 10223 Percentage of the requests served within a certain time (ms) 50% 3428 66% 3588 75% 3743 80% 3968 90% 4427 95% 5179 98% 6285 99% 6770 100% 10223 (longest request) [root@node105.yinzhengjie.org.cn ~]#
[root@node105.yinzhengjie.org.cn ~]# ab -n100000 -c 2000 http://node101.yinzhengjie.org.cn/static/access.log This is ApacheBench, Version 2.3 <$Revision: 1430300 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking node101.yinzhengjie.org.cn (be patient) Completed 10000 requests Completed 20000 requests Completed 30000 requests Completed 40000 requests Completed 50000 requests Completed 60000 requests Completed 70000 requests Completed 80000 requests Completed 90000 requests Completed 100000 requests Finished 100000 requests Server Software: Tengine Server Hostname: node101.yinzhengjie.org.cn Server Port: 80 Document Path: /static/access.log Document Length: 348797 bytes Concurrency Level: 2000 Time taken for tests: 173.924 seconds Complete requests: 100000 Failed requests: 0 Write errors: 0 Total transferred: 34905100000 bytes HTML transferred: 34879700000 bytes Requests per second: 574.96 [#/sec] (mean) Time per request: 3478.473 [ms] (mean) Time per request: 1.739 [ms] (mean, across all concurrent requests) Transfer rate: 195988.37 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 314 451.2 148 3278 Processing: 259 3144 661.4 3197 8202 Waiting: 0 244 419.5 122 4904 Total: 466 3457 798.2 3407 8355 Percentage of the requests served within a certain time (ms) 50% 3407 66% 3493 75% 3637 80% 3845 90% 4305 95% 4623 98% 6017 99% 6370 100% 8355 (longest request) [root@node105.yinzhengjie.org.cn ~]#
[root@node105.yinzhengjie.org.cn ~]# ab -n100000 -c 2000 http://node101.yinzhengjie.org.cn/static/access.log This is ApacheBench, Version 2.3 <$Revision: 1430300 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking node101.yinzhengjie.org.cn (be patient) Completed 10000 requests Completed 20000 requests Completed 30000 requests Completed 40000 requests Completed 50000 requests Completed 60000 requests Completed 70000 requests Completed 80000 requests Completed 90000 requests Completed 100000 requests Finished 100000 requests Server Software: Tengine Server Hostname: node101.yinzhengjie.org.cn Server Port: 80 Document Path: /static/access.log Document Length: 348797 bytes Concurrency Level: 2000 Time taken for tests: 172.614 seconds Complete requests: 100000 Failed requests: 0 Write errors: 0 Total transferred: 34905100000 bytes HTML transferred: 34879700000 bytes Requests per second: 579.33 [#/sec] (mean) Time per request: 3452.287 [ms] (mean) Time per request: 1.726 [ms] (mean, across all concurrent requests) Transfer rate: 197474.96 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 275 338.1 153 3355 Processing: 2 3166 464.8 3246 8819 Waiting: 0 189 222.7 117 1900 Total: 2 3442 562.0 3436 9899 Percentage of the requests served within a certain time (ms) 50% 3436 66% 3495 75% 3556 80% 3636 90% 4157 95% 4449 98% 4866 99% 5166 100% 9899 (longest request) [root@node105.yinzhengjie.org.cn ~]#
[root@node105.yinzhengjie.org.cn ~]# ab -n100000 -c 2000 http://node101.yinzhengjie.org.cn/static/access.log This is ApacheBench, Version 2.3 <$Revision: 1430300 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking node101.yinzhengjie.org.cn (be patient) Completed 10000 requests Completed 20000 requests Completed 30000 requests Completed 40000 requests Completed 50000 requests Completed 60000 requests Completed 70000 requests Completed 80000 requests Completed 90000 requests Completed 100000 requests Finished 100000 requests Server Software: Tengine Server Hostname: node101.yinzhengjie.org.cn Server Port: 80 Document Path: /static/access.log Document Length: 348797 bytes Concurrency Level: 2000 Time taken for tests: 172.927 seconds Complete requests: 100000 Failed requests: 0 Write errors: 0 Total transferred: 34905100000 bytes HTML transferred: 34879700000 bytes Requests per second: 578.28 [#/sec] (mean) Time per request: 3458.539 [ms] (mean) Time per request: 1.729 [ms] (mean, across all concurrent requests) Transfer rate: 197117.96 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 1 318 440.8 155 3311 Processing: 285 3130 615.1 3215 7415 Waiting: 0 230 361.8 119 4259 Total: 424 3448 744.7 3436 7538 Percentage of the requests served within a certain time (ms) 50% 3436 66% 3491 75% 3561 80% 3739 90% 4243 95% 4692 98% 5574 99% 5928 100% 7538 (longest request) [root@node105.yinzhengjie.org.cn ~]#
[root@node105.yinzhengjie.org.cn ~]# ab -n100000 -c 2000 http://node101.yinzhengjie.org.cn/static/access.log This is ApacheBench, Version 2.3 <$Revision: 1430300 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking node101.yinzhengjie.org.cn (be patient) Completed 10000 requests Completed 20000 requests Completed 30000 requests Completed 40000 requests Completed 50000 requests Completed 60000 requests Completed 70000 requests Completed 80000 requests Completed 90000 requests Completed 100000 requests Finished 100000 requests Server Software: Tengine Server Hostname: node101.yinzhengjie.org.cn Server Port: 80 Document Path: /static/access.log Document Length: 348797 bytes Concurrency Level: 2000 Time taken for tests: 172.397 seconds Complete requests: 100000 Failed requests: 0 Write errors: 0 Total transferred: 34905100000 bytes HTML transferred: 34879700000 bytes Requests per second: 580.06 [#/sec] (mean) Time per request: 3447.941 [ms] (mean) Time per request: 1.724 [ms] (mean, across all concurrent requests) Transfer rate: 197723.90 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 412 613.4 156 7132 Processing: 309 3020 694.8 3134 7696 Waiting: 0 241 409.1 118 4803 Total: 378 3432 894.5 3408 10841 Percentage of the requests served within a certain time (ms) 50% 3408 66% 3506 75% 3725 80% 3914 90% 4318 95% 4864 98% 5992 99% 6444 100% 10841 (longest request) [root@node105.yinzhengjie.org.cn ~]# [root@node105.yinzhengjie.org.cn ~]#
未配置Nginx緩存時，對Nginx壓力測試我們以每秒完成請求數(Requests per second)作為參照點: 第一次測試Requests per second值: 571.35 [#/sec] (mean) 第二次測試Requests per second值: 574.96 [#/sec] (mean) 第三次測試Requests per second值: 579.33 [#/sec] (mean) 第四次測試Requests per second值: 578.28 [#/sec] (mean) 第五次測試Requests per second值: 580.06 [#/sec] (mean) 去掉一個最高值(580.06)和一個最低值(571.35)，算得平均數為:"577 [#/sec] (mean)"。 溫馨提示: 相比未加速前的"425 [#/sec] (mean)"來說，的確是優速提示，在原來的基礎上提升了35%的速度(計算方式:"(577-425) * 100 / 425 ")
5>.查看緩存目錄
[root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/web/nginx/proxycache/ total 0 drwx------ 3 nginx nginx 16 Dec 26 17:37 4 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/web/nginx/proxycache/4/ total 0 drwx------ 3 nginx nginx 16 Dec 26 17:37 61 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/web/nginx/proxycache/4/61/ total 0 drwx------ 2 nginx nginx 46 Dec 26 17:37 8a [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/web/nginx/proxycache/4/61/8a/ #我們發現被緩存的文件后綴5為個數字是該文件存儲的目錄名稱，但是該緩存文件明顯比真正的文件要大一點。這是為什么呢？你先猜猜看，接下來跟我一起來揭曉答案。 total 344 -rw------- 1 nginx nginx 349426 Dec 26 17:37 8163c0ca4d4d0c1ec72229042cf8a614 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# tail -1 /yinzhengjie/data/web/nginx/proxycache/4/61/8a/8163c0ca4d4d0c1ec72229042cf8a614 #出於好奇，我查看了緩存文件的內容，的確保存着數據的。 {"@timestamp":"2019-12-26T16:37:14+08:00","host":"172.30.1.101","clientip":"172.30.1.254","size":25214,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host ":"node101.yinzhengjie.org.cn","uri":"/favicon.ico","domain":"node101.yinzhengjie.org.cn","xff":"-","referer":"http://node101.yinzhengjie.org.cn/","tcp_xff":"","http_user_a gent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36","status":"200"} [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ssh node108.yinzhengjie.org.cn #為了驗證我的猜想，我遠程到Apache httpd服務器上，查看該文件的最后一行數據進行對比。 Last login: Thu Dec 26 15:27:02 2019 from 172.30.1.254 [root@node108.yinzhengjie.org.cn ~]# [root@node108.yinzhengjie.org.cn ~]# ll /var/www/html/static/ total 348 -rw-r--r-- 1 root root 348797 Dec 26 17:00 access.log -rw-r--r-- 1 root root 39 Dec 26 14:18 index.html [root@node108.yinzhengjie.org.cn ~]# [root@node108.yinzhengjie.org.cn ~]# tail -1 /var/www/html/static/access.log #登錄到服務器后我來查看該文件的最后一行，發現數據和Nginx緩存的完全吻合，但是我很好奇為什么Nginx緩存的文件會稍微打一點呢？我猜測是Apache httpd發送給Nginx數據時可能會夾雜着響應報文之類的信息，這才是導致Nginx緩存文件要稍大於Apche Httpd幀數保存的數據文件。 {"@timestamp":"2019-12-26T16:37:14+08:00","host":"172.30.1.101","clientip":"172.30.1.254","size":25214,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host ":"node101.yinzhengjie.org.cn","uri":"/favicon.ico","domain":"node101.yinzhengjie.org.cn","xff":"-","referer":"http://node101.yinzhengjie.org.cn/","tcp_xff":"","http_user_a gent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36","status":"200"} [root@node108.yinzhengjie.org.cn ~]# [root@node108.yinzhengjie.org.cn ~]# exit #為了驗證我的想法，立馬返回Nginx服務器 logout Connection to node108.yinzhengjie.org.cn closed. [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# head -15 /yinzhengjie/data/web/nginx/proxycache/4/61/8a/8163c0ca4d4d0c1ec72229042cf8a614 #我們查看緩存文件的頭部，果不其然，該文件的頭部的確是有響應報文，這樣用戶訪問時，可以直接將這個文件發送給請求的用戶而無需再將數據封裝成報文啦！從而速度得到提示。 ^ v^5^鑥㧁u"5527d-59a979b183760" KEY: /static/access.log HTTP/1.1 200 OK Date: Thu, 26 Dec 2019 10:20:21 GMT Server: Apache/2.4.6 (CentOS) Last-Modified: Thu, 26 Dec 2019 09:00:16 GMT ETag: "5527d-59a979b183760" Accept-Ranges: bytes Content-Length: 348797 Connection: close Content-Type: text/plain; charset=UTF-8 {"@timestamp":"2019-12-26T16:35:12+08:00","host":"172.30.1.101","clientip":"172.30.1.254","size":566,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host": "node101.yinzhengjie.org.cn","uri":"/index.html","domain":"node101.yinzhengjie.org.cn","xff":"-","referer":"-","tcp_xff":"","http_user_agent":"Mozilla/5.0 (Windows NT 10.0;
Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36","status":"200"}
{"@timestamp":"2019-12-26T16:35:12+08:00","host":"172.30.1.101","clientip":"172.30.1.254","size":1025154,"responsetime":0.004,"upstreamtime":"-","upstreamhost":"-","http_ho st":"node101.yinzhengjie.org.cn","uri":"/css/01.png","domain":"node101.yinzhengjie.org.cn","xff":"-","referer":"http://node101.yinzhengjie.org.cn/","tcp_xff":"","http_user_
agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36","status":"200"}
{"@timestamp":"2019-12-26T16:35:24+08:00","host":"172.30.1.101","clientip":"172.30.1.254","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"n ode101.yinzhengjie.org.cn","uri":"/index.html","domain":"node101.yinzhengjie.org.cn","xff":"-","referer":"-","tcp_xff":"","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; W
in64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36","status":"304"}
[root@node101.yinzhengjie.org.cn ~]#
6>.等待10分鍾后，再查看緩存目錄
[root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/web/nginx/proxycache/ total 0 drwx------ 3 nginx nginx 16 Dec 26 17:37 4 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/web/nginx/proxycache/4/ total 0 drwx------ 3 nginx nginx 16 Dec 26 17:37 61 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/web/nginx/proxycache/4/61/ total 0 drwx------ 2 nginx nginx 6 Dec 26 18:08 8a [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/web/nginx/proxycache/4/61/8a/ #10分鍾后我發現之前的緩存文件自動被清理啦。這是因為我們在Nginx的子配置文件中已經定義好了文件的緩存時間。 total 0 [root@node101.yinzhengjie.org.cn ~]#
三.Nginx服務器緩存相關功能常用的配置指令說明
1>.proxy_cache
指明調用的緩存，或關閉緩存機制，默認關閉，可配置在http,server,location區域。 博主推薦閱讀: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache
2>.proxy_cache_key
緩存中用於“鍵”的內容，默認值:"proxy_cache_key $scheme$proxy_host$request_uri;",可配置在http, server, location區域。 博主推薦閱讀: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_key
3>.proxy_cache_valid
定義對特定響應碼的響應內容的緩存時長,可配置在:https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_valid 博主推薦閱讀: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_valid
4>.proxy_cache_path
定義可用於proxy功能的緩存,默認是沒有配置的，可配置在http中，語法格式如下: proxy_cache_path path [levels=levels] [use_temp_path=on|off] keys_zone=name:size [inactive=time] [max_size=size] [manager_files=number] [manager_sleep=time] [manager_threshold=time] [loader_files=number] [loader_sleep=time] [loader_threshold=time] [purger=on|off] [purger_files=number] [purger_sleep=time] [purger_threshold=time]; 示例: proxy_cache_path /yinzhengjie/data/web/nginx/proxycache levels=1:2:2 keys_zone=proxycache:512m inactive=10m max_size=1g; 以上參數說明: "/yinzhengjie/data/web/nginx/proxycache":
定義緩存保存路徑，proxycache目錄會自動創建。 "levels=1:2:2":
定義目錄層級結構，1:2:2可以生成2^4*2^8*2^8=1048576個目錄。 "keys_zone=proxycache:512m":
指定內存中緩存的大小，主要用於存放key和metadata(如緩存文件命中次數)，建議不要設置的太小，也不要設置的過大。根據自己的服務器剩余空間來設置合理的值。 "inactive=10m" 指定緩存有效時間，若超出該時間的緩存文件會被刪除喲~ "max_size=1g": 最大磁盤占用空間，磁盤存入文件內容的緩存空間最大值。 博主推薦閱讀: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_path
5>.proxy_cache_use_stale
在被代理的后端服務器出現哪種情況下，可直接使用過期的緩存響應客戶端，默認是關閉的，可配置在http, server, location。語法格式如下： proxy_cache_use_stale error | timeout | invalid_header | updating | http_500 | http_502 | http_503 | http_504 | http_403 | http_404 | http_429 | off ...; 博主推薦閱讀: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_use_stale
6>.proxy_cache_methods
對哪些客戶端請求方法對應的響應進行緩存，默認GET和HEAD方法總是被緩存。可配置於 http, server, location。 博主推薦閱讀: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_methods