openresty + lua-resty-weedfs + weedfs + graphicsmagick動態生成縮略圖（類似淘寶方案）
--大部分的網站都要涉及到圖片縮略圖的處理，比如新聞配圖，電商，微信朋友圈等，貼別是電商類網站，每個商品圖對應多個不同尺寸的縮略圖，用於不同的頁面。
網站剛開始的時候，大家為了趕工期，盡快完成開發任務，基本上都會選擇比較簡單的方式實現功能，直接使用web服務器的圖片上傳和下載的功能。而且是單機部署。但問題是，圖片響應速度比較慢，而且當訪問量逐漸加大，服務器由單台變為多台時，這種方式擴展性較差。
我們自己的解決方案規划了幾種：
- 使用web server上傳，后面使用openresty +lua + graphicsmagick 將生成縮略圖功能獨立出來，與web程序解耦。（目前在投入使用，准備換掉）
- 將圖片的上傳，管理交由ftpserver處理，后面使用openresty +lua + graphicsmagick 將生成縮略圖功能獨立出來，與web程序解耦。（目前在投入使用）
- 將圖片的上傳，分布式管理 交由seaweedfs管理，可提供更快的響應速度，更方便，無痛的分布式擴展節點，billion 級別的。后面使用openresty + lua-resty-weedfs + graphicsmagick 生成縮略圖功能獨立處理。（准備投入使用）
- 使用七牛、又拍雲提供的雲存儲及數據處理服務，解決圖片的處理、存儲、多節點訪問速度的問題，這種方式優點是方案成熟，相應的有一定費用和開發工作，另外有一些小概率的風險，比如雲服務掛掉影響本站訪問。（方便的解決方案，下一節階段重點考慮）
1，2 中解決方案，比較簡單，就不具體細說，下面重點配置實做一下方案3.
所用軟件版本信息：
服務器環境: ubuntu 13.10
openresty: 1.7.10.2
lua-resty-weedfs: master
weedfs(seaweedfs): v0.70beta
graphicsmagick: 1.3.16
安裝步驟如下：
1.安裝依賴的軟件包：
sudo apt-get install libreadline-dev libpcre3-dev libssl-dev perl
sudo apt-cache search libjpeg
sudo apt-cache search libpng
sudo apt-get install libjpeg62 libpng12-0
2:安裝openresty，現在最新版本的openresty1.7.10.2：
sudo mkdir working
cd working
sudo wget https://openresty.org/download/ngx_openresty-1.7.10.2.tar.gz
tar xzvf ngx_openresty-VERSION.tar.gz
cd ngx_openresty-VERSION/
sudo ./configure --with-luajit \
--with-http_iconv_module \
--with-http_stub_status_module --with-openssl=/usr/local/ssl/lib --with-http_ssl_module
后面兩個with，如果不能滿足條件，不安裝也可以
sudo make
sudo make install
安裝成功后，在/usr/local/openresty/
3: 安裝graphicsmagic
sudo add-apt-repository ppa:dhor/myway
sudo apt-get update
sudo apt-get install graphicsmagick
gm help
4:配置，並啟動openresty 也即nginx
cd ~/working
mkdir logs/ conf/
sudo vi conf/nginx.conf
輸入一下內容
=======start===========
worker_processes 1;
error_log logs/error.log;
events {
worker_connections 1024;
}
http {
server {
listen 8090;
location / {
default_type text/html;
content_by_lua '
ngx.say("<p>hello, world</p>")
';
}
}
}
=========end==============
添加nginx 到path，方便執行
PATH=/usr/local/openresty/nginx/sbin:$PATH
export PATH
nginx -p `pwd`/ -c conf/nginx.conf
curl http://localhost:8090/
輸出：
<p>hello, world</p>
5.安裝weedfs
sudo wget https://bintray.com/artifact/download/chrislusf/seaweedfs/weed_0.70beta_linux_amd64.tar.gz
tar zxvf weed_0.70beta_linux_amd64.tar.gz
start master server
./weed master
start volume server
cd ~/working
mkdir data
weed volume -dir="../data" -max=5 -mserver="localhost:9333" -port=9080 &
weed volume -dir="../data" -max=5 -mserver="localhost:9333" -port=9081 &
write file
curl -X POST http://localhost:9333/dir/assign
output:
{"fid":"7,02a3a52531","url":"127.0.0.1:9080","publicUrl":"127.0.0.1:9080","count":1}
second:
curl -X PUT -F file=@./detail_01.jpg http://127.0.0.1:9080/7,02a3a52531
output:
{"name":"detail_01.jpg","size":176946}
read file:
curl http://localhost:9333/dir/lookup?volumeId=7
output:
{"volumeId":"7","locations":[{"url":"127.0.0.1:9080","publicUrl":"127.0.0.1:9080"}]}
或者通過網頁顯示圖片：
http://192.168.15.201:9080/7/02a3a52531/detail_01.jpg
http://192.168.15.201:9080/7/02a3a52531.jpg
http://192.168.15.201:9080/7,02a3a52531.jpg
http://192.168.15.201:9080/7/02a3a52531
http://192.168.15.201:9080/7,02a3a52531
6.openresty 與 weedfs結合,配置 lua_resty_weedfs
cd ~/woring
git clone https://github.com/medcl/lua-resty-weedfs.git
cd /usr/local/openresty/nginx/conf
mv nginx.conf nginx.conf.backup
mv ~/woring/lua-resty-weedfs/resty ./resty
mv ~/woring/lua-resty-weedfs/nginx.conf ./nginx.conf
mv ~/woring/lua-resty-weedfs/weedfs.lua ./weedfs.lua
創建 www-data 組和用戶，創建路徑/home/wwwroot/weedfs.
nginx.conf 如下：
user www-data www-data;
worker_processes 8;
daemon on;
master_process off;
error_log logs/error.log error;
#error_log logs/error.log info;
pid logs/nginx.pid;
env MOCKEAGAIN_VERBOSE;
env MOCKEAGAIN_WRITE_TIMEOUT_PATTERN;
env LD_PRELOAD;
env DYLD_INSERT_LIBRARIES;
worker_rlimit_nofile 65535;
events {
worker_connections 65535;
}
http {
include mime.types;
default_type application/octet-stream;
# default_type text/plain;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log logs/access.log main;
server_names_hash_bucket_size 128;
client_header_buffer_size 32k;
large_client_header_buffers 4 32k;
client_max_body_size 8m;
keepalive_timeout 0;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
fastcgi_connect_timeout 300;
fastcgi_send_timeout 300;
fastcgi_read_timeout 300;
fastcgi_buffer_size 64k;
fastcgi_buffers 4 64k;
fastcgi_busy_buffers_size 128k;
fastcgi_temp_file_write_size 128k;
##cache##
proxy_connect_timeout 5;
proxy_read_timeout 60;
proxy_send_timeout 5;
proxy_buffer_size 16k;
proxy_buffers 4 64k;
proxy_busy_buffers_size 128k;
proxy_temp_file_write_size 128k;
proxy_temp_path temp_dir;
proxy_cache_path cache levels=1:2 keys_zone=cache_one:200m inactive=1d max_size=30g;
##end##
open_file_cache max=1000 inactive=20s;
open_file_cache_min_uses 5;
open_file_cache_valid 30s;
gzip on;
gzip_min_length 1k;
gzip_buffers 4 16k;
gzip_http_version 1.1;
gzip_comp_level 2;
gzip_types text/plain application/x-javascript text/css application/xml;
gzip_disable "MSIE [1-6]\.";
gzip_vary on;
#keepalive_timeout 65;
#problematic
#lua_code_cache off;
lua_package_path "/usr/local/openresty/lualib/?.lua;;";
resolver 8.8.8.8;
server {
listen 80;
server_name localhost;
rewrite_log on;
charset utf-8,gbk;
#access_log logs/host.access.log main;
#note:must end with“/”
set $weed_audio_root_url "http://127.0.0.1:9080/";
set $weed_audio_root_url "http://127.0.0.1:9081/";
set $weed_img_root_url "http://127.0.0.1:9080/";
set $weed_img_root_url "http://127.0.0.1:9081/";
set $local_img_fs_root "/home/wwwroot/weedfs/";
set $local_audio_fs_root "/home/wwwroot/weedfs/";
location / {
root /home/wwwroot/weedfs/;
index index.html index.htm;
}
#sample:/_img/?size=orig&volumn=1&id=1234
location /_img/{
default_type image/jpeg;
if ($request_method = 'DELETE' ) {
return 405;
}
if ($request_method = 'PUT' ) {
return 405;
}
if ($request_method = 'POST' ) {
return 405;
}
#content_by_lua 'ngx.say("<p>process here</p>")';
content_by_lua_file conf/weedfs.lua;
expires 30d;
# access_log off;
}
# location /img/orig/{
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $remote_addr;
# proxy_set_header Host $http_host;
#
# # if ($uri ~* "/img/orig/([0-9]+)/([a-z0-9]+)(\.[a-z]+)?") {
# # rewrite "/img/orig/([0-9]+)/([a-z0-9]+)(\.[a-z]+)" /img/orig/$1,$2$3 permanent;
# # break;
# # }
#
# proxy_pass http://192.168.2.100:8080/;
# expires 7d;
# break;
# }
location /img/{
rewrite "/img/([0-9]+x[0-9]+s?)/([0-9]+)/([a-z0-9]+)(\.[a-z]+)?" /_img/?type=img&size=$1&volumn=$2&id=$3 last;
rewrite "/img/([0-9]+x[0-9]+s?)/([0-9]+),([a-z0-9]+)(\.[a-z]+)?" /_img/?type=img&size=$1&volumn=$2&id=$3 last;
rewrite "/img/orig/([0-9]+)[,/]([a-z0-9]+)(\.[a-z]+)?" /_img/?type=img&size=orig&volumn=$1&id=$2 last;
expires 30d;
# access_log off;
}
location /_audio/{
default_type audio/mp3;
if ($request_method = 'DELETE' ) {
return 405;
}
if ($request_method = 'PUT' ) {
return 405;
}
if ($request_method = 'POST' ) {
return 405;
}
content_by_lua_file conf/weedfs.lua;
expires 30d;
# access_log off;
}
#if you specified audio_fs_root separately,you should change this.
#location /audios{
# default_type audio/mp3;
# root /home/wwwroot/audios;
# expires 30d;
# access_log off;
# }
location /audio/{
rewrite "/audio/(mp3)/([0-9]+)/([a-z0-9]+)(\.[a-z]+)?" /_audio/?type=audio&size=$1&volumn=$2&id=$3 last;
rewrite "/audio/(mp3)/([0-9]+),([a-z0-9]+)(\.[a-z]+)?" /_audio/?type=audio&size=$1&volumn=$2&id=$3 last;
rewrite "/audio/orig/([0-9]+),([a-z0-9]+)(\.[a-z]+)?" /_audio/?type=audio&size=orig&volumn=$1&id=$2 last;
expires 30d;
# access_log off;
}
# location /upload{
# if ($request_method = 'DELETE' ) {
# return 405;
# }
# if ($request_method = 'PUT' ) {
# return 405;
# }
#
# proxy_pass $weed_xxx_root_url;
# proxy_redirect default ;
# }
location /favicon.ico{
root /home/wwwroot/;
# access_log off;
}
error_page 404 /404.html;
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
location ~ /\.ht {
deny all;
}
location /status {
stub_status on;
access_log off;
}
}
}
7.重新啟動nginx 運行
訪問 http://192.168.15.201/img/100x100/7,02a3a52531.jpg 成功
http://192.168.15.201/img/100x100s/7,02a3a52531.jpg
增加s后，自動適應比例生成縮略圖
8、參考文檔