nginx和Docker的打怪升級

需求問題

最近做的一個項目有個需求是http與webscoket服務共存在一個nginx.conf文件中，最開始做的配置如下：

upstream app_server {
    # fail_timeout=0 means we always retry an upstream even if it failed
    # to return a good HTTP response

    # for UNIX domain socket setups
    # server unix:/tmp/gunicorn.sock fail_timeout=0;
    # for a TCP configuration
    server 10.6.14.200:8000 fail_timeout=0;
}

server {
    listen 80;
    # gzip config
    gzip on;
    gzip_min_length 1k;
    gzip_comp_level 9;
    gzip_types text/plain text/css text/javascript application/json application/javascript application/x-javascript application/xml;
    gzip_vary on;
    gzip_disable "MSIE [1-6]\.";

    root /usr/share/nginx/html;

    location / {
        try_files $uri $uri/ /index.html;
    }

    location /websocket {
        proxy_pass http://app_server;

        proxy_http_version 1.1;
        proxy_read_timeout 360s;   
        proxy_redirect off;   
        proxy_set_header Upgrade $http_upgrade; 
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host:$server_port;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header REMOTE-HOST $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

以上配置，http 協議的請求是沒有問題的，但是前端 websocket無法建立連接, 也不知道哪里出了問題。

優化策略: 既支持http又支持 ws 的配置

通過nginx官方關於WebSocket的配置得知，可以自定義變量。故配置如下，就可以做到既支持 ws 請求，又支持 http請求。

upstream app_server {
    # fail_timeout=0 means we always retry an upstream even if it failed
    # to return a good HTTP response

    # for UNIX domain socket setups
    # server unix:/tmp/gunicorn.sock fail_timeout=0;
    # for a TCP configuration
    server 10.6.14.200:8000 fail_timeout=0;
}

map $http_upgrade $connection_upgrade {
        default          keep-alive;  # 默認為keep-alive 可以支持 一般http請求
        'websocket'      upgrade;  # 如果為websocket 則為 upgrade 可升級的。
    }

server {
    listen 80;
    # gzip config
    gzip on;
    gzip_min_length 1k;
    gzip_comp_level 9;
    gzip_types text/plain text/css text/javascript application/json application/javascript application/x-javascript application/xml;
    gzip_vary on;
    gzip_disable "MSIE [1-6]\.";

    root /usr/share/nginx/html;

    location / {
        try_files $uri $uri/ /index.html;
    }

    location /websocket {
        proxy_pass http://app_server;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade; # 此處配置 上面定義的變量
        proxy_set_header Connection $connection_upgrade;
    }
}

優化再升級：通過環境變量配置后端host和port，啟動時動態連接后端

正常情況下，nginx是不支持直接讀取環境變量的，官方說明如下：

Using environment variables in nginx configuration:

Out-of-the-box, Nginx doesn't support using environment variables inside most configuration blocks.

But envsubst may be used as a workaround if you need to generate your nginx configuration dynamically before nginx starts.

envsubst這個命令可以進行傳參動態生成模板，所以算是曲線讀取環境變量

# nginx.conf.template
upstream app_server {
    # fail_timeout=0 means we always retry an upstream even if it failed
    # to return a good HTTP response

    # for UNIX domain socket setups
    # server unix:/tmp/gunicorn.sock fail_timeout=0;
    # for a TCP configuration
    server ${BACKEND_HOST}:${BACKEND_PORT} fail_timeout=0;
}

map $http_upgrade $connection_upgrade {
        default          keep-alive;  # 默認為keep-alive 可以支持 一般http請求
        'websocket'      upgrade;  # 如果為websocket 則為 upgrade 可升級的。
    }

server {
    listen 80;
    # gzip config
    gzip on;
    gzip_min_length 1k;
    gzip_comp_level 9;
    gzip_types text/plain text/css text/javascript application/json application/javascript application/x-javascript application/xml;
    gzip_vary on;
    gzip_disable "MSIE [1-6]\.";

    root /usr/share/nginx/html;

    location / {
        try_files $uri $uri/ /index.html;
    }

    location /websocket {
        proxy_pass http://app_server;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade; # 此處配置 上面定義的變量
        proxy_set_header Connection $connection_upgrade;
    }
}

通過設置環境變量BACKEND_HOST和BACKEND_PORT並執行envsubst '${BACKEND_HOST} ${BACKEND_PORT}' < nginx.conf.template > default.conf即可生成帶有環境變量的配置文件，然后再啟動nginx。

以上配置可以在打包鏡像時進行，所以前端目錄樹如下：

├── dist
├── ./docker-entrypoint.sh
├── ./Dockerfile
└── ./nginx.conf.template

# docker-entrypoint.sh

#!/usr/bin/env bash
set -eu

# shellcheck disable=SC2016
envsubst '${BACKEND_HOST} ${BACKEND_PORT}' < /etc/nginx/conf.d/default.conf.template > /etc/nginx/conf.d/default.conf

nginx -g "daemon off;"

# Dockerfile

FROM nginx:latest

COPY nginx.conf.template /etc/nginx/conf.d/default.conf.template

COPY docker-entrypoint.sh /

COPY dist /usr/share/nginx/html

EXPOSE 80

CMD ["/docker-entrypoint.sh"]

# nginx.conf.template

upstream app_server {
    # fail_timeout=0 means we always retry an upstream even if it failed
    # to return a good HTTP response

    # for UNIX domain socket setups
    # server unix:/tmp/gunicorn.sock fail_timeout=0;
    # for a TCP configuration
    server ${BACKEND_HOST}:${BACKEND_PORT} fail_timeout=0;
}

map $http_upgrade $connection_upgrade {
        default          keep-alive;  # 默認為keep-alive 可以支持 一般http請求
        'websocket'      upgrade;  # 如果為websocket 則為 upgrade 可升級的。
    }

server {
    listen 80;
    # gzip config
    gzip on;
    gzip_min_length 1k;
    gzip_comp_level 9;
    gzip_types text/plain text/css text/javascript application/json application/javascript application/x-javascript application/xml;
    gzip_vary on;
    gzip_disable "MSIE [1-6]\.";

    root /usr/share/nginx/html;

    location / {
        try_files $uri $uri/ /index.html;
    }

    location /websocket {
        proxy_pass http://app_server;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade; # 此處配置 上面定義的變量
        proxy_set_header Connection $connection_upgrade;
    }
}

• dist目錄即為前端項目打包后的文件目錄

• 構建鏡像命令為docker build -t iamgeName:Tag[] .

• 運行鏡像命令為docker run -d --restart=always -p 10086:80 -e BACKEND_HOST=10.6.14.200 -e BACKEND_PORT=8000 --name containerName imageName:Tag[]

以上內容一般適用於單鏡像運行，對於使用docker-compose或者k8s可以直接通過內部機制進行訪問會更加方便，就可以省去這些步驟啦