k8s- nginx ingress 高可用部署（最新版，支持 k8s 1.22---1.19）第2篇

 nginx-ingress-controller 安裝完畢，接下來開始對  nginx-ingress-controller 實現高可用。我們通過 keepalive+nginx 實現 nginx-ingress-controller 高可用！

注意：這里的keepalive+nginx 僅僅是對  nginx-ingress-controller 實現高可用！本教程里都是在Worker節點操作，即安裝了nginx-ingress-controller 的節點操作。

 

一、安裝keepalive+nginx 

3台安裝了nginx-ingress-controller的worker節點上分別安裝。

[root@k8snode1 mytest]# yum install nginx keepalived -y

[root@k8snode2 mytest]# yum install nginx keepalived -y

[root@k8snode3 mytest]# yum install nginx keepalived -y

 

二、修改 nginx 配置文件。主備一樣

vim /etc/nginx/nginx.conf

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
# 四層負載均衡，為兩台 Master apiserver 組件提供負載均衡
stream {
log_format main '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';

access_log /var/log/nginx/k8s-access.log main;

upstream k8s-apiserver {
server 192.168.157.202:80; # Master1 APISERVER IP:PORT
server 192.168.157.203:80; # Master2 APISERVER IP:PORT

server 192.168.157.204:80; # Master2 APISERVER IP:PORT
}

server {
listen 10080;
proxy_pass k8s-apiserver;
}
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';

access_log /var/log/nginx/access.log main;

sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
server {
listen 8080 default_server;
server_name _;

location / {
}
}
}

 請將 upstream k8s-apiserver里的ip設置為安裝了 nginx-ingress controller對應的宿主機的IP地址。端口號為80.

三、keepalive 配置

雙機熱備，其中主節點和備節點的配置稍稍有所不同。

vim   /etc/keepalived/keepalived.conf

global_defs {
   notification_email {
     [e-mail]
     [e-mail]
     [e-mail]
   }
   notification_email_from [e-mail]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id NGINX_MASTER
}

vrrp_script check_nginx {
    script "/etc/keepalived/check_nginx.sh"
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33  # 修改為實際網卡名
    virtual_router_id 51 # VRRP 路由 ID實例，每個實例是唯一的
    priority 100    # 優先級，備服務器設置 90
    advert_int 1    # 指定VRRP 心跳包通告間隔時間，默認1秒
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    # 虛擬IP
    virtual_ipaddress {
        192.168.157.199/24
    }
    track_script {
        check_nginx
    }
}

#vrrp_script：指定檢查nginx工作狀態腳本（根據nginx狀態判斷是否故障轉移）
#virtual_ipaddress：虛擬IP（VIP）

需要修改4個地方：

1、 interface +本機的網卡名

2、state MASTER ，若為主節點則為MASTER，備節點為BACKUP ,都是大寫字母。

3、priority 100  優先級，主節點MASTER則設置100，備節點設置比100小即可，比如90 .

4、virtual_ipaddress  虛擬IP，設置為當前主機環境一個未被使用的IP。

 從該配置文件可以看出，還需要創建一個check_nginx.sh 文件。

 

三、創建check_nginx.sh

vim /etc/keepalived/check_nginx.sh

#!/bin/bash
#1、判斷Nginx是否存活
counter=`ps -C nginx --no-header | wc -l`
if [ $counter -eq 0 ]; then
    #2、如果不存活則嘗試啟動Nginx
    service nginx start
    sleep 2
    #3、等待2秒后再次獲取一次Nginx狀態
    counter=`ps -C nginx --no-header | wc -l`
    #4、再次進行判斷，如Nginx還不存活則停止Keepalived，讓地址進行漂移
    if [ $counter -eq 0 ]; then
        service  keepalived stop
    fi
fi

四、3台機器上執行

1、需要安裝一個插件

yum install nginx-mod-stream -y

2、開啟

systemctl daemon-reload

systemctl enable nginx keepalived

systemctl start nginx

systemctl start keepalived

 

教程中涉及到的文件可以下載：

鏈接：https://pan.baidu.com/s/1oRvhN2_nfVT2ndE2VEN2QQ
提取碼：muxx