Docker+Nginx+Keepalived實現架構高可用

一、背景

通過keepalived實現nginx高可用，由於在家不想弄多台主機來搞，所以將運行環境用docker封裝來模擬跨主機

docker基礎鏡像：centos

 

說之前，簡單介紹一下：

Keepalived是基於vrrp協議的一款高可用軟件。Keepailived有一台主服務器和多台備份服務器，在主服務器和備份服務器上面部署相同的服務配置，使用一個虛擬IP地址對外提供服務，當主服務器出現故障時，虛擬IP地址會自動漂移到備份服務器。

 

雙機高可用方法目前分為兩種：

• 雙機主從（也叫雙機熱備）
• 雙機主主 （也叫雙機互備）

下述介紹，高可用中的雙機主從模式，雙機主主模式，主要是keepalived.conf配置會有所不同，方法都是一樣。

二、具體操作

1、安裝centos 鏡像

docker pull centos

說明：通過用centos鏡像來安裝高可用所需要的所有環境，再啟兩個容器，再真實模擬跨主機的場景

 

2、在centos上安裝所需環境（nginx和其它工具）

運行centos容器

docker run -it centos /bash/bin

 

安裝依賴和所需要的包　　

#使用yum安裝nginx需要包括Nginx的庫，安裝Nginx的庫

rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm

# 使用下面命令安裝nginx
#yum install nginx

#安裝網絡包(需要使用ifconfig和ping命令)
yum install net-tools

#安裝vim
yum install vim

 

3、在centos安裝keepalvied

#安裝keepalived環境依賴

yum install -y gcc openssl-devel popt-devel

#安裝keepalived

通過yum install keepalived



#或者通過源碼安裝

wget http://124.205.69.132/files/90630000053A2BB4/www.keepalived.org/software/keepalived-1.3.4.tar.gz

tar zxvf keepalived-1.3.4.tar.gz 

cd keepalived-1.3.4

./configure --prefix=/usr/local/keepalived

make && make install



拷貝幾個文件到CentOS7環境中：

cp keepalived-1.3.4/keepalived/etc/init.d/keepalived /etc/init.d/
mkdir /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp keepalived-1.3.4/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/

 

4、修改/etc/keepalived/keepalived.conf文件

! Configuration File for keepalived
global_defs {
    notification_email {
    	[e-mail]
    }
    notification_email_from [e-mail]
    smtp_server mail.example.com
    smtp_connect_timeout 30
    router_id LVS_DEVEL  
}


vrrp_script chk_nginx {
    script "/etc/keepalived/nginx_check.sh"  
    interval 2
    weight -5
    fall 3
    rise 2
}


vrrp_instance VI_1 {
    state MASTER   
    interface eth0   
    virtual_router_id 2  
    priority 101  
    advert_int 2
    authentication {
        auth_type PASS  
        auth_pass 1111
    }
    virtual_ipaddress {
        172.17.0.210   
    }
    track_script {
       chk_nginx 
    }

}

　　

/etc/keepalived/check_nginx.sh文件

A=`ps -ef | grep nginx | grep -v grep | wc -l`
if [ $A -eq 0 ];then
    nginx
    sleep 2
    if [ `ps -ef | grep nginx | grep -v grep | wc -l` -eq 0 ];then
        #killall keepalived
        ps -ef|grep keepalived|grep -v grep|awk '{print $2}'|xargs kill -9 
    fi

fi　　

 

再對check_nginx.sh賦於執行權限：

chmod +x check_nginx.sh

 

注：keepalived是通過檢測keepalived進程是否存在判斷服務器是否宕機，如果keepalived進程在但是nginx進程不在了那么keepalived是不會做主備切換，所以我們需要寫個腳本來監控nginx進程是否存在，如果nginx不存在就將keepalived進程殺掉。

在主nginx上需要編寫nginx進程檢測腳本（check_nginx.sh），判斷nginx進程是否存在，如果nginx不存在就將keepalived進程殺掉，並將vip漂移到備份機器上

 

5、設置開機啟動

chkconfig keepalived on

或者

systemctl enable keepalived.service  設置開機自動啟動

啟動keepalived服務：

systemctl start keepalived.service 啟動

 

6、安裝所有需要的依賴和環境后，將容器新增的內容重新提交

docker commit 5d112 centos_keepalived_nginx:v1

注：5d112為，上述安裝軟件所對應的容器id

 

6、啟動含有(keepalived+nginx)的容器

docker run --privileged  -tid --name  keepalived_master centos_keepalived_nginx:v1 /usr/sbin/init

 

 

進入keepalived_master容器：

docker exec -it keepalived_master bash

進入/usr/share/nginx/html，修改index.html文件

修改標題為:

Welcome to nginx Master!

7、啟動keepalived_salve容器

#啟動一個容器

docker run --privileged  -tid --name  keepalived_slave centos_keepalived_nginx:v1 /usr/sbin/init

#進入容器

docker exec -it keepalived_slave bash

 

8、修改keepalived_salve容器中nginx index.html文件

vim /usr/share/nginx/html/index.html

 

 

修改標題為:

Welcome to nginx Slave!

 

9、修改keepalived_salve容器中keepalived.conf文件 (master容器中，保持和鏡像中設置一樣即可，不需要更改)

! Configuration File for keepalived
global_defs {
notification_email {
[e-mail]
}
notification_email_from [e-mail]
smtp_server mail.example.com
smtp_connect_timeout 30
router_id LVS_DEVEL
}

vrrp_script chk_nginx {
script "/etc/keepalived/nginx_check.sh"
interval 2
weight -5
fall 3
rise 2
}


vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 2
priority 100
advert_int 2
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.17.0.210
}
track_script {
chk_nginx
}

}

其實，由配置中可以看出，主要是state和priority兩個參數的調整，其中master節點的priority值一定要比backup大才行！

 

原理說明：
1、 通過vrrp協議廣播，每個keepalived vrrp都去爭取master
2、 以virtual_router_id為組隊標識。  同為一個vip服務的keepalived的virtual_router_id要保持相同

3、 以priority 為權值，同一個virtual_router_id下那個priority大那個就是master,其它為backup

 

 

改完之后，重新加載

systemctl daemon-reload 
systemctl restart keepalived.service

 

10、驗證

查看兩個容器中keepalived服務狀態

systemctl status keepalived.service 

keepalived_master服務狀態效果：

keepalived_slave服務狀態效果圖：

 

可以看到，keepalived服務運行正常

 

啟動nginx: nginx

 

在master容器中 curl 172.17.0.210

 

在slave容器中 curl 172.17.0.210:

可以看現，此時master和slave容器兩邊通過虛擬vip : 172.17.0.210 訪問nginx數據，請求返回的數據都是master容器中nginx配置的數據： welcome to nginx master

 

繼續驗證，關掉master容器的keepalived服務：

 

 驗證得到的結果是當master容器中的keepalived服務關掉后，curl 172.17.0.210請求返回的數據來自slave，welcome to nginx slave

 

再繼續驗證，把關掉master容器的keepalived服務再開啟：

 

可以看到，當master容器中的keepalived服務開啟后，請求返回的數據會再次轉到master中。

 

到此，所有的驗證和預期的一致，也達到我們借助docker為基礎來實現了整套基於Nginx+Keepalived高可用的方案了。

 

三、Keepalived服務命令

• systemctl daemon-reload  重新加載
• systemctl enable keepalived.service  設置開機自動啟動
• systemctl disable keepalived.service 取消開機自動啟動
• systemctl start keepalived.service 啟動
• systemctl stop keepalived.service停止
• systemctl status keepalived.service  查看服務狀態