keepalived綁定單播地址、非搶占模式及LVS的TCP模式的高可用

背景：keepalived默認是組播地址進行播放，且默認地址是224.0.0.18，如果配置多個keepalived主機，會導致虛擬IP地址存在沖突問題，這種問題怎么解決呢？

解決辦法：就是將keepalived主機的多播地址修改為單播地址，綁定固定IP地址，避免在多播模式下，通過VRRP進行廣播地址，造成IP地址地址沖突。

vrrp_strict   #嚴格遵守VRRP協議,不允許狀況，在配置單播IP地址時，此行需要刪除或者注釋掉即可。

1、沒有VIP地址

2、單播鄰居

3、在VRRP版本2中有IPv6地址

實驗一：實現keepalived單播地址配置

1、在主節點配置keepalived文件

[root@centos_17~]#vim /etc/keepalived/keepalived.conf
global_defs {
   notification_email {
     [e-mail]
   }
   notification_email_from [e-mail]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka1
   vrrp_skip_check_adv_addr
   vrrp_iptables
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 50
    priority 100
    unicast_src_ip 192.168.37.17  配置源地址的IP地址
    unicast_peer {
       192.168.37.7  配置從節點的目標IP地址
    }
    advert_int 2
    authentication {
        auth_type pass
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100 dev ens33 label ens33:1
    }
}

 配置完成之后重啟keepalived：

[root@centos_17~]#systemctl restart keepalived

2、在從節點配置keepalived文件　　

[root@centos7~]#vim /etc/keepalived/keepalived.conf
global_defs {
   notification_email {
     [e-mail]
   }
   notification_email_from [e-mail]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka2
   vrrp_skip_check_adv_addr
   vrrp_iptables
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 50
    priority 80
    unicast_src_ip 192.168.37.7
    unicast_peer {
       192.168.37.17
        }
    advert_int 2
    authentication {
        auth_type pass
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100 dev ens33 label ens33:1
    }
}

　配置完成之后重啟keepalived：　

[root@centos_17~]#systemctl restart keepalived

 3、驗證鎖單播IP地址效果。

此時由於綁定了雙方keepalived主機的IP地址，就只會對雙方進行廣播，避免多個keepalived的虛擬IP地址進行沖突。

 

 實驗二：實現非搶占模式漂移VIP

 原理：關閉VIP搶占模式，需要VIP state都為BACKUP，此時哪個keepalived的優先級大，優先占用哪個keepalived服務器，當此占用的服務器宕機后，另一個BACKUP主機才會進行占用，就算優先級高的恢復了，也不能進行搶占，除非優先級低的服務器宕機后，才會繼續占用到高優先級的keepalived服務器上。

1、在優先級高的keepalived主機進行配置

 vim   /etc/keepalived/keepalived.conf

global_defs {
   notification_email {
     [e-mail]
   }
   notification_email_from [e-mail]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka1
   vrrp_skip_check_adv_addr
   vrrp_iptables
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VIP_1 {
    state BACKUP  角色必須是BACKUP
    interface ens33
    virtual_router_id 50
    priority 100
    unicast_src_ip 192.168.37.17
    unicast_peer {
       192.168.37.7
    }
    advert_int 2
    nopreempt  設置為非搶占模式
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100 dev ens33 label ens33:1
    }
}

 

 2、在優先級低的keepalived主機配置

 vim   /etc/keepalived/keepalived.conf

global_defs {
   notification_email {
     [e-mail]
   }
   notification_email_from [e-mail]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka2
   vrrp_skip_check_adv_addr
   vrrp_iptables
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VIP_1 {
    state BACKUP   角色必須是BACKUP
    interface ens33
    virtual_router_id 60
    priority 80
    unicast_src_ip 192.168.37.7
    unicast_peer {
       192.168.37.17
    }
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100 dev ens33 label ens33:1
    }
}

 測試效果：

 此時手動將keepalived主機優先級高的進行宕機，VIP就會漂移到優先級低的主機上。

[root@centos_17~]#systemctl stop keepalived

 此時查看優先級低的主機IP地址情況：

 

 此時就算高優先級的主機恢復，也無法搶占VIP地址，此時需要宕機優先級低的主機才會占用VIP地址。

  啟動優先級高的主機：systemctl start keepalived

  停掉優先級低的主機：systemctl stop keepalived

 

 實戰三：實現兩個以上的Keepalived主機

 背景：當公司需求量較大時，兩個keepalived已經不能滿足公司需求，此時需要配置兩台以上的keepalived，應該怎么配置？

 實現方法如下：

1、在A主機配置keepalived

  vim   /etc/keepalived/keepalived.conf

global_defs {
   notification_email {
     [e-mail]
   }
   notification_email_from [e-mail]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka1
   vrrp_skip_check_adv_addr
   vrrp_iptables
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VIP_1 {
    state MASTER  主節點服務器
    interface ens33
    virtual_router_id 60 
    priority 100   優先級為100
    unicast_src_ip 192.168.37.7  綁定單播地址，防止IP地址與其他keepalived地址沖突
    unicast_peer {
       192.168.37.17   目標keepalived主機IP地址
       192.168.37.27   目標keepalived主機IP地址                                                                                                                          
    }
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100 dev ens33 label ens33:1  添加VIP地址，也可以多添加幾個地址
    }
}

 2、在B主機配置keepalived　　

global_defs {
   notification_email {
     [e-mail]
   }
   notification_email_from [e-mail]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka1
   vrrp_skip_check_adv_addr
   vrrp_iptables
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VIP_1 {
    state BACKUP  從節點服務器
    interface ens33
    virtual_router_id 50
    priority 80 優先級為80
    unicast_src_ip 192.168.37.17   綁定單播地址，源keepalived的IP地址
    unicast_peer {
       192.168.37.7   兩個目標的keepalived的IP地址
       192.168.37.27
    }
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100 dev ens33 label ens33:1
    }
}

3、在C主機配置keepalived　　

global_defs {
   notification_email {
     [e-mail]
   }
   notification_email_from [e-mail]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka1
   vrrp_skip_check_adv_addr
   vrrp_iptables
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VIP_1 {
    state BACKUP   從節點服務器
    interface ens33
    virtual_router_id 50
    priority 60  優先級為60，要比前兩個的主機優先級都要低
    unicast_src_ip 192.168.37.27  綁定單播地址，源地址
    unicast_peer {
       192.168.37.7  兩個keepalived的目標地址
       192.168.37.17
    }   
    advert_int 2
    authentication {                                                                                                                             
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100 dev ens33 label ens33:1
    }
}

 測試效果：　　

將A主機的keepalived宕機，可以觀察此時的B主機成為MASTER主機，VIP地址就會漂移到B主機上。　　

 

 當B主機的keepalived也宕機之后，此時的VIP就會漂移到C主機從節點的keepalived服務器上

 

  同理：當優先級高的keepalived服務器恢復后，VIP地址就又會漂移回去。

實戰四：實現TCP模式keepalived及LVS-DR模式高可用（此用法多於HTTP用法）

 參數含義：

delay_loop<INT>：檢查后端服務器的時間間隔
lb_algorr|wrr|lc|wlc|lblc|sh|dh：定義調度方法
lb_kindNAT|DR|TUN：集群的類型
persistence_timeout<INT>：持久連接時長
protocol TCP|UDP|SCTP：指定服務協議
sorry_server<IPADDR> <PORT>：所有RS故障時，備用服務器地址
real_server<IPADDR> <PORT>
{
weight <INT> RS權重
notify_up<STRING>|<QUOTED-STRING> RS上線通知腳本
notify_down<STRING>|<QUOTED-STRING> RS下線通知腳本
HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHEC K { ... }：定義當前主機的健康狀態檢測方法
}

 HTTP監測含義

HTTP_GET|SSL_GET：應用層檢測
HTTP_GET|SSL_GET {
url{
path <URL_PATH>：定義要監控的URL
status_code<INT>：判斷上述檢測機制為健康狀態的響應碼
}
connect_timeout<INTEGER>：連接請求的超時時長
nb_get_retry<INT>：重試次數
delay_before_retry<INT>：重試之前的延遲時長
connect_ip<IP ADDRESS>：向當前RS哪個IP地址發起健康狀態檢測請求
connect_port<PORT>：向當前RS的哪個PORT發起健康狀態檢測請求
bindto<IP ADDRESS>：發出健康狀態檢測請求時使用的源地址
bind_port<PORT>：發出健康狀態檢測請求時使用的源端口
}

TCP監測

 傳輸層檢測TCP_CHECK

TCP_CHECK {
connect_ip<IP ADDRESS>：向當前RS的哪個IP地址發起健康狀態檢測請求
connect_port<PORT>：向當前RS的哪個PORT發起健康狀態檢測請求
bindto<IP ADDRESS>：發出健康狀態檢測請求時使用的源地址
bind_port<PORT>：發出健康狀態檢測請求時使用的源端口
connect_timeout<INTEGER>：連接請求的超時時長
}

1、在A主機修改keepalived配置文件

vim  /etc/keepalived/keepalived.conf

global_defs {
   notification_email {
     [e-mail]
   }
   notification_email_from [e-mail]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka2
   vrrp_skip_check_adv_addr
   vrrp_iptables
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VIP_1 {
    state MASTER
    interface ens33
    virtual_router_id 50
    priority 100
    unicast_src_ip 192.168.37.7
    unicast_peer {
       192.168.37.17
    }
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100 dev ens33 label ens33:1
    }
}

include  /etc/keepalived/conf/*.conf   定義一個單獨管理的目錄，定義keepalived配置文件

 在A主機定義LVS-DR模式配置文件

如果后期keepalived的配置文件修改過大，我們可以調用include，新建一個目錄，並在此目錄下進行存放配置文件。

[root@centos7keepalived]#mkdir conf
[root@centos7keepalived]#vim conf/tcp.conf 
virtual_server 192.168.37.100 80 {     VIP地址                                                                                                          
        delay_loop 6
        lb_algo wrr  權重輪詢
        lb_kind DR   DR模式
        protocol TCP
        sorry_server 192.168.37.47  80  配置后端sorry服務器，當兩個keepalived主機都宕機之后，就在此主機進行顯示信息。

   real_server 192.168.37.27 80 {  后端RS1服務器IP地址
        weight 1
        TCP_CHECK {
        connect_timeout 5
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
   }

   real_server 192.168.37.37 80 {  后端RS2服務器
        weight 1
        TCP_CHECK {
        connect_timeout 5
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }
}

2、在B主機修改keepalived配置文件　

　vim  /etc/keepalived/keepalived.conf

global_defs {
   notification_email {
     [e-mail]
   }
   notification_email_from [e-mail]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka1
   vrrp_skip_check_adv_addr
   vrrp_iptables
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VIP_1 {
    state BACKUP
    interface ens33
    virtual_router_id 50
    priority 80
    unicast_src_ip 192.168.37.17
    unicast_peer {
       192.168.37.7
    }
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.37.100 dev ens33 label ens33:1
    }
}

include  /etc/keepalived/conf/*.conf

 在B主機定義LVS-DR模式的配置文件　

 新建conf目錄，並在此目錄下新建一個配置文件

[root@centos_17keepalived]#mkdir conf
[root@centos_17keepalived]#vim conf/tcp.conf 
virtual_server 192.168.37.100 80 {
        delay_loop 6
        lb_algo wrr
        lb_kind DR
        protocol TCP
        sorry_server 192.168.37.47  80  定義sorry server的后端主機，當兩個keepalived主機宕機后，就會提示此信息。

   real_server 192.168.37.27 80 {
        weight 1
        TCP_CHECK {
        connect_timeout 5
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
   }

   real_server 192.168.37.37 80 {
        weight 1
        TCP_CHECK {
        connect_timeout 5
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }
}

 3、后端服務器RS1上修改配置　　

  在RS1綁定VIP地址及lo回環網卡

[root@centos27~]#vim lvs_dr_rs.sh 
#!/bin/bash
#Author:wangxiaochun
#Date:2017-08-13                                                                                                                                 
vip=192.168.37.100   綁定VIP地址
mask='255.255.255.255'
dev=lo:1  綁定在lo回環網卡上
rpm -q httpd &> /dev/null || yum -y install httpd &>/dev/null
service httpd start &> /dev/null && echo "The httpd Server is Ready!"
echo "<h1>`hostname`</h1>" > /var/www/html/index.html

case $1 in
start)
    echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
    ifconfig $dev $vip netmask $mask #broadcast $vip up
    #route add -host $vip dev $dev
    echo "The RS Server is Ready!"
    ;;
stop)
    ifconfig $dev down
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
    echo "The RS Server is Canceled!"
    ;;
*)
    echo "Usage: $(basename $0) start|stop"
    exit 1
    ;;
esac

執行腳本：

[root@centos27~]#bash lvs_dr_rs.sh  start

 4、在RS2后端服務器上修改配置　　

 修改RS2的配置腳本，綁定VIP地址及lo回環網卡

#!/bin/bash
#Author:wangxiaochun
#Date:2017-08-13
vip=192.168.37.100                                                                                                                               
mask='255.255.255.255'
dev=lo:1
rpm -q httpd &> /dev/null || yum -y install httpd &>/dev/null
service httpd start &> /dev/null && echo "The httpd Server is Ready!"
echo "<h1>`hostname`</h1>" > /var/www/html/index.html

case $1 in
start)
    echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
    ifconfig $dev $vip netmask $mask #broadcast $vip up
    #route add -host $vip dev $dev
    echo "The RS Server is Ready!"
    ;;
stop)
    ifconfig $dev down
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
    echo "The RS Server is Canceled!"
    ;;
*)
    echo "Usage: $(basename $0) start|stop"
    exit 1
    ;;
esac

 執行以上腳本

[root@centos37~]#bash lvs_dr_rs.sh  start

 4、在sorry后端服務器上修改配置　　

 修改sorry后端服務器配置腳本，綁定VIP地址及lo回環網卡

#!/bin/bash
#Author:wangxiaochun
#Date:2017-08-13
vip=192.168.37.100                                                                                                                               
mask='255.255.255.255'
dev=lo:1
rpm -q httpd &> /dev/null || yum -y install httpd &>/dev/null
service httpd start &> /dev/null && echo "The httpd Server is Ready!"
echo "<h1>`hostname`</h1>" > /var/www/html/index.html

case $1 in
start)
    echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
    ifconfig $dev $vip netmask $mask #broadcast $vip up
    #route add -host $vip dev $dev
    echo "The RS Server is Ready!"
    ;;
stop)
    ifconfig $dev down
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
    echo "The RS Server is Canceled!"
    ;;
*)
    echo "Usage: $(basename $0) start|stop"
    exit 1
    ;;
esac

 執行以上腳本

[root@centos47~]#bash lvs_dr_rs.sh  start

5、測試效果：　

將RS1和RS2及sorry后端服務器添加測試頁面，並啟動httpd服務：systemctl  start httpd

[root@cenots277~]#yum install httpd
[root@cenots27~]#cd /var/www/html
[root@cenots27html]#cat index.html 
<h1>cenots27</h1>
[root@cenots37html]#cat index.html 
<h1>cenots37</h1>
[root@cenots37html]#cat index.html 
sorry server !!

 客戶端訪問LVS的VIP地址，此時LVS將調度到后端服務器，進行輪詢訪問。

 

 當后端RS1和RS2后端服務器宕機后，此時就會顯示sorry server信息。