詳解keepalived配置和使用

 

一、keepalived簡介：

keepalived是一個類似於layer3, 4 & 5交換機制的軟件，也就是我們平時說的第3層、第4層和第5層交換。Keepalived的作用是檢測web服務器的狀態，如果有一台web服務器死機，或工作出現故障，Keepalived將檢測到，並將有故障的web服務器從系統中剔除，當web服務器工作正常后Keepalived自動將web服務器加入到服務器群中，這些工作全部自動完成，不需要人工干涉，需要人工做的只是修復故障的web服務器。

 

工作原理

Layer3,4&5工作在IP/TCP協議棧的IP層，TCP層，及應用層,原理分別如下：

Layer3：Keepalived使用Layer3的方式工作式時，Keepalived會定期向服務器群中的服務器發送一個ICMP的數據包（既我們平時用的Ping程序）,如果發現某台服務的IP地址沒有激活，Keepalived便報告這台服務器失效，並將它從服務器群中剔除，這種情況的典型例子是某台服務器被非法關機。Layer3的方式是以服務器的IP地址是否有效作為服務器工作正常與否的標准。

Layer4:如果您理解了Layer3的方式，Layer4就容易了。Layer4主要以TCP端口的狀態來決定服務器工作正常與否。如web server的服務端口一般是80，如果Keepalived檢測到80端口沒有啟動，則Keepalived將把這台服務器從服務器群中剔除。

Layer5：Layer5就是工作在具體的應用層了，比Layer3,Layer4要復雜一點，在網絡上占用的帶寬也要大一些。Keepalived將根據用戶的設定檢查服務器程序的運行是否正常，如果與用戶的設定不相符，則Keepalived將把服務器從服務器群中剔除。

 

二、實驗步驟：

1.創建管理節點在node1上，建立雙機互信node1和node2，然后同步時間，安裝keepalived

[root@node1~]# ansible all -m yum -a 'name=keepalived state=present' [root@node1keepalived]# rpm -qc keepalived /etc/keepalived/keepalived.conf//生成的主配置文件 /etc/sysconfig/keepalived

 

 

2.在node1上配置文件需要做一下修改

global_defs{ notification_email { root@localhost //收郵件人，可以定義多個 } notification_email_from kaadmin@localhost //發郵件人可以偽裝 smtp_server 127.0.0.1 //發送郵件的服務器地址 smtp_connect_timeout 30 //連接超時時間 router_id LVS_DEVEL } vrrp_instanceVI_1 { //每一個vrrp_instance就是定義一個虛擬路由器的 state MASTER //由初始狀態狀態轉換為master狀態 interface eth0 virtual_router_id 51 //虛擬路由的id號，一般不能大於255的 priority 100 //初始化優先級 advert_int 1 //初始化通告 authentication { //認證機制 auth_type PASS auth_pass 1111 //密碼 } virtual_ipaddress { //虛擬地址vip 172.16.2.8 } }

 

3.把配置文件復制到node2上一份，並修改初始狀態和優先級

[root@node1keepalived]# scp keepalived.conf node2:/etc/keepalived/ [root@node2~]# cd /etc/keepalived/ [root@node2keepalived]# ls keepalived.conf [root@node2keepalived]# vim keepalived.conf vrrp_instanceVI_1 { state BACKUP //初始化狀態 interface eth0 virtual_router_id 51 priority 99 //優先級，一定要比master的優先級要低 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.16.2.8 } }

 

在node1上開始啟動服務[root@node1 ~]# servicekeepalived start

然后檢查ip地址

[root@node1~]# ip addr show 1:lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast stateUP qlen 1000 link/ether 00:0c:29:4e:22:fb brdff:ff:ff:ff:ff:ff inet 172.16.2.1/16 brd 172.16.255.255 scopeglobal eth0 inet 172.16.2.8/32 scopeglobal eth0 inet 172.16.10.8/16 brd 172.16.255.255 scopeglobal secondary eth0:0 inet6 fe80::20c:29ff:fe4e:22fb/64 scopelink valid_lft forever preferred_lft forever 3:pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN link/ether 2e:79:b3:b2:3e:31 brdff:ff:ff:ff:ff:ff

 

4.現在把node1的keepalived停掉

[root@node1keepalived]# service keepalived stop

Stoppingkeepalived: [ OK ]

驗證node2是否把virtual_ipaddress拿走

[root@node2~]# ip addr show 1:lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast stateUP qlen 1000 link/ether 00:0c:29:74:c7:7b brdff:ff:ff:ff:ff:ff inet 172.16.2.16/16 brd172.16.255.255 scope global eth0 inet 172.16.2.8/32 scopeglobal eth0 inet6 fe80::20c:29ff:fe74:c77b/64 scopelink valid_lft forever preferred_lft forever 3:pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN link/ether0a:b1:ef:7b:93:18 brd ff:ff:ff:ff:ff:ff

 

驗證成功

 

可以在配置文件中手動通過vrrp_script定義一個外圍的檢測機制，並在vrrp_instance中通過定義track_script來追蹤腳本執行過程，實現節點轉移

實驗測試在/etc/keepalived/keepalived.conf中做一下修改

global_defs{ notification_email { root@localhost } notification_email_from kaadmin@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_maintainace { //檢測機制的腳本名稱為chk_maintainace script "[[ -e/etc/keepalived/down ]] && exit 1 || exit 0" //可以是個腳本路徑，也可以是腳本命令 interval 1 //每隔1秒中檢測一次 weight -2 //優先級減2 } vrrp_instanceVI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.16.2.8 } track_script { //調用外圍腳本，追蹤外圍腳本執行過程 chk_maintainace } } [root@node1 keepalived]# touch down //在node1上創建down文件 [root@node1 keepalived]# ls down keepalived.conf keepalived.conf.bak

 

在node2上做同樣的操作，但不創建down文件，之后一起重啟服務

[root@node1 keepalived]# ansible all -m shell -a 'service keepalivedrestart' node2.magedu.com| success | rc=0 >> Stoppingkeepalived: [FAILED] Startingkeepalived: [ OK ] node1.magedu.com| success | rc=0 >> Stoppingkeepalived: [ OK ] Startingkeepalived: [ OK ]

 

進行檢測

[root@node2keepalived]# ip addr show 1:lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast stateUP qlen 1000 link/ether 00:0c:29:74:c7:7b brdff:ff:ff:ff:ff:ff inet 172.16.2.16/16 brd172.16.255.255 scope global eth0 inet 172.16.2.8/32 scopeglobal eth0 inet6 fe80::20c:29ff:fe74:c77b/64 scopelink valid_lft forever preferred_lft forever 3:pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN link/ether0a:b1:ef:7b:93:18 brd ff:ff:ff:ff:ff:ff

 

此時將node1中/etc/keepalived/下的down刪除，進行查看

[root@node1keepalived]# ls down keepalived.conf keepalived.conf.bak [root@node1keepalived]# rm down rm:remove regular empty file `down'? y [root@node1keepalived]# ls keepalived.conf keepalived.conf.bak [root@node1 keepalived]# ip addr show 1:lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast stateUP qlen 1000 link/ether 00:0c:29:4e:22:fb brdff:ff:ff:ff:ff:ff inet 172.16.2.1/16 brd 172.16.255.255 scopeglobal eth0 inet 172.16.2.8/32 scopeglobal eth0 inet 172.16.10.8/16 brd 172.16.255.255scope global secondary eth0:0 inet6 fe80::20c:29ff:fe4e:22fb/64 scopelink valid_lft forever preferred_lft forever 3:pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN link/ether0a:bd:4f:a9:ed:67 brd ff:ff:ff:ff:ff:ff

 

 

驗證成功

 

三、詳細介紹以下四個功能實現操作

1.如何在狀態轉換時進行通知？

2.如何配置Ipvs?

3.如何對某特定服務做高可用

4.如何實現基於多虛擬路由的master/master模型？

 

1.要在狀態轉換是進行通知，需要定義通知腳本可以在

vrrp_sync_group{

}中定義，也可以在

vrrp_instance{

｝中定義

通過man keepalived命令可以查看通知腳本定義的兩種方法

第一種

# to MASTER transition

notify_master /path/to_master.sh

# to BACKUP transition

notify_backup /path/to_backup.sh

# FAULT transition

notify_fault "/path/fault.sh VG_1"

第二種

#arguments

# $1 ="GROUP"|"INSTANCE"

# $2 = name of group or instance

# $3 = target state of transition

# ("MASTER"|"BACKUP"|"FAULT")

notify /path/notify.sh

 

例如：

轉換為MASTER的狀態通知

#!/bin/bash # vip=172.16.2.8 contact='root@localhost' thisip=`ifconfigeth0 | awk '/inet addr:/{print $2}' | awk -F: '{print $2}'` notify(){ mailbody="vrrp transaction, $vipfloated to $thisip." subject="$thisip is to be $vipmaster" echo $mailbody | mail -s $subject $contact } notify

 

其他狀態轉換類似

下面用一個腳本notify.sh實現狀態轉換通知的簡單示例：

#!/bin/bash #Author: MageEdu <[e-mail]> #description: An example of notify script # vip=172.16.2.8 contact='root@localhost' notify(){ mailsubject="`hostname` to be $1: $vipfloating" mailbody="`date '+%F %H:%M:%S'`: vrrptransition, `hostname` changed to be $1" echo $mailbody | mail -s"$mailsubject" $contact } case"$1" in master) notify master exit 0 ;; backup) notify backup exit 0 ;; fault) notify fault exit 0 ;; *) echo 'Usage: `basename $0`{master|backup|fault}' exit 1 ;; esac

 

進行測試

[root@node1keepalived]# ./notify.sh backup [root@node1keepalived]# mail HeirloomMail version 12.4 7/29/08. Type ? forhelp. "/var/spool/mail/root":6 messages 1 new 6 unread U [e-mail].c Sat Aug 1709:34 17/644 "*** SECURITY" U 2Cron Daemon Tue Aug 2700:01 22/747 "Cron <root@s" U 3Cron Daemon Fri Aug 3000:01 22/747 "Cron <root@s" U 4Mail Delivery System Fri Aug 3017:42 91/2751 "Undelivered " U 5Cron Daemon Tue Sep 3 00:01 22/747 "Cron<root@s" >N 6 root Thu Sep 26 21:19 18/700 "node1.magedu" &6 Message 6: [e-mail] Thu Sep 2621:19:32 2013 Return-Path:<[e-mail]> X-Original-To:root@localhost Delivered-To:[e-mail] Date:Thu, 26 Sep 2013 21:19:32 +0800 To:[e-mail] Subject:node1.magedu.com to be backup: 172.16.2.8 floating User-Agent:Heirloom mailx 12.4 7/29/08 Content-Type:text/plain; charset=us-ascii From:[e-mail] (root) Status:R 2013-09-26 21:19:32: vrrp transition, node1.magedu.com changed to bebackup &quit Held6 messages in /var/spool/mail/root Youhave mail in /var/spool/mail/root

 

通過傳參數master|backup|fault驗證都可以成功

在配置文件keepalived.conf中進行腳本調用

vrrp_instanceVI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.16.2.8 } track_script { chk_maintainace } notify_master "/etc/keepalived/notify.shmaster" notify_backup"/etc/keepalived/notify.sh backup" notify_fault"/etc/keepalived/notify.sh fault" }

 

為node2提供同樣的配置然后進行測試

[root@node1keepalived]# ls

down keepalived.conf keepalived.conf.bak notify.sh

[root@node1keepalived]# rm -f down

[root@node1keepalived]# mail

>N18 root Thu Sep 2621:57 18/700 "node1.magedu.comto be master: 172.16.2.8 floating"截取了一條

驗證都可以成功

 

2、如何配置ipvs

virtual_server172.16.2.8 80{ delay_loop 6 lb_algo rr lb_kind NAT nat_mask 255.255.0.0 persistence_timeout 0 protocol TCP # real_server 172.16.2.1 80 { weight 1 HTTP_GET { url { path / state_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 172.16.2.16 80 { weight 1 HTTP_GET { url { path / state_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } }

 

在node2上做同樣的修改，啟動httpd服務,keepalived能自動生成規則，然后查看ipvsadm規則

[root@node1keepalived]# ipvsadm -L -n IPVirtual Server version 1.2.1 (size=4096) ProtLocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.16.2.8:80 rr -> 172.16.2.1:80 Local 1 0 0 -> 172.16.2.16:80 Masq 1 0 0

 

3、如何對某特定服務做高可用？以nginx為例進行講解

在兩個節點上安裝nginx

[root@node1~]# ansible all -m yum -a 'name=nginx state=present'

啟動nginx服務，啟動之前注意要停止httpd服務

[root@node1~]# ansible all -m shell -a 'service nginx start' node2.magedu.com| success | rc=0 >> Startingnginx: [ OK ] node1.magedu.com| success | rc=0 >> Startingnginx: [ OK ]

 

對node1和node2中/etc/keepalived/下的notify.sh腳本進行修改

#!/bin/bash #Author: MageEdu <[e-mail]> #description: An example of notify script # vip=172.16.2.8 contact='root@localhost' notify(){ mailsubject="`hostname` to be $1: $vipfloating" mailbody="`date '+%F %H:%M:%S'`: vrrptransition, `hostname` changed to be $1" echo $mailbody | mail -s"$mailsubject" $contact } case"$1" in master) notify master /etc/rc.d/init.d/nginx start exit 0 ;; backup) notify backup /etc/rc.d/init.d/nginx stop exit 0 ;; fault) notify fault /etc/rc.d/init.d/nginx stop exit 0 ;; *) echo 'Usage: `basename $0`{master|backup|fault}' exit 1 ;; esac

 

然后啟動keepalived服務，可以看到在node1上80端口開始啟用

[root@node1keepalived]# ss -tanl | grep :80

LISTEN 0 128 *:80 *:*

然后在/etc/keepalive/下創建down文件，看nginx服務是否可以轉移到node2上

[root@node1keepalived]# ls keepalived.conf keepalived.conf.bak notify.sh [root@node1keepalived]# touch down [root@node1keepalived]# ss -tanl | grep :80 [root@node1keepalived]# 在node2上進行查看 [root@node2keepalived]# ss -tanl | grep :80 LISTEN 0 128 *:80 *:*

 

驗證成功，說明實現了nginx的高可用服務

總結：要對某特定服務做高可用有兩個要點

一是：要提供監控服務腳本

二是：在vrrp實例中追蹤服務

修改配置文件keepalived.conf

vrrp_script chk_nginx { script "killall -0 nginx" interval 1 weight -2 } vrrp_instanceVI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.16.2.8 } track_script { chk_maintainace chk_nginx }

 

在node2上做同樣的修改

測試：

[root@node2keepalived]# killall nginx

Youhave new mail in /var/spool/mail/root

[root@node2keepalived]# ss -tanl | grep :80

[root@node2keepalived]#

在node1上

[root@node1keepalived]# ss -tanl | grep :80

LISTEN 0 128 *:80 *:*

驗證成功

4、如何實現基於多虛擬路由的master/master模型？

要實現雙主模型需要定義兩個vrrp_instance，在node1的配置文件中要一下修改：

vrrp_instanceVI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.16.2.8 } track_script { chk_maintainace chk_nginx } notify_master"/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.shbackup" notify_fault "/etc/keepalived/notify.shfault" } vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 55 priority 99 advert_int 1 authentication { auth_type PASS auth_pass 2111 } virtual_ipaddress { 172.16.2.18 } track_script { chk_maintainace chk_nginx } notify_master"/etc/keepalived/notify.sh master" notify_backup"/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.shfault" }

 

在node2上做同樣的修改，重啟keepalived，進行測試

[root@node1keepalived]# service nginx status nginx(pid 28688) is running... [root@node1keepalived]# ip addr show 1:lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast stateUP qlen 1000 link/ether 00:0c:29:4e:22:fb brdff:ff:ff:ff:ff:ff inet 172.16.2.1/16 brd 172.16.255.255 scopeglobal eth0 inet 172.16.2.8/32 scopeglobal eth0 inet 172.16.10.8/16 brd 172.16.255.255scope global secondary eth0:0 inet6 fe80::20c:29ff:fe4e:22fb/64 scopelink valid_lft forever preferred_lft forever 3:pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN link/ether 6a:7a:4f:e0:c1:8a brdff:ff:ff:ff:ff:ff Youhave new mail in /var/spool/mail/root

 

在node2上

[root@node2keepalived]# service nginx start Startingnginx: [ OK ] [root@node2keepalived]# ip addr show 1:lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast stateUP qlen 1000 link/ether 00:0c:29:74:c7:7b brd ff:ff:ff:ff:ff:ff inet 172.16.2.16/16 brd172.16.255.255 scope global eth0 inet 172.16.2.18/32 scopeglobal eth0 inet6 fe80::20c:29ff:fe74:c77b/64 scopelink valid_lft forever preferred_lft forever 3:pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN link/ether3a:4e:e8:4c:57:04 brd ff:ff:ff:ff:ff:ff 讓node2的keepalived停掉，查看地址是否發生轉移 [root@node1keepalived]# ip addr show 1:lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast stateUP qlen 1000 link/ether 00:0c:29:4e:22:fb brd ff:ff:ff:ff:ff:ff inet 172.16.2.1/16 brd172.16.255.255 scope global eth0 inet 172.16.2.8/32 scopeglobal eth0 inet 172.16.2.18/32 scopeglobal eth0 inet 172.16.10.8/16 brd 172.16.255.255scope global secondary eth0:0 inet6 fe80::20c:29ff:fe4e:22fb/64 scopelink valid_lft forever preferred_lft forever 3:pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN link/ether 6a:7a:4f:e0:c1:8a brdff:ff:ff:ff:ff:ff Youhave new mail in /var/spool/mail/root

 

驗證成功