一、簡介
LVS是Linux Virtual Server的簡寫，意即Linux虛擬服務器，是一個虛擬的服務器集群系統。本項目在1998年5月由章文嵩博士成立，是中國國內最早出現的自由軟件項目之一。
目前有三種IP負載均衡技術（VS/NAT、VS/TUN和VS/DR）；十種調度算法（rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq）。
Keepalived在這里主要用作RealServer的健康狀態檢查以及LoadBalance主機和BackUP主機之間failover的實現。
LVS+Keepalived能實現的功能：利用LVS控制器主備模式避免單點故障以及自動刪除故障WEB服務器結點並當它恢復后再自動添加到群集中。
拓撲圖：
二、系統環境
系統平台：RHEL 5.4
LVS版本：ipvsadm-1.24
keepalived版本：keepalived-1.1.15
三、安裝
1、在兩台LVS Server上安裝lvs、keepalived軟件（LVS_MASTER & LVS_BACKUP）
lvs install -------------
[root@LVS-MASTER ~]# wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
[root@LVS-MASTER ~]# ln -s /usr/src/kernels/2.6.18-194.el5-i686//usr/src/linux/
[root@LVS-MASTER ~]# tar zxvf ipvsadm-1.24.tar.gz
[root@LVS-MASTER ~]# cd ipvsadm-1.24
[root@LVS-MASTER ipvsadm-1.24]# make && make install
Keepalived install -------------
[root@LVS-MASTER ~]# wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz
[root@LVS-MASTER ~]# tar zxvf keepalived-1.1.15.tar.gz
[root@LVS-MASTER ~]# cd keepalived-1.1.15
[root@LVS-MASTER keepalived-1.1.15]# ./configure && make && make install
######### 將keepalived做成啟動服務，方便管理##########
[root@LVS-MASTER ~]# cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/
[root@LVS-MASTER ~]# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
[root@LVS-MASTER ~]# mkdir /etc/keepalived/
[root@LVS-MASTER ~]# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
[root@LVS-MASTER ~]# cp /usr/local/sbin/keepalived /usr/sbin/
[root@LVS-MASTER ~]# service keepalived start | stop
2、keepalived的配置
################### MASTER ###################
! Configuration File for keepalived
global_defs {
notification_email {
root@linux.tang.chao #設置報警郵件地址，可以設置多個，每行一個。
mchina_tang@qq.com #需開啟本機的sendmail服務
}
notification_email_from keepalived@localhost #設置郵件的發送地址
smtp_server 127.0.0.1 #設置smtp server地址
smtp_connect_timeout 30 #設置連接smtp server的超時時間
router_id LVS_DEVEL #表示運行keepalived服務器的一個標識。發郵件時顯示在郵件主題的信息
}
vrrp_instance VI_1 {
state MASTER #指定keepalived的角色，MASTER表示此主機是主服務器，BACKUP表示此主機是備用服務器
interface eth0 #指定HA監測網絡的接口
virtual_router_id 51 #虛擬路由標識，這個標識是一個數字，同一個vrrp實例使用唯一的標識。即同一vrrp_instance下，MASTER和BACKUP必須是一致的
priority 100 #定義優先級，數字越大，優先級越高，在同一個vrrp_instance下，MASTER的優先級必須大於BACKUP的優先級
advert_int 1 #設定MASTER與BACKUP負載均衡器之間同步檢查的時間間隔，單位是秒
authentication { #設置驗證類型和密碼
auth_type PASS #設置驗證類型，主要有PASS和AH兩種
auth_pass 1111 #設置驗證密碼，在同一個vrrp_instance下，MASTER與BACKUP必須使用相同的密碼才能正常通信
}
virtual_ipaddress { #設置虛擬IP地址，可以設置多個虛擬IP地址，每行一個
10.0.0.148
}
}
virtual_server 10.0.0.148 80 { #設置虛擬服務器，需要指定虛擬IP地址和服務端口，IP與端口之間用空格隔開
delay_loop 6 #設置運行情況檢查時間，單位是秒
lb_algo rr #設置負載調度算法，這里設置為rr，即輪詢算法
lb_kind DR #設置LVS實現負載均衡的機制，有NAT、TUN、DR三個模式可選
persistence_timeout 50 #會話保持時間，單位是秒。這個選項對動態網頁是非常有用的，為集群系統中的session共享提供了一個很好的解決方案。
#有了這個會話保持功能，用戶的請求會被一直分發到某個服務節點，直到超過這個會話的保持時間。
#需要注意的是，這個會話保持時間是最大無響應超時時間，也就是說，用戶在操作動態頁面時，如果50秒內沒有執行任何操作，
#那么接下來的操作會被分發到另外的節點，但是如果用戶一直在操作動態頁面，則不受50秒的時間限制
protocol TCP #指定轉發協議類型，有TCP和UDP兩種
real_server 10.0.0.137 80 { #配置服務節點1，需要指定real server的真實IP地址和端口，IP與端口之間用空格隔開
weight 3 #配置服務節點的權值，權值大小用數字表示，數字越大，權值越高，設置權值大小可以為不同性能的服務器
#分配不同的負載，可以為性能高的服務器設置較高的權值，而為性能較低的服務器設置相對較低的權值，這樣才能合理地利用和分配系統資源
TCP_CHECK { #realserver的狀態檢測設置部分，單位是秒
connect_timeout 10 #表示3秒無響應超時
nb_get_retry 3 #表示重試次數
delay_before_retry 3 #表示重試間隔
connect_port 80
}
}
real_server 10.0.0.139 80 {
weight 3
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
#################### BACKUP ###################
! Configuration File for keepalived
global_defs {
notification_email {
root@linux.tang.chao
mchina_tang@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.148
}
}
virtual_server 10.0.0.148 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 10.0.0.137 80 {
weight 3
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 10.0.0.139 80 {
weight 3
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
3、在兩台Web Server上執行realserver.sh腳本，為lo:0綁定VIP地址10.0.0.148、抑制ARP廣播。
[root@web1 ~]# cat realserver.sh
#!/bin/bash #description: Config realserver VIP=10.0.0.148 /etc/rc.d/init.d/functions case "$1" in start) /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP /sbin/route add -host $VIP dev lo:0 echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce sysctl -p >/dev/null 2>&1 echo "RealServer Start OK" ;; stop) /sbin/ifconfig lo:0 down /sbin/route del $VIP >/dev/null 2>&1 echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce echo "RealServer Stoped" ;; *) echo "Usage: $0 {start|stop}" exit 1 esac exit 0
[root@web1 ~]# sh realserver.sh start
4、分別在LVS-MASTER、LVS-BACKUP上執行service keepalived start啟動keepalived就可實現負載均衡及高可用集群了；
[root@LVS-MASTER keepalived]# service keepalived start
[root@LVS-MASTER keepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.148:http rr persistent 50
-> 10.0.0.139:http Route 3 0 0
-> 10.0.0.137:http Route 3 0 0
[root@LVS-MASTER keepalived]#
查看日志/var/log/messages
[root@LVS-MASTER keepalived]# tail -n 30/var/log/messages
Apr 19 03:59:58 LVS-MASTER avahi-daemon[2845]: Withdrawing address record for10.0.0.148 on eth0.
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting Keepalived v1.2.1 (04/17,2012)
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting Healthcheck child process,pid=5960
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.136 added
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.159 added
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Registering Kernelnetlink reflector
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Registering Kernelnetlink command channel
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Configuration is using :12257 Bytes
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Using LinkWatch kernelnetlink reflector...
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Activating healtcheckerfor service [10.0.0.137:80]
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Activating healtcheckerfor service [10.0.0.139:80]
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.136 added
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.159 added
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering Kernel netlinkreflector
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering Kernel netlink commandchannel
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering gratutious ARP sharedchannel
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Opening file'/etc/keepalived/keepalived.conf'.
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Configuration is using : 36698Bytes
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Using LinkWatch kernel netlinkreflector...
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: VRRP sockpool: [ifindex(2),proto(112), fd(11,12)]
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting VRRP child process, pid=5961
Apr 19 04:00:20 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Transition toMASTER STATE
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTERSTATE
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) settingprotocol VIPs.
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
Apr 19 04:00:21 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.136 added
Apr 19 04:00:21 LVS-MASTER avahi-daemon[2845]: Registering new address recordfor 10.0.0.148 on eth0.
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.136 added
Apr 19 04:00:26 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
[root@LVS-MASTER keepalived]#
四、測試
接下來做高可用性測試&故障切換測試......
####高可用性測試####
模擬故障，將LVS-MASTER上的keepalived服務停掉，然后觀察LVS-BACKUP上的日志，信息如下
[root@LVS-BACKUP keepalived]# tail -f /var/log/messages
Apr 19 03:33:04 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1)Transition to MASTER STATE
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTERSTATE
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) settingprotocol VIPs.
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: Netlink reflector reports IP10.0.0.157 added
Apr 19 03:33:05 LVS-BACKUP Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.157 added
Apr 19 03:33:05 LVS-BACKUP avahi-daemon[2825]: Registering new address recordfor 10.0.0.148 on eth0.
Apr 19 03:33:10 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
從日志中可知，主機出現故障后，備機立刻檢測到，此時備機變為MASTER角色，並且接管了主機的虛擬IP資源，最后將虛擬IP綁定在etho設備上。
將LVS-MASTER 上的keepalived服務開啟后，LVS-BACKUP的日志狀態。
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advert
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1)Entering BACKUP STATE
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: Netlink reflector reports IP 10.0.0.157 removed
Apr 19 03:34:02 LVS-BACKUP Keepalived_healthcheckers: Netlink reflector reports IP 10.0.0.157 removed
Apr 19 03:34:02 LVS-BACKUP avahi-daemon[2825]: Withdrawing address record for 10.0.0.148 on eth0.
從日志可知，備機在檢測到主機重新恢復正常后，釋放了虛擬IP資源重新成為BACKUP角色
####故障切換測試####
故障切換是測試當某個節點出現故障后，Keepalived監制模塊是否能及時發現然后屏蔽故障節點，同時將服務器轉移到正常節點來執行。
將web2節點服務停掉，假設這個節點出現故障，然后主、備機日志信息如下
Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers:TCP connection to [10.0.0.139:80] failed!!!
Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers:Removing service [10.0.0.139:80] from VS [10.0.0.148:80]
Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers: Remote SMTP server [127.0.0.1:25] connected.
[root@LVS-MASTER keepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.148:http rr persistent 50
-> 10.0.0.137:http Route 3 0 0
[root@LVS-MASTER keepalived]#
從以上可以看出，Keepalived監控模塊檢測到10.0.0.139這台主機出現故障后，將些web2從集群系統中剔除掉了。 此時訪問http://10.0.0.148只能看到web1了)
重新啟動web2節點的服務，日志信息如下：
Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers:TCP connection to [10.0.0.139:80] success.
Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers:Adding service [10.0.0.139:80] to VS [10.0.0.148:80]
Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers: Remote SMTP server [127.0.0.1:25] connected.
[root@LVS-MASTERkeepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.148:http rr persistent 50
-> 10.0.0.139:http Route 3 0 0
-> 10.0.0.137:http Route 3 0 0
[root@LVS-MASTER keepalived]#
Keepalived監控模塊檢測到10.0.0.139這台主機恢復正常后，又將此節點加入集群系統中，再次訪問就可以訪問到web2頁面了）
CentOS 6.3下部署LVS(NAT)+keepalived實現高性能高可用負載均衡
http://www.cnblogs.com/mchina/archive/2012/08/27/2644391.html