Keepalived+nginx實現高可用負載均衡

圖1基本邏輯圖

                                                                                                     

 

圖2 為IP地址分配。

 

主要用途	IP
Haproxy+ningx_master	192.168.236.143
Haproxy+nginx_backup	192.168.236.192
Webser1	192.168.236.150
Webser2	192.168.236.151

 

一、配置Nginx

一、安裝如下環境

yum -y install make gcc gcc-c++ gcc-g77 flex bison file libtool libtool-libs autoconf kernel-devel libjpeg libjpeg-devel libpng libpng-devel libpng10 libpng10-devel gd gd-devel freetype freetype-devel libxml2 libxml2-devel zlib zlib-devel glib2 glib2-devel bzip2 bzip2-devel libevent libevent-devel ncurses ncurses-devel curl curl-devel e2fsprogs e2fsprogs-devel krb5 krb5-devel libidn libidn-devel openssl openssl-devel gettext gettext-devel ncurses-devel gmp-devel pspell-devel unzip libcap lsof

 

 

 

編譯pcre的包

tar zxf pcre-8.31.tar.gz cd pcre-8.31 ./configure make && make install useradd -s /sbin/nologno -g nginx -M nginx tar zxf nginx-1.10.2.tar.gz cd nginx-1.10.2 ./configure --prefix=/usr/local/nginx --sbin-path=/usr/local/nginx/bin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx/nginx.pid  --lock-path=/var/lock/nginx.lock --user=nginx --group=nginx --with-http_ssl_module --with-http_flv_module --with-http_stub_status_module --with-http_gzip_static_module --http-client-body-temp-path=/var/tmp/nginx/client/ --http-proxy-temp-path=/var/tmp/nginx/proxy/ --http-fastcgi-temp-path=/var/tmp/nginx/fcgi/ --http-uwsgi-temp-path=/var/tmp/nginx/uwsgi --http-scgi-temp-path=/var/tmp/nginx/scgi --with-pcre make make install /usr/local/nginx/bin/nginx –t ./nginx: error while loading shared libraries: libpcre.so.1: cannot open shared object file: No such file or directory 解決prce的問題 #find / -name libpcre.so*
/usr/local/lib/libpcre.so.1.0.1
/usr/local/lib/libpcre.so /usr/local/lib/libpcre.so.1
/lib64/libpcre.so.0.0.1
/lib64/libpcre.so.0 出現了這么多結果。我們安裝的PCRE庫的位置在/usr/local/pcre中，我們就用這個位置 vim /etc/ld.so.conf 在尾行加入/usr/local/bin root@mail2 bin]# ldconfig #/usr/local/nginx/bin/nginx -t nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful

 

 

 

這就正常了

啟動nginx 

/usr/local/nginx/bin/nginx vim /usr/local/nginx/conf/nginx.conf   

 

在最后面的大括號前面添加一行

include  /usr/local/nginx/conf.d/*.conf;
建立這個目錄
mkdir /usr/local/nginx/conf.d
vim /usr/local/nginx/conf.d/lkq.conf 
upstream backend 
{
  server 192.168.236.150:80  weight=1;
  server 192.168.236.151:80  weight=2;
  #ip_hash;
}
server 
{
    listen      80;
    server_name  www.lkq.com;
    location ~ ^/*
    {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_buffering off;
    proxy_pass http://backend;
    }
}

　　

 

客戶端測試一下

記得在客戶端添加本地hosts

二、配置keepalived

一：安裝過程，在兩台HA機器上分別keepalived:

#ln -s /usr/src/kernels/2.6.18-128.el5-i686/ /usr/src/linux http://www.keepalived.org/software/ keepalived 的下載地址。版本的話自己可以選擇一下版本。樓主選擇的版本是1.2.23的版本
[root@mail2 keepalived-1.2.23]# ./configure –sysconf=/etc [root@mail2 keepalived-1.2.23]# make && make install [root@mail2 keepalived-1.2.23]# ln –s /usr/local/sbin/keepalived /sbin [root@mail2keepalived-1.2.23]#cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak [root@mail4keepalived-1.2.23]# ln -s /etc/init.d/keepalived /etc/rc.d/rc3.d/S99keepalived [root@mail4keepalived-1.2.23]# ln -s /etc/init.d/keepalived /etc/rc.d/rc5.d/S99keepalived

 

二、修改配置文件

[root@Lserver-1 keepalived]# cat keepalived.conf Master : global_defs { notification_email { [e-mail] [e-mail] } notification_email_from [e-mail] smtp_server mail.example.com smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_nginx { # script "killall -0 nginx" script "/etc/keepalived/check_nginx.sh" interval 2 weight -5 fall 3 rise 2 } global_defs { router_id LVS_DEVEL } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 150 advert_int 1 authentication { auth_type PASS auth_pass 1111 } track_script { chk_nginx } virtual_ipaddress { 192.168.236.230 } } }

 

 

Backup

global_defs { notification_email { [e-mail] [e-mail] } notification_email_from [e-mail] smtp_server mail.example.com smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_nginx { # script "killall -0 nginx" script "/etc/keepalived/check_nginx.sh" interval 2 weight -5 fall 3 rise 2 } global_defs { router_id LVS_DEVEL } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 priority 120 advert_int 1 authentication { auth_type PASS auth_pass 1111 } track_script { chk_nginx } virtual_ipaddress { 192.168.236.230 } } }

 

2.2 nginx監控腳本

 

該腳本檢測ngnix的運行狀態，並在nginx進程不存在時嘗試重新啟動ngnix，如果啟動失敗則停止keepalived，准備讓其它機器接管。 /etc/keepalived/check_nginx.sh ： #!/bin/bash counter=$(ps -C nginx --no-heading|wc -l) if [ "${counter}" = "0" ]; then /usr/local/nginx/bin/nginx sleep 2 counter=$(ps -C nginx --no-heading|wc -l) if [ "${counter}" = "0" ]; then /etc/init.d/keepalived stop fi fi

 

 

你也可以根據自己的業務需求，總結出在什么情形下關閉keepalived，如 curl 主頁連續2個5s沒有響應則切換：

#!/bin/bash # curl -IL http://localhost/member/login.htm
# curl --data "memberName=fengkan&password=22" http://localhost/member/login.htm
 count = 0
for (( k=0; k<2; k++ )) do check_code=$( curl --connect-timeout 3 -sL -w "%{http_code}\\n" http://localhost/login.html -o /dev/null )
    if [ "$check_code" != "200" ]; then count = count +1
        continue
    else count = 0
        break fi done if [ "$count" != "0" ]; then # /etc/init.d/keepalived stop exit 1
else exit 0 fi

 

啟動keepalived 

[root@Rserver-1 keepalived]# /etc/init.d/keepalived start Starting keepalived: [ OK ] [root@Rserver-1 keepalived]# ip addr list|grep eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 inet 192.168.236.143/24 brd 192.168.236.255 scope global eth0 inet 192.168.236.230/32 scope global eth0 [root@Rserver-1 keepalived]#

 

 

后面測試一下關閉主上面的keepalived 查看一下備節點的IP

 

 

[root@Lserver-1 keepalived]# ip addr list|grep eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 inet 192.168.236.192/24 brd 192.168.236.255 scope global eth0 inet 192.168.236.230/32 scope global eth0 [root@Lserver-1 keepalived]# ps -ef |grep nginx root 19499      1  0 23:00 ?        00:00:00 nginx: master process /usr/local/nginx/bin/nginx nginx 19501  19499  0 23:00 ?        00:00:00 nginx: worker process root 21102   3820  0 23:12 pts/0    00:00:00 grep nginx [root@Rserver-1 keepalived]#

 

 

已經成功了。

EOF