架構如上,但是其實keepalived、haproxy、Mycat都可以多台(比如keepalived、haproxy、Mycat各3台,3台keepalived搶占vip,然后搶到vip的haproxy提供服務,每台haproxy后面掛3個mycat)
如下是我根據自己的開發環境,自己安裝的步驟:
keepalived-2.0.6.tar.gz
haproxy-1.8.16.tar.gz
Mycat-server-1.6.6.1-release-20181031195535-linux.tar.gz
主機1:ip192.168.0.200 安裝keepalived、haproxy、Mycat
主機2:ip192.168.0.199 安裝keepalived、haproxy、Mycat
vip:192.168.0.206
keepalived與haproxy需安裝在同一台機器上
運作理解:主機1與主機2上的keepalived會為該服務器搶占vip,搶占到vip后,對該主機的訪問可以通過原來的ip(ip192.168.0.200或者192.168.0.199)訪問,也可以直接通過vip(192.168.0.206)訪問,當其中一台宕機時,vip會漂移到另一台機器上,當我們訪問192.168.0.206時haproxy會根據配置將請求分發到兩台Mycat上實現負載均衡,並且haproxy會檢測mycat是否存活。
一、haproxy安裝
1、安裝haproxy(200和199都安裝,安裝方式一樣)
下載地址:
https://src.fedoraproject.org/repo/pkgs/haproxy/
國內鏡像地址:http://pkgs.fedoraproject.org/repo/pkgs/haproxy/
查看內核版本
uname -r
[root@localhost haproxy]# uname -r
3.10.0-957.5.1.el7.x86_64
根據內核版本選擇編譯參數:

這里內核是3.10的可以選擇TARGET=linux310 或者 選擇 linux2628,安裝haproxy
tar -zxvf haproxy-1.8.16.tar.gz
cd haproxy-1.8.16 make TARGET=linux310 ARCH=x86_64 PREFIX=/usr/local/haproxy make install PREFIX=/usr/local/haproxy cd /usr/local/haproxy
安裝成功后,查看版本
[root@localhost haproxy]# /usr/local/haproxy/sbin/haproxy -v
HA-Proxy version 1.8.16-5c3f237 2018/12/21
Copyright 2000-2018 Willy Tarreau <willy@haproxy.org>
2、配置haproxy
cd /usr/local/haproxy touch haproxy.cfg
vim /usr/local/haproxy/haproxy.cfg
寫入一下內容:
global
#設置日志
log 127.0.0.1 local0
chroot /usr/local/haproxy
#用戶與用戶組
user root
group root
#定義每個haproxy進程的最大連接數 ,由於每個連接包括一個客戶端和一個服務器端,所以單個進程的TCP會話最大數目將是該值的兩倍。
maxconn 4096
# 以守護進程的方式運行
daemon
defaults
log global
#日志中不記錄空連接
option dontlognull
# 定義連接后端服務器的失敗重連次數,連接失敗次數超過此值后將會將對應后端服務器標記為不可用
retries 3
option redispatch
# 設置成功連接到一台服務器的最長等待時間,默認單位是毫秒
timeout connect 5000
# 設置連接客戶端發送數據時的成功連接最長等待時間,默認單位是毫秒
timeout client 50000
# 設置服務器端回應客戶度數據發送的最長等待時間,默認單位是毫秒
timeout server 60000
#統計頁面
listen admin_stats
bind 192.168.0.206:48800
mode http
#采用http日志格式
option httplog
#統計頁面自動刷新時間
stats refresh 30s
#統計頁面url
stats uri /admin_stats
#統計頁面密碼框上提示文本
stats realm Haproxy Manager
#統計頁面用戶名和密碼設置
stats auth admin:admin
#隱藏統計頁面上HAProxy的版本信息
stats hide-version
listen mycat_service
# 綁定192.168.0.206:8067端口訪問mycat8066端口
bind 192.168.0.206:8067
# 定義為tcp模式
mode tcp
#采用http日志格式
option tcplog
# 開啟對后端服務器的健康檢測
option httpchk OPTIONS * HTTP/1.1\r\nHost:\ www
# 設置haproxy的調度算法
balance roundrobin
#根據調度分配到真實的后台地址,參數解釋:port 48700:檢測端口48700, inter 5s:5秒檢測一次,rise 2:檢測成功2次表示服務器可用,fall 3:檢測失敗3次后表示服務器不可用
server mycat_200 192.168.0.200:8066 check port 48700 inter 5s rise 2 fall 3
server mycat_199 192.168.0.199:8066 check port 48700 inter 5s rise 2 fall 3
#設置服務器端回應客戶度數據發送的最長等待時間,默認單位是毫秒
timeout server 60000
listen mycat_admin
#綁定192.168.0.206:9067端口訪問mycat9066端口
bind 192.168.0.206:9067
mode tcp
option tcplog
option httpchk OPTIONS * HTTP/1.1\r\nHost:\ www
balance roundrobin
server mycat_200 192.168.0.200:9066 cookie 1 check port 48700 inter 5s rise 2 fall 3
server mycat_199 192.168.0.199:9066 cookie 2 check port 48700 inter 5s rise 2 fall 3
timeout server 60000
上面的配置中 timeout server 可以配置大一點,不然經常會出現有時候閃斷連接mysql的情況
啟動和停止haproxy命令
/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg #啟動 killall haproxy #停止
[root@localhost haproxy]# /usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg
[WARNING] 170/114135 (88204) : config : proxy 'mycat_admin' : ignoring cookie for server 'mycat_200' as HTTP mode is disabled.
[WARNING] 170/114135 (88204) : config : proxy 'mycat_admin' : ignoring cookie for server 'mycat_199' as HTTP mode is disabled.
[ALERT] 170/114135 (88204) : Starting proxy admin_stats: cannot bind socket [192.168.0.206:48800]
[ALERT] 170/114135 (88204) : Starting proxy mycat_service: cannot bind socket [192.168.0.206:8067]
[ALERT] 170/114135 (88204) : Starting proxy mycat_admin: cannot bind socket [192.168.0.206:9067]
3、配置haproxy記錄日志功能
默認haproxy是不記錄日志的,為了記錄日志還需要配置syslog模塊,在linux下是rsyslogd服務,需要先安裝rsyslog
yum install rsyslog -y
[root@localhost haproxy]# yum install rsyslog -y
Loaded plugins: fastestmirror, langpacks
Determining fastest mirrors
* base: mirrors.163.com
* extras: mirrors.163.com
* updates: mirrors.cn99.com
base | 3.6 kB 00:00:00
extras | 3.4 kB 00:00:00
packages-microsoft-com-prod | 2.9 kB 00:00:00
updates | 3.4 kB 00:00:00
(1/3): extras/7/x86_64/primary_db | 205 kB 00:00:00
(2/3): packages-microsoft-com-prod/primary_db | 170 kB 00:00:00
(3/3): updates/7/x86_64/primary_db | 6.4 MB 00:00:01
Package rsyslog-8.24.0-34.el7.x86_64 already installed and latest version
Nothing to do
如果沒有目錄/etc/rsyslog.d 就執行
mkdir /etc/rsyslog.d
cd /etc/rsyslog.d/ touch haproxy.conf
vim haproxy.conf
寫入一下內容
$ModLoad imudp $UDPServerRun 514 local0.* /var/log/haproxy.log
vim /etc/rsyslog.conf
修改兩處:1在#### RULES ####上面一行加入以下內容
# Include all config files in /etc/rsyslog.d/ $IncludeConfig /etc/rsyslog.d/*.conf

第二處在local7.* /var/log/boot.log下面增加
local0.* /var/log/haproxy.log

重啟rsyslog服務
systemctl restart rsyslog.service
將rsyslog加入自動啟動服務
systemctl enable rsyslog.service
二、配置監聽mycat是否存活
在Mycat server1 Mycat server2上都需要添加檢測端口48700的腳本,為此需要用到xinetd,xinetd為linux系統的基礎服務,在xinetd目錄下面增加腳本與端口的映射配置文件。
yum install xinetd -y
cd /etc/xinetd.d touch mycat_status
vim /etc/xinetd.d/mycat_status
寫入以下內容
service mycat_status
{
flags = REUSE
socket_type = stream
port = 48700
wait = no
user = root
server =/usr/local/bin/mycat_status
log_on_failure += USERID
disable = no
}
創建xinetd啟動服務腳本
vim /usr/local/bin/mycat_status
#!/bin/bash #/usr/local/bin/mycat_status.sh # This script checks if a mycat server is healthy running on localhost. It will # return: # # "HTTP/1.x 200 OK\r" (if mycat is running smoothly) # # "HTTP/1.x 503 Internal Server Error\r" (else) mycat=`/root/mycat/bin/mycat status |grep 'not running'| wc -l` if [ "$mycat" = "0" ]; then /bin/echo -e "HTTP/1.1 200 OK\r\n" else /bin/echo -e "HTTP/1.1 503 Service Unavailable\r\n" fi
修改腳本文件權限
chmod 777 /usr/local/bin/mycat_status chmod 777 /etc/xinetd.d/mycat_status
將啟動腳本加入服務
vim /etc/services
在末尾加入
mycat_status 48700/tcp # mycat_status
重啟xinetd服務
systemctl restart xinetd.service
驗證mycat_status服務是否啟動成功
netstat -antup|grep 48700

注意:/etc/xinetd.d/mycat_status里的service mycat_status(服務名)和port = 48700(端口)要與 /etc/services里的相同
三、keepalived安裝
1、安裝依賴
yum install -y openssl-devel openssl libnl libnl-devel libnfnetlink-devel
2、keepalived安裝
(安裝keepalived的時候經常keepalived啟動不了,或者說啟動了但是haproxy切不了,后來試了keepalived2.0.6 2.0.16 1.4.5 都還是一樣,最后按照mycat權威指南換成1.2.13還是不行,后來終於發現可能是
我改了權威指南上寫的/etc/keepalived/scripts/check_haproxy.sh 把兩行的
$STARTHAPROXY 改成了第二行是$STOPKEEPALIVED 導致每次keepalived一啟動就被停了,后面我把兩行都寫成
$STARTHAPROXY就沒問題了,出問題的時候還可以多查看 /usr/local/keepalived/var/log/keepalived-haproxy-state.log 和/var/log/messages 看看是否check或者其他什么的。 還有就是下面這一段我試試了去掉后手動打字也有影響,但是可能也沒影響,
后續再安裝一次驗證一下
vrrp_script chkhaproxy {
script "/etc/keepalived/scripts/check_haproxy.sh"
insterval 2
weight 2
}
)
cd /root
wget http://www.keepalived.org/software/keepalived-1.2.13.tar.gz (如果wget下載不了可以去keepalived官網下載 https://www.keepalived.org/download.html)
tar -zxvf keepalived-1.2.13.tar.gz
cd keepalived-1.2.13 ./configure --prefix=/usr/local/keepalived make make install cp /usr/local/keepalived/sbin/keepalived /usr/sbin/ cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/ (這里不同的keepalived版本路徑可能不同) mkdir /etc/keepalived cd /etc/keepalived/ cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived mkdir -p /usr/local/keepalived/var/log
2、keepalived配置
a、查看網卡
ifconfig

我這里網卡是ens33所以配置里的interface為ens33,主機1與主機2配置基本一樣,state與priority 不同。b、先配置主機2 192.168.0.199的keepalived
vim /etc/keepalived/keepalived.conf
! Configuration Fileforkeepalived
global_defs {
router_id ip199
}
vrrp_script chkhaproxy {
script "/etc/keepalived/scripts/check_haproxy.sh"
insterval 2
weight 2
}
#vrrp實例
vrrp_instance VI_1 {
state BACKUP
#本機網卡
interface ens33
##VRRP組名,兩個節點的設置必須一樣,以指明各個節點屬於同一VRRP組
virtual_router_id 51
#優先級,數值愈大,優先級越高
priority 120
#檢查間隔,默認1秒
advert_int 1
#設置認證
authentication {
#認證方式
auth_type PASS
#認證密碼
auth_pass 1111
}
#虛擬IP地址池,可以有多個IP,每個IP占一行
virtual_ipaddress {
192.168.0.206 dev ens33 scope global
}
track_script {
#調用腳本check_haproxy.sh檢查haproxy是否存活
chkhaproxy
}
#指定當切換到master時,執行的腳本
notify_master /etc/keepalived/scripts/haproxy_master.sh
#指定當切換到backup時,執行的腳本
notify_backup /etc/keepalived/scripts/haproxy_backup.sh
#故障時執行的腳本
notify_fault /etc/keepalived/scripts/haproxy_fault.sh
notify_stop /etc/keepalived/scripts/haproxy_stop.sh
}
c、配置主機1 192.168.0.200的keepalived
! Configuration Fileforkeepalived
global_defs {
router_id ip200
}
vrrp_script chkhaproxy {
script "/etc/keepalived/scripts/check_haproxy.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.206 dev ens33 scope global
}
track_script {
chkhaproxy
}
notify_master /etc/keepalived/scripts/haproxy_master.sh
notify_backup /etc/keepalived/scripts/haproxy_backup.sh
notify_fault /etc/keepalived/scripts/haproxy_fault.sh
notify_stop /etc/keepalived/scripts/haproxy_stop.sh
}
3、創建配置文件和腳本
mkdir /etc/keepalived/scripts
a、創建檢查haproxy是否存活的腳本
vim /etc/keepalived/scripts/check_haproxy.sh
#!/bin/bash
STARTHAPROXY="/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg"
STOPKEEPALIVED="/etc/init.d/keepalived stop"
LOGFILE="/usr/local/keepalived/var/log/keepalived-haproxy-state.log"
echo "[check_haproxy status]" >> $LOGFILE
A=`ps -C haproxy --no-header |wc -l`
echo "[check_haproxy status]" >> $LOGFILE
date >> $LOGFILE
#如果沒有haproxy進程存在,就啟動haproxy,停止keepalived,並寫入日志
if [ $A -eq 0 ];then
echo $STARTHAPROXY >> $LOGFILE
$STARTHAPROXY >> $LOGFILE 2>&1
sleep 5
fi
if [ `ps -C haproxy --no-header |wc -l` -eq 0 ];then
exit 0
else
exit 1
fi
b、創建切換到MASTER與BACKUP腳本
vim /etc/keepalived/scripts/haproxy_master.sh
vim /etc/keepalived/scripts/haproxy_backup.sh
在兩個文件中寫入以下內容
#!/bin/bash
STARTHAPROXY="/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg"
STOPHAPROXY=`ps -ef | grep sbin/haproxy | grep -v grep | awk '{print $2}'| xargs kill -s 9`
LOGFILE="/usr/local/keepalived/var/log/keepalived-haproxy-state.log"
echo "[master]" >> $LOGFILE
date >> $LOGFILE
echo "Being master...." >> $LOGFILE 2>&1
echo "stop haproxy...." >> $LOGFILE 2>&1
$STOPHAPROXY >> $LOGFILE 2>&1
echo "start haproxy...." >> $LOGFILE 2>&1
$STARTHAPROXY >> $LOGFILE 2>&1
echo "haproxy stared ..." >> $LOGFILE
c、創建故障時執行的腳本
vim /etc/keepalived/scripts/haproxy_fault.sh
#!/bin/bash LOGFILE=/usr/local/keepalived/var/log/keepalived-haproxy-state.log echo "[fault]" >> $LOGFILE date >> $LOGFILE
d、創建停止vrrp腳本
vim /etc/keepalived/scripts/haproxy_stop.sh
#!/bin/bash LOGFILE=/usr/local/keepalived/var/log/keepalived-haproxy-state.log echo "[stop]" >> $LOGFILE date >> $LOGFILE
4、賦予腳本可執行權限
chmod 777 /etc/keepalived/scripts/*
5、啟動keepalived
systemctl start keepalived.service
6、加入開機啟動
systemctl enable keepalived.service
四、測試
在瀏覽器輸入http://192.168.0.206:48800/admin_stats

查看8067與9067端口是否在監聽
連接8067端口
mysql -uroot -p123456 -h192.168.0.206 -P8067