galera+mycat高可用集群部署

 環境描述

10.30.162.29  client

  環境描述
10.30.162.29 client
10.30.162.72 mysql1
10.30.162.73 mysql2
10.30.162.74 mysql3
10.30.162.78 proxy1
10.30.162.77 proxy2
環境准備
1）關閉SELinux
2）關閉防火牆
3）ntp同步時間 ntpdate time1.aliyun.com
4）主機之間相互做好解析 /etc/hosts

一、安裝galera

1)配置Yum源

[root@mysql1 ~]# vim /etc/yum.repos.d/galera.repo

[galera]
name=galera
baseurl=http://releases.galeracluster.com/galera-3/centos/7/x86_64/
gpgcheck=0

[root@mysql1 ~]# vim /etc/yum.repos.d/mysql_wsrep.repo

[mysql_wsrep]
name=mysql_wsrep
baseurl=http://releases.galeracluster.com/mysql-wsrep-5.7/centos/7/x86_64/
gpgcheck=0

[root@mysql1 ~]# yum list | grep -E "wsrep|galera"
galera.x86_64 25.3.12-2.el7 epel 
galera-3.x86_64 25.3.24-2.el7 galera 
mysql-wsrep-5.7.x86_64 5.7.23-25.15.el7 mysql_wsrep
mysql-wsrep-client-5.7.x86_64 5.7.23-25.15.el7 mysql_wsrep
mysql-wsrep-common-5.7.x86_64 5.7.23-25.15.el7 mysql_wsrep
mysql-wsrep-devel-5.7.x86_64 5.7.23-25.15.el7 mysql_wsrep
mysql-wsrep-libs-5.7.x86_64 5.7.23-25.15.el7 mysql_wsrep
mysql-wsrep-libs-compat-5.7.x86_64 5.7.23-25.15.el7 mysql_wsrep
mysql-wsrep-server-5.7.x86_64 5.7.23-25.15.el7 mysql_wsrep
mysql-wsrep-test-5.7.x86_64 5.7.23-25.15.el7 mysql_wsrep


[root@mysql1 ~]# yum -y install mysql-wsrep-5.7.x86_64 galera-3.x86_64 

　　此處可以只在一台機器上做，然后開啟緩存(keepcache=1)，下載下來包之后，搭建一個ftp服務器，另外兩台機器直接做一個yum源即可安裝

2)開啟服務並且數據庫修改密碼

[root@mysql1 ~]# systemctl start mysqld
[root@mysql1 ~]# systemctl enable mysqld
[root@mysql1 ~]# grep "temporary password" /var/log/mysqld.log | awk '{print $NF}'
[root@mysql1 ~]# mysqladmin -p'uT1K:*kkeDtZ' password "Qfcloud123."

 

 

3)創建並授權用於數據同步的用戶

[root@mysql1 ~]# mysql -p'Qfcloud123.'
mysql> grant all on *.* to galera@'10.30.162.%' identified by 'Qfcloud123.';
mysql> flush privileges;

　　

二、配置galera

 

[root@mysql1 ~]# vim /etc/my.

server_id=1                          //每個節點的server_id不同
default_storage_engine=InnoDB

wsrep_on=on
wsrep_provider=/usr/lib64/galera-3/libgalera_smm.so
wsrep_cluster_name="GaleraCluster"
wsrep_cluster_address="gcomm://"      //后兩個節點要在//后添加另外兩台的ip，做了解析可以直接寫主機名，逗號分隔
wsrep_node_name="galera1"             //節點名，不要重復
wsrep_node_address="10.30.162.72"     //每個節點的ip
wsrep_sst_auth=galera:Qfcloud123.
wsrep_sst_method=rsync

　　

=====================================================================================
注釋：
wsrep_on=on：啟用wsrep，=on或者=1都可以
wsrep_provider：指定集群同步數據使用的庫文件
wsrep_cluster_name：自定義集群的名字
wsrep_cluster_address：mysql節點加入集群需要的地址，第一個節點的寫法比較特殊
wsrep_node_name：指定集群節點名稱，一般寫本機的主機名即可
wsrep_node_address：指定本機地址
wsrep_sst_auth：指定三台服務器之間進行數據同步時使用的帳號和密碼，需要創建
wsrep_sst_method：數據復制的兩種方式：rsync和xtrabackup，rsync是系統自帶，比較常用。libgalera_smm.so庫只是提供了復制的功能，而真正進行數據同步時要靠rsync機制實現

==========================================================================================

[root@mysql1 ~]# yum -y install rsync
[root@mysql1 ~]# systemctl restart mysqld
[root@mysql1 ~]# ss -antp | grep -E "3306|4567"
LISTEN 0 128 *:4567 *:* users:(("mysqld",pid=17071,fd=12))
ESTAB 0 0 10.30.162.72:4567 10.30.162.73:35708 users:(("mysqld",pid=17071,fd=55))
ESTAB 0 0 10.30.162.72:4567 10.30.162.75:55016 users:(("mysqld",pid=17071,fd=41))
LISTEN 0 80 :::3306 :::* users:(("mysqld",pid=17071,fd=27))

　　

三個節點配置完成之后，可以通過如下命令查詢是否成功

[root@mysql1 ~]# mysql -pQfcloud123.
mysql> show status like "wsrep%";

 

三、安裝mycat

1)安裝JDK環境

[root@tomcat1 ~]# tar xf jdk-8u181-linux-x64.tar.gz -C /usr/local/
[root@tomcat1 ~]# ln -s /usr/local/jdk1.8.0_181/ /usr/local/java
[root@tomcat1 ~]# vim /etc/profile
export JAVA_HOME=/usr/local/java
export PATH=$PATH:$JAVA_HOME/bin
[root@tomcat1 ~]# source /etc/profile
[root@tomcat1 ~]# java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

　　關於JDK下載鏈接的問題，可以看我另外一篇文章《tomcat啟用中8005端口未被占用》

2)安裝mycat

[root@proxy1 ~]# wget http://dl.mycat.io/1.6-RELEASE/Mycat-server-1.6-RELEASE-20161028204710-linux.tar.gz
[root@proxy1 ~]# tar xf Mycat-server-1.6-RELEASE-20161028204710-linux.tar.gz -C /usr/local

3)創建庫，並且授權給mycat一個賬戶

[root@mysql1 ~]# mysql -pQfcloud123.
mysql> create database galera;
mysql> grant all on galera.* to mycat@'10.30.162.%' identified by 'Qfcloud123.';
mysql> flush privileges;

 

4）配置mycat

[root@proxy1 ~]# cd /usr/local/mycat/conf/
[root@proxy1 conf]# vim server.xml

　　

<user name="galera">
<property name="password">123456</property>
<property name="schemas">galera_test</property>
</user>

 

[root@proxy1 conf]# vim schema.xml

　　

<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="">http://io.mycat/">

<schema name="galera_test" checkSQLschema="false" sqlMaxLimit="100" dataNode="dn1">
</schema>
<dataNode name="dn1" dataHost="localhost1" database="galera" />
<dataHost name="localhost1" maxCon="1000" minCon="10" balance="1"
writeType="0" dbType="mysql" dbDriver="native" switchType="3" slaveThreshold="100">
<heartbeat>show status like 'wsrep%'</heartbeat>
<writeHost host="galera1" url="10.30.162.72:3306" user="galera"
password="Qfcloud123.">
</writeHost>
<writeHost host="galera2" url="10.30.162.73:3316" user="galera"
password="Qfcloud123." >
</writeHost>
<writeHost host="galera3" url="10.30.162.75:3326" user="galera"
password="Qfcloud123." >
</writeHost>
</dataHost>
</mycat:schema>

　　

5)驗證

[root@proxy1 ~]# /usr/local/mycat/bin/mycat start
Starting Mycat-server...
[root@proxy1 ~]# ss -antp | grep java
LISTEN 0 1 127.0.0.1:32000 *:* users:(("java",pid=1665,fd=4),("wrapper-linux-x",pid=1663,fd=4))
LISTEN 0 50 :::1984 :::* users:(("java",pid=1665,fd=55))
LISTEN 0 50 :::41825 :::* users:(("java",pid=1665,fd=54))
LISTEN 0 50 :::44913 :::* users:(("java",pid=1665,fd=56))

[root@proxy1 ~]# jps
1350 WrapperSimpleApp
1367 Jps

　　

四、keepalived

1)安裝keepalived

[root@proxy1 ~]# yum -y install keepalived

2)配置keepalived

[root@proxy1 ~]# vim /etc/keepalived/keepalived.conf 
Configuration File for keepalived

global_defs {
    router_id galera1
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.30.162.110
    }
}

virtual_server 10.30.162.100 80 {
    delay_loop 3
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 10.30.162.73 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
        }
    }
    real_server 10.30.162.75 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
        }
    }
    real_server 10.30.162.72 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
        }
    }
}                                            

　　

BACKUP端
修改router_id priority state

[root@proxy1 ~]# systemctl restart keepalived

3)驗證

[root@proxy1 ~]# ip a | grep ens33
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
inet 10.30.162.78/24 brd 10.30.162.255 scope global noprefixroute dynamic ens33
inet 10.30.162.110/32 scope global ens33

[root@physical ~]# mysql -ugalera -p123456 -h 10.30.162.110 -P8066
mysql> show databases;
+----------------+
| DATABASE       |
+----------------+
| galera_test    |
+----------------+
1 row in set (0.00 sec)

mysql> select * from test2;
+---------------+
| id            |
+---------------+
| 100000000     |
+---------------+
1 rows in set (0.01 sec)
    

　　

4)編寫腳本，達到mycat如果宕機后，嘗試重啟服務，未啟動則關閉keepalived

[root@proxy1 ~]# vim /etc/keepalived/mycat.sh
#/bin/bash
/usr/local/mycat/bin/mycat status &>/dev/null
if [ $? -ne 0 ];then
    /usr/local/mycat/bin/mycat restart &>/dev/null
    sleep 5
    if [ "$state" = "running" ];then
        systemctl restart keepalived
    else
        systemctl stop keepalived
    fi
fi

[root@proxy1 ~]# chmod +x /etc/keepalived/mycat.sh
[root@proxy1 ~]# vim /etc/keepalived/keepalived.conf 
···
vrrp_script check_mycat { 
    script "/etc/keepalived/mycat.sh"
    interval 5
} 
track_script {
    check_mycat
} 
···

　　

問題

[root@mysql1 ~]# vim /etc/my.cnf
max_allowed_packet=16M
[root@mysql1 ~]# systemctl restart mysqld

　　

如果將三台數據庫服務器都宕機了，想要再啟動的話，會報如下錯誤

 

[root@mysql1 ~]# vim /var/lib/mysql/grastate.dat
safe_to_bootstrap: 1

修改完第一個節點，第一個節點起來后，再重啟其他兩個節點，即可啟動

第一個節點因為沒有"推薦人"，所以一旦他宕機，如果想要讓它重新加入到之前的集群中，必須
在它的推薦人填上另外機器的ip(解析的話，可以直接填解析名)