MySQL Router

本文轉載自查看原文 2019-08-13 17:35 2914 MySQL/ 08-MySQL

一、介紹
二、環境准備
三、安裝和配置MySQL Router
- 3.1bootstrap模式安裝MySQL Router
- 3.2簡單模式安裝MySQL Router
四、功能測試
- 4.1路由驗證
  - 4.1.1測試通過MySQL Router連接6446 RW端口，查看連接的是否為primary
  - 4.1.2測試通過MySQL Router連接6447 RO端口，查看連接的是否為seconde節點，rr算法是否生效
- 4.2高可用測試
五、運維相關
- 5.1數據庫節點變更
六、問題匯總
七、個人總結

一、介紹

1.1MySQL Router簡介

MySQL Router是MySQL官方提供的一個輕量級中間件，是InnoDB Cluster的一部分，可在應用程序和后端MySQL服務器之間提供透明路由。主要用以解決MySQL主從庫集群的高可用、負載均衡、易擴展等問題。Router作為一個流量轉發層，位於應用與MySQL服務器之間，其功能類似於LVS。

1.2為什么要用MySQL Router？

基於組復制(MGR)的機制，當主節點宕機離開集群，剩余的其他節點會基於paxos協議選舉一個新的主節點。這里有一個問題，應用程序端如果連接到了主節點，這時主節點宕機離開集群，可用的數據庫IP地址發生變化，客戶端應用程序這個時候還是會向失敗的節點嘗試連接，雖然可以修改客戶端應用程序的連接配置，但是這種情況基本是不現實的。

1.3MySQL Router關系圖

結合MySQL Group Replication和MySQL Shel如下：

上圖充分說明了MySQL Router在InnoDB集群里面的角色，主要作用是為數據庫集群提供一個虛擬IP作為應用程序單一連接點，通過這個單一的連接點實現負載均衡，讀寫分離，故障轉移等數據庫高可用方案。

MySQL Router推薦安裝在應用程序所在的機器上，原因包括：

通過本地Unix套接字連接，而不是TCP/IP，提升性能
降低網絡延遲
MySQL實例不需要額外的賬號，只需要一個router@198.51.100.45, 而不是myapp@%
提升應用程序服務器的可擴展性

1.4MySQL Cluster簡介

Cluster是這個高可用方案中的一個虛擬節點，它會在MGR的所有成員上創建一個名為MySQL_innodb_cluster_metadata的數據庫，存儲集群的元數據信息，包括集群信息、集群成員、組復制信息、連接的MySQL Router等信息，以提供MySQL Router查詢。它相當於對組復制上的成員做了一層邏輯上的封裝，以一個集群的模式展現出來，各節點的狀態與對應實例在組復制中成員的狀態實時同步，但是集群的節點與組復制的成員只在創建集群時同步，后期組復制的成員變更並不自動同步到集群中，可以在集群中做手動的節點增減，這樣使得面向應用端的具體實例實現了更可控更靈活的高可用。

二、環境准備

主機名	IP地址	角色
oratest51	172.16.9.51	primary
oratest52	172.16.9.52	seconde
test61	172.16.9.61	seconde
node4	172.16.8.68	MySQL Router

操作系統：CentOS Linux release 7.2.1511
MySQL版本：mysql-5.7.26-linux-glibc2.12-x86_64
MySQL Router版本：mysql-router-8.0.17-linux-glibc2.12-x86_64
MySQL Shell版本：mysql-shell-8.0.17-linux-glibc2.12-x86-64bit

三、安裝和配置MySQL Router

MySQL Router有兩種部署模式：

bootstrap模式：支持failover，必須結合InnoDB Cluster使用，在--directory指定的路徑下自動生成安裝目錄，配置文件里的端口為6446和6447。
簡單模式：不支持failover，無需結合InnoDB Cluster使用，一般在主從復制或者主主復制等模式下使用，使用/usr/local/mysqlrouter/share/doc/mysqlrouter/sample_mysqlrouter.conf這個配置文件部署。

3.1bootstrap模式安裝MySQL Router

注意： 使用bootstrap的方式安裝MySQL Router的前提條件是MGR已裝好，MySQL Shell已裝好，InnoDB Cluster已經配置好

3.1.1使用yum源的方式安裝MySQL Router

# wget -P /software/ https://repo.mysql.com//mysql80-community-release-el7-3.noarch.rpm
# rpm -Uvh /software/mysql80-community-release-el7-3.noarch.rpm
# yum -y install mysql-router

3.1.2查看MySQL Router的默認配置

[root@node4 yum.repos.d]# mysqlrouter --help
MySQL Router  Ver 8.0.16 for Linux on x86_64 (MySQL Community - GPL)
Copyright (c) 2015, 2019, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Configuration read from the following files in the given order (enclosed
in parentheses means not available for reading):
  /etc/mysqlrouter/mysqlrouter.conf
  (/root/.mysqlrouter.conf)
Plugins Path:
  /usr/lib64/mysqlrouter

Default Log Directory:
  /var/log/mysqlrouter

Default Persistent Data Directory:
  /var/lib/mysqlrouter

Default Runtime State Directory:
  /run/mysqlrouter

通過mysqlrouter --help可以看到默認的一些路徑：
- 配置文件：/etc/mysqlrouter/mysqlrouter.conf
- 插件路徑：/usr/lib64/mysqlrouter
- 日志路徑：/var/log/mysqlrouter
- 持久化數據路徑：/var/lib/mysqlrouter
- 運行時狀態路徑：/run/mysqlrouter
如果在mysqlrouter命令行使用--config或-c選項傳入用戶定義的配置文件，則不會加載默認配置文件。

3.1.3初始化MySQL Router，地址填primary的地址

[root@node4 tmp]# mysqlrouter --bootstrap root@172.16.9.51:3306 --directory /data/mysqlrouter --conf-use-sockets --user=root
Please enter MySQL password for root: 
WARNING: The MySQL server does not have SSL configured and metadata used by the router may be transmitted unencrypted.
# Bootstrapping MySQL Router instance at '/data/mysqlrouter'...

- Checking for old Router accounts
  - No prior Router accounts found
- Creating mysql account mysql_router1_zhi3m2uhudci@'%' for cluster management
- Storing account in keyring
- Adjusting permissions of generated files
- Creating configuration /data/mysqlrouter/mysqlrouter.conf

# MySQL Router configured for the InnoDB cluster 'st'    #InnoDB集群名

After this MySQL Router has been started with the generated configuration

    $ mysqlrouter -c /data/mysqlrouter/mysqlrouter.conf

the cluster 'st' can be reached by connecting to:

## MySQL Classic protocol

- Read/Write Connections: localhost:6446, /data/mysqlrouter/mysql.sock
- Read/Only Connections:  localhost:6447, /data/mysqlrouter/mysqlro.sock

## MySQL X protocol

- Read/Write Connections: localhost:64460, /data/mysqlrouter/mysqlx.sock
- Read/Only Connections:  localhost:64470, /data/mysqlrouter/mysqlxro.sock

--bootstrap：引導並配置Router以便與MySQL InnoDB集群一起運行。

3.1.4編輯配置文件，一般只需要配置bootstrap_server_addresses這一行即可

# File automatically generated during MySQL Router bootstrap
[DEFAULT]
user=root
logging_folder=/data/mysqlrouter/log
runtime_folder=/data/mysqlrouter/run
data_folder=/data/mysqlrouter/data
keyring_path=/data/mysqlrouter/data/keyring
master_key_path=/data/mysqlrouter/mysqlrouter.key
connect_timeout=15
read_timeout=30
#dynamic_state=/data/mysqlrouter/data/state.json

[logger]
level = INFO

[metadata_cache:st]
router_id=14
bootstrap_server_addresses=mysql://oratest51:3306,mysql://oratest52:3306,mysql://test61:3306
user=mysql_router14_ebhje7bsnckc
metadata_cluster=st
ttl=300
use_gr_notifications=0

[routing:st_default_rw]
bind_address=0.0.0.0
bind_port=6446
socket=/data/mysqlrouter/mysql.sock
destinations=metadata-cache://st/default?role=PRIMARY
routing_strategy=first-available
protocol=classic

[routing:st_default_ro]
bind_address=0.0.0.0
bind_port=6447
socket=/data/mysqlrouter/mysqlro.sock
destinations=metadata-cache://st/default?role=SECONDARY
routing_strategy=round-robin-with-fallback
protocol=classic

[routing:st_default_x_rw]
bind_address=0.0.0.0
bind_port=64460
socket=/data/mysqlrouter/mysqlx.sock
destinations=metadata-cache://st/default?role=PRIMARY
routing_strategy=first-available
protocol=x

[routing:st_default_x_ro]
bind_address=0.0.0.0
bind_port=64470
socket=/data/mysqlrouter/mysqlxro.sock
destinations=metadata-cache://st/default?role=SECONDARY
routing_strategy=round-robin-with-fallback
protocol=x

3.1.5啟動MySQL Router

[root@node2 mysqlrouter]# cd /data/mysqlrouter/
[root@node2 mysqlrouter]# sh start.sh

啟動過程較慢，日志里可以看到啟動信息

3.1.6測試連接MySQL Router

默認情況下，使用6446作為RW端口，使用6447作為RO端口

# mysql -uroot -p123456 -h172.16.8.68 -P6446
# mysql -uroot -p123456 -h172.16.8.68 -P6447
# netstat -ntlp |grep mysqlrouter

3.2簡單模式安裝MySQL Router

3.2.1下載軟件包並上傳到服務器

[root@node4 etc]# mkdir /software;cd /software
[root@node4 etc]# wget https://cdn.mysql.com//Downloads/MySQL-Router/mysql-router-8.0.17-linux-glibc2.12-x86_64.tar.xz

3.2.2解壓到指定目錄

[root@node4 etc]# tar -Jxvf /software/mysql-router-8.0.17-linux-glibc2.12-x86_64.tar.xz -C /usr/local/
[root@node4 etc]# cd /usr/local
[root@node4 etc]# mv mysql-router-8.0.17-linux-glibc2.12-x86_64/ mysqlrouter

3.2.3設置環境變量

[root@node4 etc]# echo 'export PATH=/usr/local/mysqlrouter/bin/:$PATH' >> /etc/profile
[root@node4 etc]# source /etc/profile

3.2.4編輯配置文件

# mkdir -p /usr/local/mysqlrouter/etc
# cp /usr/local/mysqlrouter/share/doc/mysqlrouter/sample_mysqlrouter.conf /usr/local/mysqlrouter/etc
# mv /usr/local/mysqlrouter/etc/sample_mysqlrouter.conf /usr/local/mysqlrouter/etc/mysqlrouter.conf
# vim /usr/local/mysqlrouter/etc/mysqlrouter.conf
##部分省略
[DEFAULT]
logging_folder =/usr/local/mysqlrouter/log/
plugin_folder = /usr/local/mysqlrouter/lib/mysqlrouter/
config_folder = /usr/local/mysqlrouter/etc/
runtime_folder = /usr/local/mysqlrouter/run/
data_folder = /usr/local/mysqlrouter/data/
keyring_path = /var/lib/keyring-data
master_key_path = /var/lib/keyring-key

[logger]
level = INFO

[routing:masters]
bind_address = 172.16.8.68:7002
destinations = 172.16.9.61:3306
mode = read-write
connect_timeout = 2

[routing:slaves]
bind_address = 172.16.8.68:7001
destinations = 172.16.9.51:3306,172.16.9.52:3306
mode = read-only
connect_timeout = 1
##部分省略

/usr/local/mysqlrouter/share/doc/mysqlrouter/sample_mysqlrouter.conf這個是MySQL Router的參照配置文件，所以這里先把模板復制出來。

mysql router默認會尋找安裝目錄下的"mysqlrouter.conf"和家目錄下的".mysqlrouter.conf"。也可以在二進制程序mysqlrouter命令下使用"-c"或者"--config"手動指定配置文件。

MySQL Router的配置文件是片段式的，常用的就3個片段：[DEFAULT]、[logger]、[routing:NAME]，片段名稱區分大小寫，且只支持單行"#"或";"注釋。

bind_address：添加MySQL Router所在機器的IP
destinations：后端的mysql服務器IP+端口
mode：提供兩種mode，read-only和read-write，read-write的調度方式為first-available，read-only的調度方式為round-robin
connect_timeout：連接超時時間
routing_strategy：路由策略，有round-robin和first-available策略
更多配置選項可以參考：https://dev.mysql.com/doc/mysql-router/8.0/en/mysql-router-conf-options.html

如果是多主模式，配置文件如下

[routing:mutili_rw]
bind_address=172.16.8.68
bind_port=7003
destinations=172.16.9.51:3306,172.16.9.52:3306,172.16.9.61:3306
mode=read-write
connect_timeout=2
protocol=classic

3.2.5創建上一步中配置文件里指定的目錄

[root@node4 etc]# mkdir -p /usr/local/mysqlrouter/run/
[root@node4 etc]# mkdir -p /usr/local/lib/mysqlrouter
[root@node4 etc]# mkdir -p /usr/local/mysqlrouter/etc/
[root@node4 etc]# mkdir -p /usr/local/mysqlrouter/run/
[root@node4 etc]# mkdir -p /usr/local/mysqlrouter/data/
[root@node4 etc]# mkdir -p /usr/local/mysqlrouter/log/

3.2.6啟動MySQL Router

[root@node4 etc]# mysqlrouter --config /usr/local/mysqlrouter/etc/mysqlrouter.conf &

3.2.7查看MySQL Router的端口監聽是否已開啟

[root@node4 etc]# netstat -ntlp |grep mysqlrouter
tcp        0      0 172.16.8.68:7001        0.0.0.0:*               LISTEN      9221/mysqlrouter    
tcp        0      0 172.16.8.68:7002        0.0.0.0:*               LISTEN      9221/mysqlrouter

MySQL Router的停止暫時沒有停止腳本，如果要停止需要用kill -9的方式殺掉進程

四、功能測試

以下測試都是在InnoDB Cluster下，MySQL Router的安裝模式是bootstrap

4.1路由驗證

4.1.1測試通過MySQL Router連接6446 RW端口，查看連接的是否為primary

[root@node4 mysqlrouter]# for ((i=0;i<=5;i++));do mysql -h172.16.8.101 -uroot -p123456 -P6446 -e"select @@hostname;";done;
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51  |
+------------+

測試結果： 使用6446 RW端口會訪問到后端InnoDB Cluster的primary

4.1.2測試通過MySQL Router連接6447 RO端口，查看連接的是否為seconde節點，rr算法是否生效

[root@node4 mysqlrouter]# for ((i=0;i<=5;i++));do mysql -h172.16.8.101 -uroot -p123456 -P6447 -e"select @@hostname;";done;
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+

測試結果： 使用6447 RO端口會連接到后端InnoDB Cluster集群中所有的seconde節點，並且以rr(round-robin)的策略調度

4.2高可用測試

4.2.1模擬其中一台seconde節點宕機，查看MySQL Router是否會自動取消該節點的路由

(1) 停掉一台seconde節點

[root@oratest52 ~]# systemctl status mysql
[root@oratest52 ~]# systemctl stop mysql
[root@oratest52 ~]# systemctl status mysql

(2) 在InnoDB Cluster中查看集群狀態，可以看到oratest52已經處於MISSING狀態

 MySQL  172.16.9.51:3306  JS > cluster.status()
{
    "clusterName": "st", 
    "defaultReplicaSet": {
        "name": "default", 
        "primary": "oratest51:3306", 
        "ssl": "DISABLED", 
        "status": "OK_NO_TOLERANCE", 
        "statusText": "Cluster is NOT tolerant to any failures. 1 member is     not active", 
        "topology": {
            "oratest51:3306": {
                "address": "oratest61:3306", 
                "mode": "R/O", 
                "readReplicas": {}, 
                "role": "HA", 
                "status": "ONLINE"
            }, 
            "oratest52:3306": {
                "address": "oratest52:3306", 
                "mode": "n/a", 
                "readReplicas": {}, 
                "role": "HA", 
                "shellConnectError": "MySQL Error 2003 (HY000): Can't connect     to MySQL server on 'oratest52' (111)", 
                "status": "(MISSING)"
            }, 
            "test61:3306": {
                "address": "test51:3306", 
                "mode": "R/W", 
                "readReplicas": {}, 
                "role": "HA", 
                "status": "ONLINE"
            }
        }, 
        "topologyMode": "Single-Primary"
    }, 
    "groupInformationSourceMember": "test61:3306"
}

(3) 在MGR中查看GR狀態，可以看到已經沒有oratest52這台機器

mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 48cdca42-a386-11e9-95f1-0050569d2587 | test61      |        3306 | ONLINE       |
| group_replication_applier | e9ee5267-a386-11e9-ac35-0050569d9ef5 | oratest51   |        3306 | ONLINE       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+

(4) 測試通過6447 RO端口連接，查看是否會自動剔除宕機的oratest52節點

[root@node4 ~]# for ((i=0;i<=5;i++));do mysql -h172.16.8.101 -uroot -p123456 -P6447 -e"select @@hostname;";done;
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+

測試結果： MySQL Router會自動剔除宕機的oratest52節點，read請求只會轉發到后端正常的MySQL實例，過程無需人工干預。

4.2.2啟動上一步中停止的MySQL實例，確認MGR和InnoDB Cluster一切正常后，查看MySQL Router是否會自動加入這台已恢復的實例進行轉發

(1) 啟動oratest52的MySQL服務

[root@oratest52 ~]# systemctl start mysql

(2)啟動MGR，並確認MGR狀態正常

mysql> start group_replication;
Query OK, 0 rows affected (2.92 sec)

mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 48cdca42-a386-11e9-95f1-0050569d2587 | test61      |        3306 | ONLINE       |
| group_replication_applier | 50bc1160-a386-11e9-92c9-0050569dc0da | oratest52   |        3306 | ONLINE       |
| group_replication_applier | e9ee5267-a386-11e9-ac35-0050569d9ef5 | oratest51   |        3306 | ONLINE       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+

(3) 檢查InnoDB Cluster是否正常

 MySQL  172.16.9.51:3306  JS > cluster.status()
{
    "clusterName": "st", 
    "defaultReplicaSet": {
        "name": "default", 
        "primary": "oratest51:3306", 
        "ssl": "DISABLED", 
        "status": "OK", 
        "statusText": "Cluster is ONLINE and can tolerate up to ONE failure.", 
        "topology": {
            "oratest51:3306": {
                "address": "oratest61:3306", 
                "mode": "R/O", 
                "readReplicas": {}, 
                "role": "HA", 
                "status": "ONLINE"
            }, 
            "oratest52:3306": {
                "address": "oratest52:3306", 
                "mode": "R/O", 
                "readReplicas": {}, 
                "role": "HA", 
                "status": "ONLINE"
            }, 
            "test61:3306": {
                "address": "test51:3306", 
                "mode": "R/W", 
                "readReplicas": {}, 
                "role": "HA", 
                "status": "ONLINE"
            }
        }, 
        "topologyMode": "Single-Primary"
    }, 
    "groupInformationSourceMember": "test61:3306"
}

(4) 查看恢復正常的實例，能不能自動加入MySQL Router進行轉發，可以看到oratest52可以被正常調度

[root@node4 mysqlrouter]# for ((i=0;i<=5;i++));do mysql -h172.16.8.101 -uroot -p123456 -P6447 -e"select @@hostname;";done;
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+

測試結果： 恢復正常的MySQL實例可以自動加入MySQL Router並進行轉發，無需人工干預。

4.2.3模擬停掉primary節點，測試MySQL Router如何處理發送RW端口的請求，MySQL Router怎樣獲取新的primary節點

(1) 當前primary節點是oratest51，在primary節點上停止MySQL，確認停止成功

[root@oratest51 data]# systemctl status mysql
[root@oratest51 data]# systemctl stop mysql
[root@oratest51 data]# systemctl status mysql
[root@oratest51 data]# ps -ef |grep mysql

(2) 查看InnoDB Cluster狀態，可以看到oratest51已經處於MISSING狀態，primary節點已經切換到了test61

 MySQL  172.16.9.61:3306  JS > cluster.status();
{
    "clusterName": "st", 
    "defaultReplicaSet": {
        "name": "default", 
        "primary": "test61:3306", 
        "ssl": "DISABLED", 
        "status": "OK_NO_TOLERANCE", 
        "statusText": "Cluster is NOT tolerant to any failures. 1 member is not active", 
        "topology": {
            "oratest51:3306": {
                "address": "oratest51:3306", 
                "mode": "n/a", 
                "readReplicas": {}, 
                "role": "HA", 
                "shellConnectError": "MySQL Error 2003 (HY000): Can't connect to MySQL server on 'oratest51' (111)", 
                "status": "(MISSING)"
            }, 
            "oratest52:3306": {
                "address": "oratest52:3306", 
                "mode": "R/O", 
                "readReplicas": {}, 
                "role": "HA", 
                "status": "ONLINE"
            }, 
            "test61:3306": {
                "address": "test61:3306", 
                "mode": "R/W", 
                "readReplicas": {}, 
                "role": "HA", 
                "status": "ONLINE"
            }
        }, 
        "topologyMode": "Single-Primary"
    }, 
    "groupInformationSourceMember": "test61:3306"
}

(3) 測試連接到RW 6446端口，查看是否可以正常連接到新的primary節點test61，並且寫入數據否正常

[root@node2 mysqlrouter]# for ((i=0;i<=5;i++));do mysql -h172.16.8.101 -uroot -p123456 -P6446 -e"select @@hostname;";done;
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+

測試結果： 當primary節點掛掉之后，MySQL Router會自動去連接新的primary節點

4.2.4啟動上一步中停掉的oratest51，將狀態恢復正常，測試MySQL Router能不能自動加入這台已恢復的節點並調度

(1) 啟動oratest51，將狀態恢復正常

[root@oratest51 ~]# systemctl start mysql
[root@oratest51 ~]# systemctl status mysql
[root@oratest51 ~]# mysql -uroot -p123456
mysql> start group_replication;
mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            |MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+------------+-------------+--------------+
| group_replication_applier | 48cdca42-a386-11e9-95f1-0050569d2587 |test61      |        3306 | ONLINE       |
| group_replication_applier | 50bc1160-a386-11e9-92c9-0050569dc0da |oratest52   |        3306 | ONLINE       |
| group_replication_applier | e9ee5267-a386-11e9-ac35-0050569d9ef5 |oratest51   |        3306 | ONLINE       |
+---------------------------+--------------------------------------+------------+-------------+--------------+
3 rows in set (0.00 sec)
mysql> quit
[root@oratest51 local]# mysqlsh
 MySQL  JS > shell.connect('root@172.16.9.51:3306')
 MySQL  172.16.9.51:3306  JS > var cluster = dba.getCluster()
 MySQL  172.16.9.51:3306  JS > cluster.status();
{
    "clusterName": "st", 
    "defaultReplicaSet": {
        "name": "default", 
        "primary": "test61:3306", 
        "ssl": "DISABLED", 
        "status": "OK", 
        "statusText": "Cluster is ONLINE and can tolerate up to ONE failure.", 
        "topology": {
            "oratest51:3306": {
                "address": "oratest51:3306", 
                "mode": "R/O", 
                "readReplicas": {}, 
                "role": "HA", 
                "status": "ONLINE"
            }, 
            "oratest52:3306": {
                "address": "oratest52:3306", 
                "mode": "R/O", 
                "readReplicas": {}, 
                "role": "HA", 
                "status": "ONLINE"
            }, 
            "test61:3306": {
                "address": "test61:3306", 
                "mode": "R/W", 
                "readReplicas": {}, 
                "role": "HA", 
                "status": "ONLINE"
            }
        }, 
        "topologyMode": "Single-Primary"
    }, 
    "groupInformationSourceMember": "test61:3306"
}

(2) 測試RO端口的Router調度

[root@node4 mysqlrouter]# for ((i=0;i<=5;i++));do mysql -h172.16.8.101 -uroot -p123456 -P6447 -e"select @@hostname;";done;
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51     |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51     |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52  |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61     |
+------------+

測試結果： 恢復正常的實例加入集群后，會自動加入到MySQL Router並進行調度

五、運維相關

5.1數據庫節點變更

MySQL Router在初始化配置的時候是連接到集群節點讀取集群的元數據的。如果在集群中新增或減少節點，需要同步更新MySQL Router的配置，否則無法生效

# mysqlrouter --bootstrap 172.16.9.51:3306 --directory /data/mysqlrouter --user=root --conf-use-sockets --force

更新了MySQL Router的配置的配置，需要重啟MySQL Router：

#systemctl restart mysqlrouter
#systemctl status mysqlrouter

六、問題匯總

6.1初始化MySQL Router前，需要停止group_replication

6.2使用--bootstrap初始化MySQL Router報錯`Unable to connect to the metadata server`

報錯如下

[root@oratest51 data]# mysqlrouter --bootstrap root@172.16.9.51:3310 --directory /data/mysqlrouter --conf-use-sockets --user=root
Please enter MySQL password for root: 
Error: Unable to connect to the metadata server: Error connecting to MySQL server at 172.16.9.51:3310: Can't connect to MySQL server on '172.16.9.51' (111) (2003)

原因： 需要配置連接用戶的權限
解決辦法： 給連接用戶配置所有權限，如下：

mysql> grant all privileges  on *.* to root@'%' identified by "123456";
mysql> flush privileges;

6.3使用--bootstrap初始化MySQL Router報錯缺少`mysql_innodb_cluster_metadata.schema_version`表

[root@oratest51 data]# mysqlrouter --bootstrap root@172.16.9.51:3306 --directory /data/mysqlrouter --conf-use-sockets --user=root
Please enter MySQL password for root: 
Error: Expected MySQL Server '172.16.9.51:3306' to contain the metadata of MySQL InnoDB Cluster, but the schema does not exist.
Checking version of the metadata schema failed with: Error executing MySQL query: Table 'mysql_innodb_cluster_metadata.schema_version' doesn't exist (1146)

See https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-creating.html for instructions on setting up a MySQL Server to act as an InnoDB Cluster Metadata server

原因： 沒有創建InnoDB Cluster集群，創建InnoDB Cluster集群后會自動生成這張表。mysql_innodb_cluster_metadata.schema_version表的用途是MySQL Router在進行調度分配的時候，需要讀取這張表的內容來做調度策略。
解決辦法： 創建InnoDB Cluster集群

6.4啟動MySQL Router hang住不動，查看日志報錯`Error: bootstrap_server_addresses is not allowed when dynamic state file is used`，具體如下：

[root@node2 log]# tailf /root/mysqlrouter/log/mysqlrouter.log 
2019-08-02 15:37:52 routing INFO [7f9721e2d700] [routing:st_default_ro] started: listening using /root/mysqlrouter/mysqlro.sock
2019-08-02 15:37:52 routing INFO [7f972162c700] [routing:st_default_rw] started: listening using /root/mysqlrouter/mysql.sock
2019-08-02 15:37:52 routing INFO [7f9720e2b700] [routing:st_default_x_ro] started: listening using /root/mysqlrouter/mysqlxro.sock
2019-08-02 15:37:52 routing INFO [7f9713fff700] [routing:st_default_x_rw] started: listening on 0.0.0.0:64460, routing strategy = first-available
2019-08-02 15:37:52 routing INFO [7f9713fff700] [routing:st_default_x_rw] started: listening using /root/mysqlrouter/mysqlx.sock
2019-08-02 15:37:52 routing INFO [7f9713fff700] [routing:st_default_x_rw] stopped
2019-08-02 15:37:52 routing INFO [7f9721e2d700] [routing:st_default_ro] stopped
2019-08-02 15:37:52 routing INFO [7f972162c700] [routing:st_default_rw] stopped
2019-08-02 15:37:52 routing INFO [7f9720e2b700] [routing:st_default_x_ro] stopped
2019-08-02 15:37:52 main ERROR [7f9726f1b880] Error: bootstrap_server_addresses is not allowed when dynamic state file is used

原因： 開啟了dynamic_state
解決辦法： 在配置文件中注釋掉dynamic_state所在行，例如下列第11行

 1 # File automatically generated during MySQL Router bootstrap
 2 [DEFAULT]
 3 user=root
 4 logging_folder=/root/mysqlrouter/log
 5 runtime_folder=/root/mysqlrouter/run
 6 data_folder=/root/mysqlrouter/data
 7 keyring_path=/root/mysqlrouter/data/keyring
 8 master_key_path=/root/mysqlrouter/mysqlrouter.key
 9 connect_timeout=15
10 read_timeout=30
11 #dynamic_state=/root/mysqlrouter/data/state.json

6.5啟動MySQL Router hang住不動，查看日志報錯`Unable to fetch live group_replication member data from any server in replicaset 'default'`，具體如下：

[root@node2 log]# tailf /root/mysqlrouter/log/mysqlrouter.log 
2019-08-02 15:46:41 metadata_cache WARNING [7f3030405700] While updating metadata, could not establish a connection to replicaset 'default' through test61:3306
2019-08-02 15:46:51 metadata_cache WARNING [7f3030405700] While updating metadata, could not establish a connection to replicaset 'default' through oratest52:3306
2019-08-02 15:47:01 metadata_cache WARNING [7f3030405700] While updating metadata, could not establish a connection to replicaset 'default' through oratest51:3306
2019-08-02 15:47:01 metadata_cache ERROR [7f3030405700] Unable to fetch live group_replication member data from any server in replicaset 'default'

原因： MySQL Router所在節點沒有配置本機DNS解析
解決辦法： 在/etc/hosts中添加MGR實例的解析

七、個人總結

MySQL Router有兩種部署模式：
- bootstrap模式：支持failover，必須結合InnoDB Cluster使用，在--directory指定的路徑下自動生成安裝目錄，配置文件里的端口為6446和6447。
- 簡單模式：不支持failover，無需結合InnoDB Cluster使用，一般在主從復制或者主主復制等模式下使用，使用/usr/local/mysqlrouter/share/doc/mysqlrouter/sample_mysqlrouter.conf這個配置文件部署。
MySQL Router使用的是一個IP地址(可以理解為VIP)加一個RW端口和一個RO端口實現讀寫分離，Router自身不能判斷應用程序的請求是讀還是寫，因此做讀寫分離時，必須指定兩個端口，如果應用程序不便指定兩個端口，也可以全部指定RW端口，這樣的話從節點就只作高可用，不做讀寫分離。
MySQL Router自身存在單點故障隱患，官方推薦在每個應用程序所在機器上部署Router，本機器連接本機器的Router。也可以在Router的上層在搭建一個高可用服務，如果是自建機房可以選擇Keepalived、pacemaker等方案；
通過bootstrap生成的配置文件，只需要在bootstrap_server_addresses這一欄里配置就可以了，格式如下：
```
bootstrap_server_addresses=mysql://oratest51:3306,mysql://oratest52:3306,mysql://test61:3306
```
MySQL Router的后端MySQL實例掛掉之后，無需人工干預，MySQL Router會自動剔除掛了的MySQL實例，當掛了的MySQL實例恢復后會自動加入MySQL Router的后端服務器，也無需人工干預。
MGR架構可以實現高可用，但是要實現failover，則需要安裝InnoDB Cluster了。MySQL Shell和MySQL Router是InnoDB Cluster集群的一部分
MySQL Router非常輕量級，性能損耗小於1%，官方建議每台應用上部署一個mysqlrouter節點，優點是節省網絡帶寬。缺點是mysqlrouter太輕量級了，只能提供簡單的基於端口的讀寫路由(Port based routing)和基於權重輪詢的負載均衡(Weighted Balance)，不支持基於SQL的讀寫分離(Query based routing)和空閑優先的負載均衡(Application Layer balancer)

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 mysql router使用配置 mysql-router的安裝與使用 mysql-router的安裝與使用 MySQL Router 測試使用轉 MySQL Router實現MySQL的讀寫分離 MySQL Router實現MySQL的讀寫分離 MySQL Router單點隱患通過Keepalived實現 mysql router 中間件配置 my01_Mysql router 安裝基於MySQL innodb cluster和MySQL router的高可用與讀寫分離