聲明：本案例僅為評估測試版本
注意：所有服務器之間必須做好時間同步
架構拓撲
IP信息：
Heartbeat安裝部署
1.安裝heartbeat（主備節點同時安裝）
[root@master1 ~]# yum install heartbeat* -y
[root@master2 ~]# yum install heartbeat* -y
heartbeat的yum安裝完的默認路徑為 /etc/ha.d/
[root@master1 ~]# cd /etc/ha.d/
[root@master1 ha.d]# ls -al
total 28
drwxr-xr-x 4 root root 4096 May 3 19:08 .
drwxr-xr-x. 67 root root 4096 May 3 19:08 ..
-rwxr-xr-x 1 root root 745 Dec 3 2013 harc
drwxr-xr-x 2 root root 4096 May 3 19:08 rc.d
-rw-r--r-- 1 root root 692 Dec 3 2013 README.config
drwxr-xr-x 2 root root 4096 May 3 19:08 resource.d
-rw-r--r-- 1 root root 2082 Mar 24 21:01 shellfuncs
一般安裝任何軟件注意閱讀README文件
從readme文件我們可以獲取到三個重要文件：
- ha.cf：主配置文件
- haresources：資源配置文件
- authkeys：認證信息文件
此三類文件在主備服務器上保持完全一致
[root@master1 ha.d]# cat README.config
You need three configuration files to make heartbeat happy,
and they all go in this directory.
They are:
ha.cf Main configuration file
haresources Resource configuration file
authkeys Authentication information
These first two may be readable by everyone, but the authkeys file
must not be.
The good news is that sample versions of these files may be found in
the documentation directory (providing you installed the documentation).
If you installed heartbeat using rpm packages then
this command will show you where they are on your system:
rpm -q heartbeat -d
If you installed heartbeat using Debian packages then
the documentation should be located in /usr/share/doc/heartbeat
需要將此文件從 /usr/share/doc/heartbeat 下copy到 /etc/ha.d/ 目錄下
[root@master1 ~]# cd /usr/share/doc/heartbeat-3.0.4/
[root@master1 heartbeat-3.0.4]# cp authkeys ha.cf haresources /etc/ha.d/
2.配置ha.cf
ha.cf文件詳解請參考：http://blog.csdn.net/lgstudyvc/article/details/48341185
[root@master1 ha.d]# egrep -v "^$|#" ha.cf
debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility local0
keepalive 2
deadtime 30
warntime 10
initdead 120
mcast eth0:2 225.0.0.1 694 1 0
auto_failback on
node master1
node master2
3.配置authkeys
[root@master1 ha.d]# cat authkeys
#
# Authentication file. Must be mode 600
#
#
# Must have exactly one auth directive at the front.
# auth send authentication using this method-id
#
# Then, list the method and key that go with that method-id
#
# Available methods: crc sha1, md5. Crc doesn't need/want a key.
#
# You normally only have one authentication method-id listed in this file
#
# Put more than one to make a smooth transition when changing auth
# methods and/or keys.
#
#
# sha1 is believed to be the "best", md5 next best.
#
# crc adds no security, except from packet corruption.
# Use only on physically secure networks.
#
auth 2
#1 crc
2 sha1 HI!
#3 md5 Hello!
[root@master1 ha.d]# chmod 600 authkeys #必須修改文件權限為600
4.配置haresources
master1 IPaddr::10.0.0.130/24/eth0:3 drbddisk::data Filesystem::/dev/drbd1::/data::ext3 mysqld
#添加主節點為master設置vip為10.0.0.130/24,網卡為eth0:3
#drbddisk掛在data目錄
#fliesystem將drbd1目錄掛在到/data目錄下，並且啟動mysql服務
DRBD安裝部署
1.兩台主庫數據分區信息
2.對/dev/sdb進行分區
[root@master1 ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x0e4d1c28.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): m
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition's system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-130, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-130, default 130): +500M
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (66-130, default 66):
Using default value 66
Last cylinder, +cylinders or +size{K,M,G} (66-130, default 130):
Using default value 130
Command (m for help): p
Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0e4d1c28
Device Boot Start End Blocks Id System
/dev/sdb1 1 65 522081 83 Linux
/dev/sdb2 66 130 522112+ 83 Linux
3.為新分區創建文件系統
注意：sdb2作為drbd的狀態信息分區不需要格式化創建文件系統，所以只對sdb1進行格式化創建文件系統即可
[root@master1 ~]# mkfs.ext3 /dev/sdb1
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
Stride=0 blocks, Stripe width=0 blocks
130560 inodes, 522080 blocks
26104 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67633152
64 block groups
8192 blocks per group, 8192 fragments per group
2040 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 33 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@master1 ~]# tune2fs -c -1 /dev/sdb1 #只允許一個設備進行掛載
tune2fs 1.41.12 (17-May-2010)
Setting maximal mount count to -1
4.掛在/dev/sdb1分區到/data目錄下，給后續mysql服務器存放data數據
[root@master1 ~]# mount /dev/sdb
sdb sdb1 sdb2
[root@master1 ~]# mount /dev/sdb1 /data/
[root@master1 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_master1-lv_root
6.5G 2.2G 4.1G 35% /
tmpfs 491M 0 491M 0% /dev/shm
/dev/sda1 477M 28M 425M 7% /boot
/dev/sdb1 494M 11M 458M 3% /data
5.安裝drbd內核模塊
[root@master1 ~]# rpm -ivh http://www.elrepo.org/elrepo-release-6-6.el6.elrepo.noarch.rpm
[root@master1 ~]# yum install drbd84* kmod-drbd84
6.確認drbd模塊已經加載到內核
[root@master1 ~]# modprobe drbd
[root@master1 ~]# lsmod | grep drbd
drbd 374888 0
libcrc32c 1246 1 drbd
7.查看drbd的配置文件
查看配置文件可知，需要單獨創建響應的.res資源文件，且有相關的實例文件
[root@master1 etc]# cat drbd.conf
# You can find an example in /usr/share/doc/drbd.../drbd.conf.example
include "drbd.d/global_common.conf";
include "drbd.d/*.res";
[root@master1 drbd.d]# cat /etc/drbd.d/mysql.res
global {
# minor-count 64;
# dialog-refresh 5; # 5 seconds
# disable-ip-verification;
usage-count no;
}
common {
protocol C;
disk {
on-io-error detach;
#size 454G;
no-disk-flushes;
no-md-flushes;
}
net {
sndbuf-size 512k;
# timeout 60; # 6 seconds (unit = 0.1 seconds)
# connect-int 10; # 10 seconds (unit = 1 second)
# ping-int 10; # 10 seconds (unit = 1 second)
# ping-timeout 5; # 500 ms (unit = 0.1 seconds)
max-buffers 8000;
unplug-watermark 1024;
max-epoch-size 8000;
# ko-count 4;
# allow-two-primaries;
cram-hmac-alg "sha1";
shared-secret "hdhwXes23sYEhart8t";
after-sb-0pri disconnect;
after-sb-1pri disconnect;
after-sb-2pri disconnect;
rr-conflict disconnect;
# data-integrity-alg "md5";
# no-tcp-cork;
}
syncer {
rate 120M;
al-extents 517;
}
}
resource data {
on master1 {
device /dev/drbd1;
disk /dev/sdb1;
address 10.0.2.110:7788;
meta-disk /dev/sdb2 [0];
}
on master2 {
device /dev/drbd1;
disk /dev/sdb1;
address 10.0.0.2.120:7788;
meta-disk /dev/sdb2 [0];
}
}
8.初始化meta分區
[root@master1 ~]# drbdadm create-md data
Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
9.初始化設備同步(覆蓋備節點，保持數據一致)
[root@master1 ~]# drbdadm -- --overwrite-data-of-peer primary data
10.啟動drbd
[root@master1 ~]# drbdadm up all
[root@master1 ~]# chkconfig drbd off
11.掛在drbd分區到data目錄下
[root@master1 ~]# drbdadm primary all
[root@master1 ~]# mount /dev/drbd1 /data
說明：/data目錄為數據庫的數據目錄
安裝MySQL數據庫（兩台主服務器上同時安裝）
1.將mysql數據的/data文件夾建立在/dev/sdb1分區的data目錄下
#到MySQL官網下載相應的mysql安裝包，可以選擇rpm，yum方式，此處選擇二進制方式，方便定制安裝路徑
[root@master1 ~]# tar zxvf mysql-5.6.32-linux-glibc2.5-x86_64.tar.gz
#直接將解壓后的mysql文件copy到安裝目錄即可
[root@master1 ~]# mv mysql-5.6.32-linux-glibc2.5-x86_64 /usr/local/mysql-5.6.32
#創建軟連接否則后續利用二進制腳本啟動過程中會保存，因為mysql.server腳本中引用的路徑為/usr/local/mysql
[root@master1 ~]# ln -s /usr/local/mysql-5.6.32/ /usr/local/mysql
#使用mysql自帶的mysql_install_db腳本進行初始化安裝
[root@master1 scripts]# useradd -s /sbin/nologin -M mysql
[root@master1 scripts]# ./mysql_install_db -user=mysql --basedir=/usr/local/mysql-5.6.32/ --datadir=/data/
2.修改my.cnf配置文件
[root@master1 mysql-5.6.32]# vim my.cnf
[root@master1 mysql-5.6.32]# cat my.cnf
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.6/en/server-configuration-defaults.html
[mysqld]
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
# These are commonly set, remove the # and set as required.
basedir = /usr/local/mysql-5.6.32/
datadir = /data
port = 3306
server_id = 110
socket = /tmp/mysql.sock
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
3.復制二進制mysql啟動文件到/etc/init.d/下，方便heartbeat調用
[root@master1 support-files]# cp mysql.server /etc/init.d/mysqld
4.啟動msyqld時出現報錯請刪除系統自帶的/etc/my.cnf配置文件
[root@master1 ~]# /etc/init.d/mysqld start
Starting MySQL... ERROR! The server quit without updating PID file (/var/run/mysqld/mysqld.pid).
[root@master1 ~]# rm -rf /etc/my.cnf
[root@master1 ~]# /etc/init.d/mysqld start
Starting MySQL. SUCCESS!
5.設置mysql數據庫密碼
#復制mysql安裝文件下的bin文件到系統環境路徑下
[root@master1 bin]# mysqladmin -uroot password
New password:
Confirm new password:
[root@master1 ~]# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.32 MySQL Community Server (GPL)
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+---------------------+
| Database |
+---------------------+
| information_schema |
| #mysql50#lost+found |
| mysql |
| performance_schema |
| test |
+---------------------+
5 rows in set (0.01 sec)
Heartbeat+Mysql+DRBD故障測試：
1.主備節點IP信息：
主節點IP：
[root@master1 /]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:8A:B3:73
inet addr:10.0.0.110 Bcast:10.0.0.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe8a:b373/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4055 errors:0 dropped:0 overruns:0 frame:0
TX packets:3888 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:494029 (482.4 KiB) TX bytes:1890864 (1.8 MiB)
eth0:1 Link encap:Ethernet HWaddr 00:0C:29:8A:B3:73
inet addr:10.0.1.110 Bcast:10.0.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth0:2 Link encap:Ethernet HWaddr 00:0C:29:8A:B3:73
inet addr:10.0.2.110 Bcast:10.0.2.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth0:3 Link encap:Ethernet HWaddr 00:0C:29:8A:B3:73
inet addr:10.0.0.130 Bcast:10.0.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
備節點IP：
[root@master2 /]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:85:D5:3D
inet addr:10.0.0.120 Bcast:10.0.0.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe85:d53d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:41412 errors:0 dropped:0 overruns:0 frame:0
TX packets:16511 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:47131293 (44.9 MiB) TX bytes:3064356 (2.9 MiB)
eth0:1 Link encap:Ethernet HWaddr 00:0C:29:85:D5:3D
inet addr:10.0.1.120 Bcast:10.0.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth0:2 Link encap:Ethernet HWaddr 00:0C:29:85:D5:3D
inet addr:10.0.2.120 Bcast:10.0.2.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
2.主備節點drbd信息
主節點：
[root@master1 /]# cat /proc/drbd
version: 8.4.9-1 (api:1/proto:86-101)
GIT-hash: 9976da086367a2476503ef7f6b13d4567327a280 build by mockbuild@Build64R6, 2016-12-13 18:38:15
1: cs:Connected ro:Primary/Secondary ds:UpToDate/Diskless C r-----
ns:1326 nr:26 dw:594 dr:28007 al:16 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:d oos:522084
備節點：
[root@master2 /]# cat /proc/drbd
version: 8.4.9-1 (api:1/proto:86-101)
GIT-hash: 9976da086367a2476503ef7f6b13d4567327a280 build by mockbuild@Build64R6, 2016-12-13 18:38:15
1: cs:Connected ro:Secondary/Primary ds:Diskless/UpToDate C r-----
ns:26 nr:1326 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
3.主備節點data目錄信息
主節點：
[root@master2 data]# ll
total 110616
-rw-rw---- 1 mysql mysql 56 May 3 12:59 auto.cnf
-rw-rw---- 1 mysql mysql 12582912 May 3 14:08 ibdata1
-rw-rw---- 1 mysql mysql 50331648 May 3 14:08 ib_logfile0
-rw-rw---- 1 mysql mysql 50331648 May 3 12:58 ib_logfile1
-rw-r----- 1 mysql root 5849 May 3 14:08 master2.err
drwx------ 2 mysql mysql 4096 May 3 12:58 mysql
drwx------ 2 mysql mysql 4096 May 3 12:58 performance_schema
drwx------ 2 mysql mysql 4096 May 3 12:58 test
備節點：
[root@master1 data]# ll
total 111081
-rw-rw---- 1 mysql mysql 56 May 3 12:47 auto.cnf
-rw-rw---- 1 mysql mysql 12582912 May 3 14:18 ibdata1
-rw-rw---- 1 mysql mysql 50331648 May 3 14:18 ib_logfile0
-rw-rw---- 1 mysql mysql 50331648 May 3 2017 ib_logfile1
drwx------ 2 root root 12288 May 3 2017 lost+found
-rw-r----- 1 mysql root 24692 May 3 14:18 master1.err
-rw-rw---- 1 mysql mysql 5 May 3 14:18 master1.pid
-rw-r----- 1 500 root 4907 May 3 14:08 master2.err
drwx------ 2 mysql mysql 4096 May 3 2017 mysql
drwx------ 2 mysql mysql 4096 May 3 12:41 performance_schema
drwx------ 2 mysql mysql 1024 May 3 2017 test