目錄
- 架構圖
- 部署詳情
- postgresql的安裝與配置
- pgpool的安裝與配置
寫在安裝前
postgresql是一款很強大的數據庫，具體有多強大呢，請谷歌。。。
網上的相關資料不是很多，參考了大神們的文檔，結合自己趟了的坑，寫了pg集群的安裝文檔。可能有一些參數沒有配置好，希望大神們指出，謝謝。參照本文檔，基本能實現pg的高可用，稍后會寫出測試文檔。
架構圖
部署詳情
| ip | host | 系統 | vip | 部署 |
|---|---|---|---|---|
| 192.168.123.180 | master | CentOS 6.8 | 192.168.123.183 | postgresql、pgpool |
| 192.168.123.181 | slave1 | CentOS 6.8 | 192.168.123.183 | postgresql、pgpool |
| 192.168.123.182 | slave2 | CentOS 6.8 | 192.168.123.183 | postgresql |
postgresql的安裝
1、安裝
1、Rpm包下載地址:
https://yum.postgresql.org/rpmchart.php
下載下面的包：
postgresql95-libs-9.5.8-1PGDG.rhel6.x86_64.rpm
postgresql95-contrib-9.5.8-1PGDG.rhel6.x86_64.rpm
postgresql95-9.5.8-1PGDG.rhel6.x86_64.rpm
postgresql95-server-9.5.8-1PGDG.rhel6.x86_64.rpm
2、安裝：
yum install -y postgresql95-libs-9.5.8-1PGDG.rhel6.x86_64.rpm yum install -y postgresql95-contrib-9.5.8-1PGDG.rhel6.x86_64.rpm yum install -y postgresql95-9.5.8-1PGDG.rhel6.x86_64.rpm yum install -y postgresql95-server-9.5.8-1PGDG.rhel6.x86_64.rpm 3、初始化db：
service postgresql-9.5 initdb
4、設置為開機啟動：
chkconfig postgresql-9.5 on
5、安裝的目錄：
/var/lib/pgsql/9.5
配置白名單與流復制
1、 master的配置
PostgreSQL流復制默認是異步的。在主服務器上提交事務和從服務器上變化可見之間有一個小的延遲，這個延遲遠小於基於文件日志傳送，通常1秒能完成。如果主服務器突然崩潰，可能會有少量數據丟失。
同步復制必須等主服務器和從服務器都寫完WAL后才能提交事務。這樣在一定程度上會增加事務的響應時間。
配置同步復制僅需要一個額外的配置步驟： synchronous_standby_names必須設置為一個非空值。synchronous_commit也必須設置為on。
這里部署的是異步的流復制
1.2、配置白名單:
vim /var/lib/pgsql/9.5/data/pg_hba.conf
在配置文件最后加上：
host all all 192.168.123.180/32 trust
host all all 192.168.123.181/32 trust
host all all 192.168.123.182/32 trust
host replication replica 192.168.123.181/32 trust
host replication replica 192.168.123.182/32 trust
保存退出
我這里由於是內網，全部用的trust，如果要用密碼，請修改為passwd
1.3、修改配置文件:
vim /var/lib/pgsql/9.5/data/postgresql.conf
data_directory = '/app/pgsql/data' #自定義data目錄 listen_addresses = '*' #監聽所有ip archive_mode = on #允許歸檔 archive_command = 'cp %p /app/pgsql/pg_archive/%f' #使用命令歸檔 wal_level = hot_standby #選擇熱備 max_wal_senders = 16 #最多多少個流復制鏈接 wal_keep_segments = 256 #流復制保留最多的xlog數 wal_sender_timeout = 60s #流復制主機發送數據超時時間 max_connections = 99 #從庫的max_connections必須大於主庫的 1.4、創建data目錄，賦權並修改啟動文件
mkdir -p /app/pgsql/data && chown postgres:postgres /app/pgsql/data
mkdir -p /app/pgsql/pg_archive && chown postgres:postgres /app/pgsql/pg_archive
cd /app/pgsql && chmod 700 data cd /app/pgsql && chmod 700 pg_archive 1.5、修改啟動文件
vim /etc/init.d/postgresql-9.5 PGDATA=/app/pgsql/data 1.6、把原data目錄下的文件copy到新的data的目錄下,並改變屬組：
cp -r /var/lib/pgsql/9.5/data/* /app/pgsql/data/ cd /app/pgsql && chown -R postgres:postgres data 1.7、啟動
service postgresql-9.5 start
以后修改配置文件就在/app/pgsql/data下面修改
1.8、創建一個數據庫用戶進行主從同步。創建用戶replica，並賦予登錄和復制的權限
登陸到數據庫里：
su postgres
psql
CREATE ROLE replica login replication encrypted password 'replica'; 
防火牆開放5432端口
在從庫上：
psql -h 192.168.123.180 -U postgres
登陸主庫成功
2、從庫的配置
2.1、自定義data目錄:
mkdir -p /app/pgsql/data/&&chmod 700 /app/pgsql/data/
chown -R postgres:postgres pgsql
mkdir /app/pgsql/pg_archive
cd /app/pgsql chmod 700 pg_archive && chown postgres:postgres pg_archive/ 2.2、切換用戶:
su – postgres
2.3、備份數據：
pg_basebackup -h 192.168.123.180 -U replica -D /app/pgsql/data -X stream –P
2.4、修改啟動文件:
vim /etc/init.d/postgresql-9.5 PGDATA=/app/pgsql/data 2.5、配置recovery.conf
su postgres
cp /usr/pgsql-9.5/share/recovery.conf.sample /app/pgsql/data/recovery.conf
vim /app/pgsql/data/recovery.conf
standby_mode = on \#該節點為從 primary_conninfo = 'host=192.168.123.180 port=5432 user=replica password=replica' \#主服務器的ip、user recovery_target_timeline = 'latest' trigger_file = '/tmp/trigger_file0' 2.6、配置postgresql.conf
vim /app/pgsql/data/postgresql.conf
max_connections = 999 #大於主的連接數 max_standby_streaming_delay = 30s wal_receiver_status_interval = 10s hot_standby_feedback = on 2.7、重啟服務：
service postgresql-9.5 restart
驗證流復制
3.1、在master上登陸psql
查看狀態:
select client_addr,sync_state from pg_stat_replication;
3.2、創建test庫
Create database test;
3.3、slave上登陸psql
查看庫
playboy => \l
發現已同步。
pgpool的安裝
pgpool-II是PostgreSQL服務器之間一種有效的中間件和PostgreSQL數據庫客戶端。它提供了以下功能。
連接池
pgpool-II保存到PostgreSQL服務器的連接，當一個相同新連接(如用戶名、數據庫、協議版本)進來時，重用他們。它減少了連接開銷，提高了系統的整體吞吐量。
復制
pgpool-II可以管理多個PostgreSQL服務器。使用復制功能可以使2個或更多的物理磁盤上創建一個實時備份，這樣服務不會因服務器的磁盤故障而中斷。
負載平衡
如果數據庫是復制的，在任何服務器上執行一個SELECT查詢會返回相同的結果。pgpool-II復制特性的優勢在於減少每個PostgreSQL服務器上的負載，因為它可以使用分布在多個服務器之間進行SELECT查詢，從而提高系統的整體吞吐量。最好是查詢和PostgreSQL服務器數量成一定比例，多用戶同時執行多查詢達到負載均衡最好的效果。
限制連接數
PostgreSQL的最大並發連接數有一定限制的，當超過限制的連接數后，連接會被拒絕。然而，設置增加最大連接數又會增加資源消耗，影響系統性能。pgpool-II也有最大連接數限制，但超過的連接進來時是進行立即排隊，而不是返回一個錯誤。
pgpool-II交互PostgreSQL的后端和前端協議時，起着繼電器的作用。因此，數據庫應用程序(前端)認為pgpool-II是真實的PostgreSQL服務器，服務器(后端)認為pgpool-II是它的客戶端之一。因為pgpool-II在服務器和客戶端是透明的，所以pgpool-II可以使用現有的數據庫應用程序而做到幾乎不修改它們。
版本：3.6
機器：192.168.123.180
192.168.123.181
4.1、免密碼登陸
安裝之前先配置密鑰使master和slave1這兩台虛擬機的postgres用戶能免密連接
先修改postgres的密碼，在root用戶下
passwd postgres
新密碼123456
Master到slave1的免密碼登陸：
在master上切換至postgres用戶，生成密鑰
su postgres ssh-keygen -t rsa
然后全輸入回車
切換到postgres用戶：
su postgres ssh-copy-id -i /var/lib/pgsql/.ssh/id_rsa 192.168.123.181
然后ssh 192.168.123.181 成功，實現master到slave1的免密碼登陸
Slave1到master的免密碼登陸：
流程同上
4.2、安裝
Rpm包下載地址:
http://www.pgpool.net/yum/rpms/3.6/redhat/rhel-6-x86_64/pgpool-II-pg95-3.6.0-1pgdg.rhel6.x86_64.rpm
安裝:
yum install pgpool-II-pg95-3.6.0-1pgdg.rhel6.x86_64.rpm
4.3、配置
Master的配置:
配置白名單：
要和pg_hba.conf登陸權限一致，這里由於是私有雲，都用trust
vim /etc/pgpool-II/pool_hba.conf
在最后改成:
配置pcp管理工具密碼:
pg_md5 pwd
密碼加密
vim /etc/pgpool-II/pcp.conf
把剛才加密的密碼粘貼到文件里
配置系統命令權限:
Root用戶下:
chmod u+s /sbin/ifconfig &&chmod u+s /usr/sbin
配置中間件配置文件:
vim /etc/pgpool-II/pgpool.conf
# CONNECTIONS listen_addresses = '*' port = 9999 pcp_listen_addresses = '*' pcp_port = 9898 # - Backend Connection Settings - backend_hostname0 = 'master' backend_port0 = 5432 backend_weight0 = 1 backend_data_directory0 = ' /app/pgsql/data' backend_flag0 = 'ALLOW_TO_FAILOVER' backend_hostname1 = 'slave' backend_port1 = 5432 backend_weight1 = 1 backend_data_directory1 = ' /app/pgsql/data'' backend_flag1 = 'ALLOW_TO_FAILOVER' # - Authentication - enable_pool_hba = off pool_passwd = 'pool_passwd' # FILE LOCATIONS pid_file_name = '/opt/pgpool/pgpool.pid' replication_mode = off load_balance_mode = on master_slave_mode = on master_slave_sub_mode = 'stream' sr_check_period = 5 sr_check_user = 'repuser' sr_check_password = 'repuser' sr_check_database = 'postgres' #------------------------------------------------------------------------------ # HEALTH CHECK 健康檢查 #------------------------------------------------------------------------------ health_check_period = 10 # Health check period # Disabled (0) by default health_check_timeout = 20 # Health check timeout # 0 means no timeout health_check_user = ' repuser ' # Health check user health_check_password = ' repuser ' #數據庫密碼 # Password for health check user health_check_database = 'postgres' #必須設置，否則primary數據庫down了，pgpool不知道，不能及時切換。從庫流復制還在連接數據，報連接失敗。 #只有下次使用pgpool登錄時，發現連接不上，然后報錯，這時候，才知道掛了，pgpool進行切換。 #主備切換的命令行配置 #------------------------------------------------------------------------------ # FAILOVER AND FAILBACK #------------------------------------------------------------------------------ failover_command = ' /opt/pgpool/failover_stream.sh %d %H /tmp/trigger_file0 ' #------------------------------------------------------------------------------ # WATCHDOG #------------------------------------------------------------------------------ # - Enabling - use_watchdog = on # - Watchdog communication Settings - wd_hostname = 'master' # Host name or IP address of this watchdog # (change requires restart) wd_port = 9000 # port number for watchdog service # (change requires restart) # - Virtual IP control Setting - delegate_IP = ' 192.168.123.183 ' # delegate IP address # If this is empty, virtual IP never bring up. # (change requires restart) if_cmd_path = '/sbin' # path to the directory where if_up/down_cmd # (change requires restart) if_up_cmd = 'ifconfig eth0:0 inet $_IP_$ netmask 255.255.255.0' # startup delegate IP command # (change requires restart) if_down_cmd = 'ifconfig eth0:0 down' # shutdown delegate IP command # (change requires restart) # -- heartbeat mode -- wd_heartbeat_port = 9694 # Port number for receiving heartbeat signal # (change requires restart) wd_heartbeat_keepalive = 2 # Interval time of sending heartbeat signal (sec) # (change requires restart) wd_heartbeat_deadtime = 30 # Deadtime interval for heartbeat signal (sec) # (change requires restart) heartbeat_destination0 = 'slave1' # Host name or IP address of destination 0 # for sending heartbeat signal. # (change requires restart) heartbeat_destination_port0 = 9694 # Port number of destination 0 for sending # heartbeat signal. Usually this is the # same as wd_heartbeat_port. # (change requires restart) heartbeat_device0 = 'eth0' # Name of NIC device (such like 'eth0') # used for sending/receiving heartbeat # signal to/from destination 0. # This works only when this is not empty # and pgpool has root privilege. # (change requires restart) # - Other pgpool Connection Settings - other_pgpool_hostname0 = 'slave' #對端 # Host name or IP address to connect to for # (change requires restart) other_pgpool_port0 = 9999 # Port number for othet pgpool 0 # (change requires restart) other_wd_port0 = 9000 # Port number for othet watchdog 0 # (change requires restart) 4.4、Slave1的配置：
配置白名單：
要和pg_hba.conf登陸權限一致，這里由於是私有雲，都用trust
vim /etc/pgpool-II/pool_hba.conf
在最后改成:
配置pcp管理工具密碼:
pg_md5 pwd
密碼加密
vim /etc/pgpool-II/pcp.conf
把剛才加密的密碼粘貼到文件里
配置系統命令權限:
Root用戶下:
chmod u+s /sbin/ifconfig &&chmod u+s /usr/sbin
配置中間件配置文件:
vim /etc/pgpool-II/pgpool.conf
# CONNECTIONS listen_addresses = '*' port = 9999 pcp_listen_addresses = '*' pcp_port = 9898 # - Backend Connection Settings - backend_hostname0 = 'master' backend_port0 = 5432 backend_weight0 = 1 backend_data_directory0 = ' /app/pgsql/data' backend_flag0 = 'ALLOW_TO_FAILOVER' backend_hostname1 = 'slave1' backend_port1 = 5432 backend_weight1 = 1 backend_data_directory1 = '/app/pgsql/data ' backend_flag1 = 'ALLOW_TO_FAILOVER' # - Authentication - enable_pool_hba = off pool_passwd = 'pool_passwd' # FILE LOCATIONS pid_file_name = '/opt/pgpool/pgpool.pid' replication_mode = off load_balance_mode = on master_slave_mode = on master_slave_sub_mode = 'stream' sr_check_period = 5 sr_check_user = 'repuser' sr_check_password = 'repuser' sr_check_database = 'postgres' #------------------------------------------------------------------------------ # HEALTH CHECK 健康檢查 #------------------------------------------------------------------------------ health_check_period = 10 # Health check period # Disabled (0) by default health_check_timeout = 20 # Health check timeout # 0 means no timeout health_check_user = ' repuser ' # Health check user health_check_password = ' repuser ' #數據庫密碼 # Password for health check user health_check_database = 'postgres' #必須設置，否則primary數據庫down了，pgpool不知道，不能及時切換。從庫流復制還在連接數據，報連接失敗。 #只有下次使用pgpool登錄時，發現連接不上，然后報錯，這時候，才知道掛了，pgpool進行切換。 #主備切換的命令行配置 #------------------------------------------------------------------------------ # FAILOVER AND FAILBACK #------------------------------------------------------------------------------ failover_command = '/opt/pgpool/failover_stream.sh %d %H /tmp/trigger_file0 ' #------------------------------------------------------------------------------ # WATCHDOG #------------------------------------------------------------------------------ # - Enabling - use_watchdog = on # - Watchdog communication Settings - wd_hostname = 'slave1' #本端 # Host name or IP address of this watchdog # (change requires restart) wd_port = 9000 # port number for watchdog service # (change requires restart) # - Virtual IP control Setting - delegate_IP = '192.168.123.183' # delegate IP address # If this is empty, virtual IP never bring up. # (change requires restart) if_cmd_path = '/sbin' # path to the directory where if_up/down_cmd exists # (change requires restart) if_up_cmd = 'ifconfig eth0:0 inet $_IP_$ netmask 255.255.255.0' # startup delegate IP command # (change requires restart) if_down_cmd = 'ifconfig eth0:0 down' # shutdown delegate IP command # (change requires restart) # -- heartbeat mode -- wd_heartbeat_port = 9694 # Port number for receiving heartbeat signal # (change requires restart) wd_heartbeat_keepalive = 2 # Interval time of sending heartbeat signal (sec) # (change requires restart) wd_heartbeat_deadtime = 30 # Deadtime interval for heartbeat signal (sec) # (change requires restart) heartbeat_destination0 = 'master' #對端 # Host name or IP address of destination 0 # for sending heartbeat signal. # (change requires restart) heartbeat_destination_port0 = 9694 # Port number of destination 0 for sending # heartbeat signal. Usually this is the # same as wd_heartbeat_port. # (change requires restart) heartbeat_device0 = 'eth0' # Name of NIC device (such like 'eth0') # used for sending/receiving heartbeat # signal to/from destination 0. # This works only when this is not empty # and pgpool has root privilege. # (change requires restart) # - Other pgpool Connection Settings - other_pgpool_hostname0 = 'master' #對端 # Host name or IP address to connect to for # (change requires restart) other_pgpool_port0 = 9999 # Port number for othet pgpool 0 # (change requires restart) other_wd_port0 = 9000 # Port number for othet watchdog 0 # (change requires restart) 下面在master和slave1這兩台機器上
mkdir /opt/pgpool &&chown postgres:postgres /opt/pgpool
master上：
cd /opt/pgpool vim failover_stream.sh # Failover command for streaming replication. # This script assumes that DB node 0 is primary, and 1 is standby. # # If standby goes down, do nothing. If primary goes down, create a # trigger file so that standby takes over primary node. # # Arguments: $1: failed node id. $2: new master hostname. $3: path to # trigger file. failed_node=192.168.123.180 new_master=192.168.123.181 trigger_file=$3 # Do nothing if standby goes down. if [ $failed_node = 1 ]; then exit 0; fi # Create the trigger file. /usr/bin/ssh -T $new_master /bin/touch $trigger_file exit 0; 保存退出,並賦權
chown postgres:postgres failover_stream.sh &&chmod 777 failover_stream.sh
slave1上：
cd /opt/pgpool vim failover_stream.sh #! /bin/sh # Failover command for streaming replication. # This script assumes that DB node 0 is primary, and 1 is standby. # # If standby goes down, do nothing. If primary goes down, create a # trigger file so that standby takes over primary node. # # Arguments: $1: failed node id. $2: new master hostname. $3: path to # trigger file. failed_node=192.168.123.181 new_master=192.168.123.180 trigger_file=$3 # Do nothing if standby goes down. if [ $failed_node = 1 ]; then exit 0; fi # Create the trigger file. /usr/bin/ssh -T $new_master /bin/touch $trigger_file exit 0; 保存退出並賦權:
chown postgres:postgres failover_stream.sh &&chmod 777 failover_stream.sh
在master和salve1上創建日志文件:
mkdir /var/log/pgpool chown -R postgres.postgres /var/log/pgpool mkdir /var/run/pgpool chown -R postgres.postgres /var/run/pgpool 配置host：
在master、slave1、slave2的/etc/hosts配置ip映射
192.168.123.180 master 192.168.123.181 slave1 192.168.123.182 slave2 在master上創建repusr賬號:
su postgres
psql
create user repuser with password 'repuser'; 防火牆開放：5432 9999 9898 9000 9694
4.5、啟動pgpool
先在master上啟動pgpool：
pgpool -n -d -D > /var/log/pgpool/pgpool.log 2>&1 &
再在slave1上啟動pgpool:
pgpool -n -d -D > /var/log/pgpool/pgpool.log 2>&1 &
Master上的ifconfig會多一個虛擬ip出來
快速停止pgpool的指令:
pgpool -m fast stop
連接虛擬ip進入數據庫:
Su postgres psql -h 192.168.123.183 -p 9999 查看集群節點狀態：
show pool_nodes;
寫在安裝后
寫文檔才發現，邊安裝邊趟坑邊記錄，還要注意格式是多么愉（xin）快（ku）的事，本着開源精神，貼出來給要用到pg集群的小伙伴們參考，本文侵刪