[原]greenplum安裝詳細過程

　　今天又幫其他項目裝了一遍GP，加上之前的兩次，這是第三次了，雖然每次都有記錄，但這次安裝還是發現漏寫了一些步驟，在此詳細記錄一下，需要的童鞋可以借鑒。

1、准備

　　這里准備了4台服務器，1台做master，1台做standby，4台都做存儲，為了保密真實的IP地址和主機名都換成“藝名”了。

OS:Red Hat Enterprise Linux Server release 6.4 (Santiago) 
GP:greenplum-db-4.3.6.2-build-1-RHEL5-x86_64.zip
JDK:1.7+

192.168.1.100　　bj-gp-node1(segment master)
192.168.1.101　　bj-gp-node2(segment standby)
192.168.1.102　　bj-gp-node3(segment)
192.168.1.103　　bj-gp-node4(segment)

greenplum官網下載地址：http://gpn.greenplum.com/download.php （注：需要注冊后才能下載）

 

2、系統參數配置

　　系統參數配置的修改需要在每個節點服務器上執行

2.1.修改Linux內核參數

[root@bj-gp-node1 ~]# vi /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
# sysctl.conf(5) for more details.
#
# Use '/sbin/sysctl -a' to list all possible parameters.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 1

# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

# Disable netfilter on bridges
#net.bridge.bridge-nf-call-ip6tables = 0
#net.bridge.bridge-nf-call-iptables = 0
#net.bridge.bridge-nf-call-arptables = 0

# Controls the default maxmimum size of a mesage queue
kernel.msgmnb = 65536

# Controls the maximum size of a message, in bytes
kernel.msgmax = 65536
kernel.msgmni = 2048

kernel.sem = 250 512000 100 2048

# Controls the maximum shared segment size, in bytes
#kernel.shmmax = 68719476736
kernel.shmmax = 500000000
kernel.shmmni = 4096

# Controls the maximum number of shared memory segments, in pages
#kernel.shmall = 4294967296
kernel.shmall = 4000000000

net.ipv4.tcp_tw_recycle=1
net.ipv4.tcp_max_syn_backlog=4096
net.ipv4.ip_local_port_range = 1025 65535
net.core.netdev_max_backlog=10000
vm.overcommit_memory=2
net.ipv4.conf.all.arp_filter = 1
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152

 

2.2.修改Linux最大限制

[root@bj-gp-node1 ~]# vi /etc/security/limits.conf
#greenplum configs
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072 
* hard nproc 131072

 

2.3.I/O調整優化

[root@bj-gp-node1 ~]# vi /boot/grub/menu.lst
#greenplum configs
elevator=deadline

 

2.4.添加所有節點到HOST

[root@bj-gp-node1 ~]# vi /etc/hosts

192.168.1.100　　bj-gp-node1
192.168.1.101　　bj-gp-node2
192.168.1.102　　bj-gp-node3
192.168.1.103　　bj-gp-node4

 

2.5.關閉防火牆

[root@bj-gp-node1 ~]# chkconfig --list iptables
[root@bj-gp-node1 ~]# chkconfig --level 0123456 iptables off

 

2.6.設置SELINUX

[root@bj-gp-node1 ~]# vi /etc/selinux/config
SELINUX=disabled

 

2.7.重啟系統使配置生效

[root@bj-gp-node1 ~]# reboot

 

3、GP安裝

　　GP的安裝操作都是在主節點master上執行的

3.1.創建gpadmin用戶

[root@bj-gp-node1 ~]# useradd gpadmin
[root@bj-gp-node1 ~]# passwd gpadmin

 

3.2.設置gpadmin用戶環境

[gpadmin@bj-gp-node1 ~]$ cd /home/gpadmin
[gpadmin@bj-gp-node1 ~]$ vi .bashrc

[gpadmin@bj-gp-node1 ~]$ vi .bash_profile

.bashrc和.bash_profile最后都添加下面兩行

source /usr/local/greenplum-db/greenplum_path.sh 
export MASTER_DATA_DIRECTORY=/data1/gpdata/master/gpseg-1

設置完后記得source一下使其立即生效

 

3.3.上傳並解壓安裝包

將greenplum-db-4.3.6.2-build-1-RHEL5-x86_64.zip上傳至master的/opt/目錄下

[root@bj-gp-node1 opt]# /bin/bash greenplum-db-4.3.6.2-build-1-RHEL5-x86_64.bin

按提示輸入回車或yes 這一步會將安裝包解壓到/usr/local/下，並建立軟連接greenplum-db

 

3.4.准備節點服務器信息文件

后面的批量安裝會用到這兩個文件，如果all_host和all_segment內容一樣，可以只創建一個文件

[root@bj-gp-node1 opt]# mkdir -p /opt/gpinit/
[root@bj-gp-node1 gpinit]# touch all_host
[root@bj-gp-node1 gpinit]# touch all_segment
all_host和all_segment內容：
bj-gp-node1
bj-gp-node2
bj-gp-node3
bj-gp-node4

 

3.5.建立節點服務器間的信任

[root@bj-gp-node1 local]# gpssh-exkeys -f /opt/gpinit/all_host

按照提示輸入root密碼，記住這一步不能輸入gpadmin的密碼，因為批量安裝時需要在/usr/local下創建目錄，需要root權限

 

3.6.批量安裝

[root@bj-gp-node1 local]# gpseginstall -f /opt/gpinit/all_host -u gpadmin -p gpadmin

這一步其實就是將master上的greenplum打包通過scp命令傳到all_host中的主機上，並賦予目錄gpadmin的權限

 

3.7.檢查批量安裝情況

[root@bj-gp-node1 local]# gpssh -f /opt/gpinit/all_host -e ls -l $GPHOME

返回結果中各節點目錄一致則成功

 

3.8.創建存儲目錄

master
[root@bj-gp-node1 local]# mkdir -p /data1/gpdata/master
[root@bj-gp-node1 local]# chown gpadmin:gpadmin /data1/gpdata/master

segment
[root@bj-gp-node1 local]# gpssh -f /opt/gpinit/all_host -e 'mkdir -p /data1/gpdata/primary'
[root@bj-gp-node1 local]# gpssh -f /opt/gpinit/all_host -e 'chown gpadmin:gpadmin /data1/gpdata/primary'

mirror
[root@bj-gp-node1 local]# gpssh -f /opt/gpinit/all_segment -e 'mkdir -p /data1/gpdata/mirror'
[root@bj-gp-node1 local]# gpssh -f /opt/gpinit/all_segment -e 'chown gpadmin:gpadmin /data1/gpdata/mirror'

 

3.9.設置時鍾同步

vi /etc/ntp.conf 在server第一行添加下面兩行 
server 192.168.1.1
server 192.168.1.2
重啟ntpd服務 /etc/init.d/ntpd restart
查看ntp同步情況 ntpq -p
使ntpd服務重啟服務器后也啟動 chkconfig --level 0123456 ntpd on

 

3.10.創建GP初始化文件

[gpadmin@bj-gp-node1 ~]$ mkdir /home/gpadmin/gpconfigs
[gpadmin@bj-gp-node1 ~]$ cp /usr/local/greenplum-db/docs/cli_help/gpconfigs/gpinitsystem_config /home/gpadmin/gpconfigs/gpinitsystem_config
[gpadmin@tj-soc-c04-csfb1 gpconfigs]$ chmod 775 gpinitsystem_config

 

3.10修改GP初始化文件

[gpadmin@bj-gp-node1 gpconfigs]$ vi gpinitsystem_config 

# FILE NAME: gpinitsystem_config

# Configuration file needed by the gpinitsystem

################################################
#### REQUIRED PARAMETERS
################################################

#### Name of this Greenplum system enclosed in quotes.
ARRAY_NAME="BJ Greenplum DW"

#### Naming convention for utility-generated data directories.
SEG_PREFIX=gpseg

#### Base number by which primary segment port numbers
#### are calculated.
PORT_BASE=40000

#### File system location(s) where primary segment data directories
#### will be created. The number of locations in the list dictate
#### the number of primary segments that will get created per
#### physical host (if multiple addresses for a host are listed in
#### the hostfile, the number of segments will be spread evenly across
#### the specified interface addresses).
declare -a DATA_DIRECTORY=(/data1/gpdata/primary /data1/gpdata/primary)

#### OS-configured hostname or IP address of the master host.
MASTER_HOSTNAME=bj-gp-node1

#### File system location where the master data directory
#### will be created.
MASTER_DIRECTORY=/data1/gpdata/master

#### Port number for the master instance.
MASTER_PORT=5432

#### Shell utility used to connect to remote hosts.
TRUSTED_SHELL=ssh

#### Maximum log file segments between automatic WAL checkpoints.
CHECK_POINT_SEGMENTS=8

#### Default server-side character set encoding.
ENCODING=UNICODE

################################################
#### OPTIONAL MIRROR PARAMETERS
################################################

#### Base number by which mirror segment port numbers
#### are calculated.
MIRROR_PORT_BASE=50000

#### Base number by which primary file replication port
#### numbers are calculated.
REPLICATION_PORT_BASE=41000

#### Base number by which mirror file replication port
#### numbers are calculated.
MIRROR_REPLICATION_PORT_BASE=51000

#### File system location(s) where mirror segment data directories
#### will be created. The number of mirror locations must equal the
#### number of primary locations as specified in the
#### DATA_DIRECTORY parameter.
declare -a MIRROR_DATA_DIRECTORY=(/data1/gpdata/mirror /data1/gpdata/mirror)

################################################
#### OTHER OPTIONAL PARAMETERS
################################################

#### Create a database of this name after initialization.
DATABASE_NAME=bj_gp

#### Specify the location of the host address file here instead of
#### with the the -h option of gpinitsystem.
#MACHINE_LIST_FILE=/home/gpadmin/gpconfigs/hostfile_gpinitsystem

 

3.11.初始化GP

[gpadmin@bj-gp-node1 ~]$ gpinitsystem -c /home/gpadmin/gpconfigs/gpinitsystem_config -h /opt/gpinit/all_host
中間需要輸入一次：Y

 

如果沒有報ERROR，GP數據庫就安裝好了。下面講一些附加操作

 

4、附加項

4.1增加standby

一個gp集群只有一個master肯定會讓人不放心，還好有備用，當master宕掉后，會自動啟用standby作為master，下面來看一下standby怎么添加

在standby服務器上執行
[root@bj-gp-node2 ~]# mkdir /data1/gpdata/master
[root@bj-gp-node2 ~]# chown gpadmin:gpadmin /data1/gpdata/master

在master服務器上執行  
[gpadmin@bj-gp-node1 ~]$ gpinitstandby -s bj-gp-node2
中間輸入一次Y

 

4.2.增加mirror

mirror就是鏡像，也叫數據備份。mirror對於數據存儲來說很重要，因為我們的服務器指不定什么時候出毛病，有mirror就好很多了，因為兩台存儲節點同時宕掉的幾率還是很小的。如果前面在GP初始化文件里忘記配置mirror了，請按照下面的方法添加

[gpadmin@bj-gp-node1 ~]$ gpaddmirrors -p 1000
運行過程中需要輸入兩次mirror路徑：/data1/gpdata/mirror

 

4.3.設置訪問權限

打開/data1/gpdata/master/gpseg-1/pg_hba.conf 按照最下面的格式添加客戶端ip或網段

#user define
host    all     all     192.168.1.0/24   trust
host    all     all     127.0.0.1/28    trust

 

4.4.訪問方式

可以通過gpAdmin桌面客戶端來訪問，也可以用命令行來訪問，下面來說一下命令行訪問的方式，loach是后面添加的用戶

[gpadmin@bj-gp-node1 ~]$ psql -d tj_csfb -h bj-gp-node1 -p 5432 -U gpadmin
[gpadmin@bj-gp-node1 ~]$ psql -d tj_csfb -h bj-gp-node1 -p 5432 -U loach

 

4.5.創建用戶

通過命令行登錄以后，執行下面的命令

CREATE ROLE loach WITH LOGIN;
ALTER ROLE loach WITH PASSWORD 'loach';