[原]Greenplum failed segment的恢復方法


  當在使用greenplum過程中有不當的操作時,可能會出現segment節點宕掉的情況(比如在greenplum運行的過程中停掉其中幾台segment節點的服務器),通過下面的方法可以恢復segment。

下面是現場出現的故障情況:

[gpadmin@tj-soc-c04-csfb1 ~]$ gpstate -m
20161010:16:35:54:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:-Starting gpstate with args: -m
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.6.2 build 1'
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.6.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Nov 12 2015 23:50:28'
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:-Obtaining Segment details from master...
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:--------------------------------------------------------------
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:--Current GPDB mirror list and status
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:--Type = Group
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:--------------------------------------------------------------
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:-   Mirror             Datadir                       Port    Status              Data Status       
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-tj-soc-c04-csfb2   /data1/gpdata/mirror/gpseg0   41000   Failed                                <<<<<<<<
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-tj-soc-c04-csfb2   /data1/gpdata/mirror/gpseg1   41001   Failed                                <<<<<<<<
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-tj-soc-c04-csfb3   /data1/gpdata/mirror/gpseg2   41000   Failed                                <<<<<<<<
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-tj-soc-c04-csfb3   /data1/gpdata/mirror/gpseg3   41001   Failed                                <<<<<<<<
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:-   tj-soc-c04-csfb4   /data1/gpdata/mirror/gpseg4   41000   Acting as Primary   Change Tracking
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:-   tj-soc-c04-csfb4   /data1/gpdata/mirror/gpseg5   41001   Acting as Primary   Change Tracking
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-tj-soc-c04-csfb1   /data1/gpdata/mirror/gpseg6   41000   Failed                                <<<<<<<<
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-tj-soc-c04-csfb1   /data1/gpdata/mirror/gpseg7   41001   Failed                                <<<<<<<<
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[INFO]:--------------------------------------------------------------
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-2 segment(s) configured as mirror(s) are acting as primaries
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-6 segment(s) configured as mirror(s) have failed
20161010:16:35:55:026100 gpstate:tj-soc-c04-csfb1:gpadmin-[WARNING]:-2 mirror segment(s) acting as primaries are in change tracking

可以看到有6個節點Failed,有2個節點的Primary和Mirror交換了。

一、首先需要停掉GP

gpstop -M fast -a    這樣會告訴你有幾個節點DOWN了

 

二、啟動GP

gpstart    啟動數據庫會忽略DOWN的節點

 

三、生成一個恢復配置文件

gprecoverseg -o ./recov    會在當前目錄生成一個recov文件,里面包含了要恢復的節點信息

recov文件內容如下:(注意:這個文件不是手動創建的,而是通過gprecoverseg -o ./recov命令生成的

filespaceOrder=
tj-soc-c04-csfb2:41000:/data1/gpdata/mirror/gpseg0 tj-soc-c04-csfb2:41001:/data1/gpdata/mirror/gpseg1 tj-soc-c04-csfb3:41000:/data1/gpdata/mirror/gpseg2 tj-soc-c04-csfb3:41001:/data1/gpdata/mirror/gpseg3 tj-soc-c04-csfb1:41000:/data1/gpdata/mirror/gpseg6 tj-soc-c04-csfb1:41001:/data1/gpdata/mirror/gpseg7

 

四、使用恢復配置文件恢復節點

$gprecoverseg -i ./recov

恢復過程中可以用gpstate -m 查看恢復狀態:Resynchronizing(表示正在恢復中),Synchronized(表示恢復完畢)

 

五、調整Primary和Mirror

上面的情況中有Primary和Mirror兌換的情況,所以需要把他們換回來,可以用下面的命令

gprecoverseg -r

等待所有的節點都是Synchronized后,segment就恢復好了

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM