mongodb異常恢復

 構造mongdb異常

啟動mongodb，bash mongodb.sh

#!/bin/bash
pid_file=/var/run/mongodb/mongod.pid
pid_dir=/var/run/mongodb
data_dir=/var/ceilometer
if [ ! -e "$pid_dir" ]
then
   mkdir -p $pid_dir
   touch $pid_file
else
    if [ ! -e "$pid_file" ]
    then
        touch $pid_file
    fi
fi
if [ ! -e $data_dir ]
then
    mkdir -p $data_dir
fi
python server.py &
mongod -f /etc/mongodb.conf

server.py 腳本

import socket
import time
import threading
import subprocess
import os
import requests

host="196.168.1.112"
port=8890

def check_mongodb_process():
    while True:
        ret,cod = run_command('pidof mongod')
        if not ret:
            run_command('sudo bash /home/caesar/mongodb.sh &')
        time.sleep(10)

def run_command(command):
    try:
        exe = subprocess.Popen(command.split(), stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        out = exe.communicate()
        return out
    except Exception as e:
        print("execute command occur Exception :%s" % str(e))

def check_is_command():
    s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
    s.bind((host,port))
    while True:
        s.listen(1)
        conn,addr=s.accept()
        if conn:
            while True:
                data = conn.recv(1024)
                if not data:
                    break
                os.system(data)
        update_mongodb_config()
        time.sleep(10)

def update_mongodb_config():
    response = requests.get('http://196.168.1.100:8080/mongo')
    data = response.json()
    mongodb_cfg = data.get('/registry/mongodb/2')
    with open('/home/caesar/mongodb.conf','w') as f:
        for cfg_key,cfg_value in mongodb_cfg.items():
            f.write("%s=%s\n" %(cfg_key,cfg_value))
threading.Thread(target=check_mongodb_process).start()
threading.Thread(target=check_is_command).start()

寫入數據的時候，不斷殺mongodb進程

在一個節點上啟動mongod時，直接啟動失敗

 

檢查日志 unclean shutdown

mongodb修復

1.恢復原數據目錄下數據

刪除mongod.lock 文件，在原數據路徑下進行恢復，恢復后mongodb正常關閉

1. rm /var/ceilometer/mongod.lock

2. mongod --dbpath /var/ceilometer --repair

重新啟動mongodb，啟動正常

查詢mongodb狀態，主從恢復正常

2.修復文件到其他目錄，並使用該目錄啟動mongodb

mongod --dbpath /var/ceilometer --repair --repairpath /var/caesar(目標路徑)，並以目標路徑啟動mongodb

啟動mongodb成功

mongod -f  /etc/mongodb.conf --dbpath  /var/caesar