Ceph Luminous手動解決pg分布不均衡問題

原文鏈接：

https://www.jianshu.com/p/afb6277dbfd6

1.設置集群僅支持 Luminous（或者L之后的）客戶端

具體命令：
ceph osd set-require-min-compat-client luminous
或
ceph osd set-require-min-compat-client luminous --yes-i-really-mean-it

[root@controller ~]# ceph features
{
    "mon": [
        {
            "features": "0x3ffddff8ffacffff",
            "release": "luminous",
            "num": 1
        }
    ],
    "osd": [
        {
            "features": "0x3ffddff8ffacffff",
            "release": "luminous",
            "num": 14
        }
    ],
    "client": [
        {
            "features": "0x3ffddff8ffacffff",
            "release": "luminous",
            "num": 20
        }
    ],
    "mgr": [
        {
            "features": "0x3ffddff8ffacffff",
            "release": "luminous",
            "num": 1
        }
    ]
}

2.獲取集群當前的osdmap信息

命令
ceph osd getmap {osdmap_filename}

[root@controller ~]# ceph osd getmap -o osd.map
got osdmap epoch 52619

3.獲取當前集群數據均衡后的優化信息

命令
osdmaptool {osdmap_filename} --upmap out.txt [--upmap-pool <pool>] [--upmap-max <max-count>] [--upmap-deviation <max-deviation>]
其中

• upmap-pool ：指定需要優化均衡的存儲池名

• upmap-max： 指定一次優化的數據條目，默認100，可根據環境業務情況調整該值，一次調整的條目越多，數據遷移會越多，可能對環境業務造成影響。

• max-deviation：最大偏差值，默認為0.01（即1％）。如果OSD利用率與平均值之間的差異小於此值，則將被視為完美。

假設只優化2條，即：

[root@controller ~]# osdmaptool osd.map --upmap out.txt --upmap-pool volumes --upmap-max=2 
osdmaptool: osdmap file 'osd.map'
writing upmap command output to: out.txt
checking for upmap cleanups
upmap, max-count 2, max deviation 0.01
 limiting to pools volumes (2)

[root@controller ~]# cat out.txt 
ceph osd pg-upmap-items 2.2de 10 5
ceph osd pg-upmap-items 2.2e1 8 5

從上述數據結果中
ceph osd pg-upmap-items 2.2de 10 5
表明pg 12.2de需從osd.10重新映射到osd.5

4.均衡

[root@controller ~]# set 2.2de  pg_upmap_items mapping to [10->5]   
[root@controller ~]# set 2.2d1  pg_upmap_items mapping to [8->5]