場景
線上很多 kafka 的 topic 的副本數為1,這樣的設置喪失了 kafka 高可用的特性,所以我們需要把 topic 的副本數至少設置為2(當前3台 kafka)
我們使用 kafkat 這個工具,這個工具是 ruby 寫的,所以我們先安裝 ruby 環境
參考 https://ruby-china.org/wiki/rvm-guide
這時候你應該已經有了 gem 命令
繼續參考https://github.com/airbnb/kafkat 把 kafkat 裝上, 並配置好你的 kafka 環境參數到 ~/.kafkatcfg
實操
我們修改 topic 的 replicas 為2
[root
@
001
kafka]# kafkat partitions IM_APP_BINSPECT_MSG_TOPIC
Topic Partition Leader Replicas ISRs
IM_APP_BINSPECT_MSG_TOPIC
0
2
[
2
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
1
3
[
3
] [
3
]
IM_APP_BINSPECT_MSG_TOPIC
2
1
[
1
] [
1
]
IM_APP_BINSPECT_MSG_TOPIC
3
2
[
2
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
4
3
[
3
] [
3
]
IM_APP_BINSPECT_MSG_TOPIC
5
1
[
1
] [
1
]
IM_APP_BINSPECT_MSG_TOPIC
6
2
[
2
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
7
3
[
3
] [
3
]
IM_APP_BINSPECT_MSG_TOPIC
8
1
[
1
] [
1
]
IM_APP_BINSPECT_MSG_TOPIC
9
2
[
2
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
10
3
[
3
] [
3
]
IM_APP_BINSPECT_MSG_TOPIC
11
1
[
1
] [
1
]
IM_APP_BINSPECT_MSG_TOPIC
12
2
[
2
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
13
3
[
3
] [
3
]
IM_APP_BINSPECT_MSG_TOPIC
14
1
[
1
] [
1
]
IM_APP_BINSPECT_MSG_TOPIC
15
2
[
2
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
16
3
[
3
] [
3
]
IM_APP_BINSPECT_MSG_TOPIC
17
1
[
1
] [
1
]
IM_APP_BINSPECT_MSG_TOPIC
18
2
[
2
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
19
3
[
3
] [
3
]
[root
@
001
kafka]# kafkat set-replication-factor IM_APP_BINSPECT_MSG_TOPIC --newrf
2
--brokers
1
,
2
,
3
This operation executes the following assignments:
Topic Partition Replicas
IM_APP_BINSPECT_MSG_TOPIC
0
[
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
1
[
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
2
[
1
,
3
]
IM_APP_BINSPECT_MSG_TOPIC
3
[
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
4
[
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
5
[
1
,
3
]
IM_APP_BINSPECT_MSG_TOPIC
6
[
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
7
[
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
8
[
1
,
3
]
IM_APP_BINSPECT_MSG_TOPIC
9
[
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
10
[
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
11
[
1
,
3
]
IM_APP_BINSPECT_MSG_TOPIC
12
[
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
13
[
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
14
[
1
,
3
]
IM_APP_BINSPECT_MSG_TOPIC
15
[
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
16
[
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
17
[
1
,
3
]
IM_APP_BINSPECT_MSG_TOPIC
18
[
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
19
[
3
,
2
]
Proceed (y/n)?
y
Beginning.
Started.
[root
@
001
kafka]# kafkat partitions IM_APP_BINSPECT_MSG_TOPIC
Topic Partition Leader Replicas ISRs
IM_APP_BINSPECT_MSG_TOPIC
0
2
[
2
,
1
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
1
3
[
3
,
2
] [
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
2
1
[
1
,
3
] [
1
]
IM_APP_BINSPECT_MSG_TOPIC
3
2
[
2
,
1
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
4
3
[
3
,
2
] [
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
5
1
[
1
,
3
] [
1
]
IM_APP_BINSPECT_MSG_TOPIC
6
2
[
2
,
1
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
7
3
[
3
,
2
] [
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
8
1
[
1
,
3
] [
1
]
IM_APP_BINSPECT_MSG_TOPIC
9
2
[
2
,
1
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
10
3
[
3
,
2
] [
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
11
1
[
1
,
3
] [
1
]
IM_APP_BINSPECT_MSG_TOPIC
12
2
[
2
,
1
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
13
3
[
3
,
2
] [
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
14
1
[
1
,
3
] [
1
]
IM_APP_BINSPECT_MSG_TOPIC
15
2
[
2
,
1
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
16
3
[
3
,
2
] [
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
17
1
[
1
,
3
] [
1
]
IM_APP_BINSPECT_MSG_TOPIC
18
2
[
2
,
1
] [
2
]
IM_APP_BINSPECT_MSG_TOPIC
19
3
[
3
,
2
] [
3
]
|
解釋
上面一共執行了3個命令
- kafkat partitions IM_APP_BINSPECT_MSG_TOPIC 查看 topic 的分區情況
- kafkat set-replication-factor IM_APP_BINSPECT_MSG_TOPIC --newrf 2 --brokers 1,2,3 重新設置副本數為2 --newrf 2 , 你有幾個 broker就在--brokers 后面指定
- kafkat partitions IM_APP_BINSPECT_MSG_TOPIC 檢查是否生效
根據 topic 數據量最后完成的時間也不同(大概幾分鍾)
這是最后的完成情況
[root
@
001
kafka]# kafkat partitions IM_APP_BINSPECT_MSG_TOPIC
Topic Partition Leader Replicas ISRs
IM_APP_BINSPECT_MSG_TOPIC
0
2
[
2
,
1
] [
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
1
3
[
3
,
2
] [
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
2
1
[
1
,
3
] [
1
,
3
]
IM_APP_BINSPECT_MSG_TOPIC
3
2
[
2
,
1
] [
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
4
3
[
3
,
2
] [
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
5
1
[
1
,
3
] [
1
,
3
]
IM_APP_BINSPECT_MSG_TOPIC
6
2
[
2
,
1
] [
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
7
3
[
3
,
2
] [
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
8
1
[
1
,
3
] [
1
,
3
]
IM_APP_BINSPECT_MSG_TOPIC
9
2
[
2
,
1
] [
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
10
3
[
3
,
2
] [
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
11
1
[
1
,
3
] [
1
,
3
]
IM_APP_BINSPECT_MSG_TOPIC
12
2
[
2
,
1
] [
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
13
3
[
3
,
2
] [
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
14
1
[
1
,
3
] [
1
,
3
]
IM_APP_BINSPECT_MSG_TOPIC
15
2
[
2
,
1
] [
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
16
3
[
3
,
2
] [
3
,
2
]
IM_APP_BINSPECT_MSG_TOPIC
17
1
[
1
,
3
] [
1
,
3
]
IM_APP_BINSPECT_MSG_TOPIC
18
2
[
2
,
1
] [
2
,
1
]
IM_APP_BINSPECT_MSG_TOPIC
19
3
[
3
,
2
] [
3
,
2
]
|