kafka之partition分區及副本replica升級

修改kafka的partition分區

bin/kafka-topics.sh --zookeeper datacollect-2:2181 --alter --partitions 3 --topic client-agent-1 

修改kafka副本數

官網解釋如下：

Increasing replication factor

Increasing the replication factor of an existing partition is easy. Just specify the extra replicas in the custom reassignment json file and use it with the --execute option to increase the replication factor of the specified partitions.
For instance, the following example increases the replication factor of partition 0 of topic foo from 1 to 3. Before increasing the replication factor, the partition's only replica existed on broker 5. As part of increasing the replication factor, we will add more replicas on brokers 6 and 7.

The first step is to hand craft the custom reassignment plan in a json file:


> cat increase-replication-factor.json
{"version":1,
"partitions":[{"topic":"foo","partition":0,"replicas":[5,6,7]}]}

 

准備json文件：

此文件結構沒錯，partition不夠，需要根據實際情況完善此json文件。
{
    "partitions":
        [
        {
            "topic": "speech-3",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "client-agent-3",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "session-manager-3",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "speech-2",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "client-agent-2",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "session-manager-2",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "speech-1",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "client-agent-1",
            "partition": 2,
            "replicas": [0,2,3]
        },
        {
            "topic": "session-manager-1",
            "partition": 2,
            "replicas": [0,2,3]
        }
        ],
    "version":1
}

 

開始執行副本升級：

bin/kafka-reassign-partitions.sh --zookeeper datacollect-2:2181 --reassignment-json-file json.json --execute

 

驗證是否完成：

　　

bin/kafka-reassign-partitions.sh --zookeeper datacollect-2:2181 --reassignment-json-file json.json --verify

 

參考：

　　kafka修改分區和副本數 http://blog.csdn.net/haifeng000haifeng/article/details/50914197

 　  kafka運維--增加topic備份因子 http://blog.csdn.net/yanshu2012/article/details/53761284

　　kafka遷移與擴容 http://liyonghui160com.iteye.com/blog/2193430

　　Kafka 消息不能接收的問題 http://blog.csdn.net/lg772ef/article/details/65633359