（一）kafka修改topic分區的位置

（一）kafka修改topic分區的位置

環境：kafka_2.10-0.8.2.1 + JDK1.7.0_80

1. 查看分區topic的分區分布

$ le-kafka-topics.sh --describe --topic http_zhixin_line1

結果如下：

[hadoop@sdf-nimbus-perf project]$ le-kafka-topics.sh --describe --topic http_zhixin_line1
Topic:http_zhixin_line1	PartitionCount:3	ReplicationFactor:1	Configs:
	Topic: http_zhixin_line1	Partition: 0	Leader: 4	Replicas: 4	Isr: 4
	Topic: http_zhixin_line1	Partition: 1	Leader: 5	Replicas: 5	Isr: 5
	Topic: http_zhixin_line1	Partition: 2	Leader: 6	Replicas: 6	Isr: 6

2. 准備分區分配的JSON配置文件

可以使用kafka的kafka-reassign-partitions.sh工具來分配topic的分區位置，操作之前得准備一個Json文件，里面詳細地記錄着每個分區的副本所在機器等信息，具體如下：

{
    "version": 1,
    "partitions": [
        {
            "topic": "http_zhixin_line1",
            "partition": 0,
            "replicas": [
                7
            ]
        },
        {
            "topic": "http_zhixin_line1",
            "partition": 1,
            "replicas": [
                8
            ]
        },
        {
            "topic": "http_zhixin_line1",
            "partition": 3,
            "replicas": [
                9
            ]
        }
    ]
}

3. 執行topic的分區位置修改操作：

kafka-reassign-partitions.sh --zookeeper $ZK_CONNECT --reassignment-json-file movePartitions.json --execute
執行日志如下：

[hadoop@sdf-nimbus-perf topic_reassgin]$ kafka-reassign-partitions.sh --zookeeper $ZK_CONNECT --reassignment-json-file movePartitions.json --execute
Current partition replica assignment

{"version":1,"partitions":[{"topic":"http_zhixin_line1","partition":2,"replicas":[6]},{"topic":"http_zhixin_line1","partition":1,"replicas":[5]},{"topic":"http_zhixin_line1","partition":0,"replicas":[4]}]}

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions {"version":1,"partitions":[{"topic":"http_zhixin_line1","partition":0,"replicas":[7]},{"topic":"http_zhixin_line1","partition":1,"replicas":[8]},{"topic":"http_zhixin_line1","partition":2,"replicas":[9]}]}

4. 檢查執行的結果

kafka-reassign-partitions.sh --zookeeper $ZK_CONNECT --reassignment-json-file movePartitions.json --verify
執行日志如下：

[hadoop@sdf-nimbus-perf topic_reassgin]$ kafka-reassign-partitions.sh --zookeeper $ZK_CONNECT --reassignment-json-file movePartitions.json --verify
Status of partition reassignment:
Reassignment of partition [http_zhixin_line1,0] completed successfully
Reassignment of partition [http_zhixin_line1,1] completed successfully
Reassignment of partition [http_zhixin_line1,2] completed successfully

5. 檢查kafka的對應的topic的日志目錄

原目錄下的數據已經遷移到新的的分區所對應的服務器下了。

分配狀態的變化：分配的狀態變化：“is still in progress” --》 “completed successfully”
如果分配腳本有錯，或者遇到異常，分配狀態會變成“failed” （具體的fail原因：如果是json配置文件的原因，命令行就會打印出error日志；kafka內部錯誤可以查看kafka的server.log 日志文件等）。

重新分配過程中數據日志文件的變化：
先將原來的分區的數據，copy到對應重新分配后的分區節點下，拷貝成功后，kakfa會將原來的分區目錄刪除掉。 copy 前后的數據量大小是一致的。