Kafka修改分区、副本数量

Love The Way You Lie 2022-09-11 05:22 415阅读 0赞

### 文章目录 ###

*   *   *  一、概述
         *  二、修改分区
         *  三、修改副本
         *   *  1.指定Topic
             *  2.生成分配计划
             *  3.修改分配计划
             *  4.执行分配计划
             *  5.查看分配进度
             *  6.查看新的副本

### 一、概述 ###

> 无需重启服务

### 二、修改分区 ###

*  方式一：直接用软件重新分区
    
    Kafka Manager：
 *  方式二：命令行分区
    
    > –zookeeper已经过时，使用–bootstrap-server
    
        # 查看分区
        bin/kafka-topics.sh --describe --bootstrap-server 172.25.18.45:9092,172.25.18.45:9093,172.25.18.45:9094 --topic mc-topic
        
        # 修改分区
        bin/kafka-topics.sh --alter --bootstrap-server 172.25.18.45:9092,172.25.18.45:9093,172.25.18.45:9094 --topic mc-topic --partitions 3

### 三、修改副本 ###

#### 1.指定Topic ####

vi reassign_topic.json
    { "topics":
      [{ "topic":"__consumer_offsets"}],
      "version": 1
    }

#### 2.生成分配计划 ####

> –generate ： 生成分配计划，不执行

bin/kafka-reassign-partitions.sh --bootstrap-server 172.25.18.45:9092,172.25.18.45:9093,172.25.18.45:9094 --topics-to-move-json-file reassign_topic.json --broker-list "0,1,2" --generate
    # 输出：
    Current partition replica assignment # 当前分区配置
    { "version":1,"partitions":[{ "topic":"__consumer_offsets","partition":0,"replicas":[1,2,0]}]}
    ...
    Proposed partition reassignment configuration # 建议分区配置
    { "version":1,"partitions":[{ "topic":"__consumer_offsets","partition":0,"replicas":[0,2,1]}]}
    ...

#### 3.修改分配计划 ####

> 复制建议配置，去掉`,"log_dirs":["any","any","any"]`

vi replcas.json

内容：

{ "version":1,"partitions":[{ "topic":"__consumer_offsets","partition":0,"replicas":[2,1,0]},{ "topic":"__consumer_offsets","partition":1,"replicas":[0,2,1]},{ "topic":"__consumer_offsets","partition":2,"replicas":[1,0,2]},{ "topic":"__consumer_offsets","partition":3,"replicas":[2,0,1]},{ "topic":"__consumer_offsets","partition":4,"replicas":[0,1,2]},{ "topic":"__consumer_offsets","partition":5,"replicas":[1,2,0]},{ "topic":"__consumer_offsets","partition":6,"replicas":[2,1,0]},{ "topic":"__consumer_offsets","partition":7,"replicas":[0,2,1]},{ "topic":"__consumer_offsets","partition":8,"replicas":[1,0,2]},{ "topic":"__consumer_offsets","partition":9,"replicas":[2,0,1]},{ "topic":"__consumer_offsets","partition":10,"replicas":[0,1,2]},{ "topic":"__consumer_offsets","partition":11,"replicas":[1,2,0]},{ "topic":"__consumer_offsets","partition":12,"replicas":[2,1,0]},{ "topic":"__consumer_offsets","partition":13,"replicas":[0,2,1]},{ "topic":"__consumer_offsets","partition":14,"replicas":[1,0,2]},{ "topic":"__consumer_offsets","partition":15,"replicas":[2,0,1]},{ "topic":"__consumer_offsets","partition":16,"replicas":[0,1,2]},{ "topic":"__consumer_offsets","partition":17,"replicas":[1,2,0]},{ "topic":"__consumer_offsets","partition":18,"replicas":[2,1,0]},{ "topic":"__consumer_offsets","partition":19,"replicas":[0,2,1]},{ "topic":"__consumer_offsets","partition":20,"replicas":[1,0,2]},{ "topic":"__consumer_offsets","partition":21,"replicas":[2,0,1]},{ "topic":"__consumer_offsets","partition":22,"replicas":[0,1,2]},{ "topic":"__consumer_offsets","partition":23,"replicas":[1,2,0]},{ "topic":"__consumer_offsets","partition":24,"replicas":[2,1,0]},{ "topic":"__consumer_offsets","partition":25,"replicas":[0,2,1]},{ "topic":"__consumer_offsets","partition":26,"replicas":[1,0,2]},{ "topic":"__consumer_offsets","partition":27,"replicas":[2,0,1]},{ "topic":"__consumer_offsets","partition":28,"replicas":[0,1,2]},{ "topic":"__consumer_offsets","partition":29,"replicas":[1,2,0]},{ "topic":"__consumer_offsets","partition":30,"replicas":[2,1,0]},{ "topic":"__consumer_offsets","partition":31,"replicas":[0,2,1]},{ "topic":"__consumer_offsets","partition":32,"replicas":[1,0,2]},{ "topic":"__consumer_offsets","partition":33,"replicas":[2,0,1]},{ "topic":"__consumer_offsets","partition":34,"replicas":[0,1,2]},{ "topic":"__consumer_offsets","partition":35,"replicas":[1,2,0]},{ "topic":"__consumer_offsets","partition":36,"replicas":[2,1,0]},{ "topic":"__consumer_offsets","partition":37,"replicas":[0,2,1]},{ "topic":"__consumer_offsets","partition":38,"replicas":[1,0,2]},{ "topic":"__consumer_offsets","partition":39,"replicas":[2,0,1]},{ "topic":"__consumer_offsets","partition":40,"replicas":[0,1,2]},{ "topic":"__consumer_offsets","partition":41,"replicas":[1,2,0]},{ "topic":"__consumer_offsets","partition":42,"replicas":[2,1,0]},{ "topic":"__consumer_offsets","partition":43,"replicas":[0,2,1]},{ "topic":"__consumer_offsets","partition":44,"replicas":[1,0,2]},{ "topic":"__consumer_offsets","partition":45,"replicas":[2,0,1]},{ "topic":"__consumer_offsets","partition":46,"replicas":[0,1,2]},{ "topic":"__consumer_offsets","partition":47,"replicas":[1,2,0]},{ "topic":"__consumer_offsets","partition":48,"replicas":[2,1,0]},{ "topic":"__consumer_offsets","partition":49,"replicas":[0,2,1]}]}

#### 4.执行分配计划 ####

bin/kafka-reassign-partitions.sh --bootstrap-server 172.25.18.45:9092,172.25.18.45:9093,172.25.18.45:9094 --reassignment-json-file replcas.json --execute
    # Successfully started partition reassignments for __consumer_offsets-0,...

#### 5.查看分配进度 ####

bin/kafka-reassign-partitions.sh --bootstrap-server 172.25.18.45:9092,172.25.18.45:9093,172.25.18.45:9094 --reassignment-json-file replcas.json --verify

#### 6.查看新的副本 ####

bin/kafka-topics.sh --bootstrap-server 172.25.18.45:9092,172.25.18.45:9093,172.25.18.45:9094 --topic __consumer_offsets --describe