Kafka：退役旧（减少）节点

淩亂°似流年 2024-03-24 15:13 29阅读 0赞

**1）执行负载均衡操作（原本有4个节点，关闭broker.id为3的节点即为hadoop105 节点）**

先按照退役一台节点，生成执行计划，然后按照服役时操作流程执行负载均衡。

**（选择任意一个节点操作）**

（1）创建一个要均衡的主题。

[root@hadoop102 kafka]$ vim topics-to-move.json
    {
     "topics": [
     {"topic": "first"}
     ],
     "version": 1
    }

（2）创建执行计划。

[root@hadoop102 kafka]$ bin/kafka-reassign-partitions.sh --bootstrap-server hadoop102:9092 --topics-to-move-json-file topics-to-move.json --broker-list "0,1,2" --generate
    
    Current partition replica assignment
    {"version":1,"partitions":[{"topic":"first","partition":0,"replicas":[2,0,1],"log_dirs":["any","any","any"]},{"topic":"first","partition":1,"replicas":[3,1,2],"log_dirs":["any","any","any"]},{"topic":"first","partition":2,"replicas":[0,2,3],"log_dirs":["any","any","any"]}]}
    
    # 以下计划等会用于拷贝
    Proposed partition reassignment configuration
    {"version":1,"partitions":[{"topic":"first","partition":0,"replicas":[2,0,1],"log_dirs":["any","any","any"]},{"topic":"first","partition":1,"replicas":[0,1,2],"log_dirs":["any","any","any"]},{"topic":"first","partition":2,"replicas":[1,2,0],"log_dirs":["any","any","any"]}]}

（3）创建副本存储计划（所有副本存储在 broker0、broker1、broker2 中）。

[root@hadoop102 kafka]$ vim increase-replication-factor.json
    
    # 下面内容，就是上面的Proposed partition reassignment configuration下面的内容
    {"version":1,"partitions":[{"topic":"first","partition":0,"replicas":[2,0,1],"log_dirs":["any","any","any"]},{"topic":"first","partition":1,"replicas":[0,1,2],"log_dirs":["any","any","any"]},{"topic":"first","partition":2,"replicas":[1,2,0],"log_dirs":["any","any","any"]}]}

（4）执行副本存储计划。

[root@hadoop102 kafka]$ bin/kafka-reassign-partitions.sh --bootstrap-server hadoop102:9092 --reassignment-json-file increase-replication-factor.json --execute

（5）验证副本存储计划。

[root@hadoop102 kafka]$ bin/kafka-reassign-partitions.sh --bootstrap-server hadoop102:9092 --reassignment-json-file increase-replication-factor.json --verify
    
    Status of partition reassignment:
    Reassignment of partition first-0 is complete.
    Reassignment of partition first-1 is complete.
    Reassignment of partition first-2 is complete.
    
    Clearing broker-level throttles on brokers 0,1,2,3
    Clearing topic-level throttles on topic first

**2）执行停止命令**

在 hadoop105 上执行停止命令即可。

[root@hadoop105 kafka]$ bin/kafka-server-stop.sh