Kafka动态调整topic分区partition

雨点打透心脏的1/2处 2022-05-06 06:57 397阅读 0赞

我们在使用kafka时，初期创建时所指定topic属性需要修改，如何动态修改kafka属性？kafka提供了命令行工具—kafka-topics.sh.

*  **kafka-topics.sh工具介绍**

kafka-topics.sh工具也是我们用来创建topic、查看topic详情的工具。  
直接运行kafka-topics.sh可以产出它是用来创建、删除、查看以及更新topic  
root@ubuntu:/opt/kafka\_2.11-1.1.0/bin\# ./kafka-topics.sh  
Create, delete, describe, or change a topic.  
Option Description

--------------------

–alter Alter the number of partitions,  
replica assignment, and/or  
configuration for the topic.  
–config <String: name=value> A topic configuration override for  
…

*  **更新或者修改topic**  
    注意：我的kafka版本是1.1.0， 并且我只有一个broker。  
    1， 首先我们创建一个topic，然后查看详情  
    root@ubuntu:/opt/kafka\_2.11-1.1.0/bin\# ./kafka-topics.sh --create --zookeeper 192.168.119.131:2181 --replication-factor 1 --partitions 4 --topic yqtopic1  
    Created topic “yqtopic1”.  
    root@ubuntu:/opt/kafka\_2.11-1.1.0/bin\#

root@ubuntu:/opt/kafka\_2.11-1.1.0/bin\# ./kafka-topics.sh --describe --zookeeper 192.168.119.131:2181 --topic yqtopic1  
Topic:yqtopic1 PartitionCount:4 ReplicationFactor:1 Configs:  
Topic: yqtopic1 Partition: 0 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 1 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 2 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 3 Leader: 0 Replicas: 0 Isr: 0  
root@ubuntu:/opt/kafka\_2.11-1.1.0/bin\#  
![在这里插入图片描述][70]

2，修改刚创建的topic，并查看修改的情况  
将分区数有4修改为12  
root@ubuntu:/opt/kafka\_2.11-1.1.0/bin\# ./kafka-topics.sh --alter --zookeeper 192.168.119.131:2181 --topic yqtopic1 --partitions 12  
WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected  
Adding partitions succeeded!  
root@ubuntu:/opt/kafka\_2.11-1.1.0/bin\#

root@ubuntu:/opt/kafka\_2.11-1.1.0/bin\# ./kafka-topics.sh --describe --zookeeper 192.168.119.131:2181 --topic yqtopic1 Topic:yqtopic1 PartitionCount:12 ReplicationFactor:1 Configs:  
Topic: yqtopic1 Partition: 0 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 1 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 2 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 3 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 4 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 5 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 6 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 7 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 8 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 9 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 10 Leader: 0 Replicas: 0 Isr: 0  
Topic: yqtopic1 Partition: 11 Leader: 0 Replicas: 0 Isr: 0  
root@ubuntu:/opt/kafka\_2.11-1.1.0/bin\# ls -al /tmp/kafka-logs/  
total 72  
drwxr-xr-x 14 root root 4096 Oct 13 14:34 .  
drwxrwxrwt 17 root root 4096 Oct 13 14:34 …  
\-rw-r–r-- 1 root root 0 Oct 13 14:10 cleaner-offset-checkpoint  
\-rw-r–r-- 1 root root 0 Oct 13 14:10 .lock  
\-rw-r–r-- 1 root root 4 Oct 13 14:33 log-start-offset-checkpoint  
\-rw-r–r-- 1 root root 54 Oct 13 14:10 meta.properties  
\-rw-r–r-- 1 root root 163 Oct 13 14:33 recovery-point-offset-checkpoint  
\-rw-r–r-- 1 root root 163 Oct 13 14:34 replication-offset-checkpoint  
drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-0  
drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-1  
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-10  
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-11  
drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-2  
drwxr-xr-x 2 root root 4096 Oct 13 **14:20** yqtopic1-3  
drwxr-xr-x 2 root root 4096 Oct 13 **14:33** yqtopic1-4  
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-5  
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-6  
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-7  
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-8  
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-9  
root@ubuntu:/opt/kafka\_2.11-1.1.0/bin\#

修改后的截图如下  
![在这里插入图片描述][70 1]

[70]: /images/20220506/df09e69e3b86481c98c748aa2cf93d9c.png
[70 1]: /images/20220506/838464e9a0bd4c1fa178ca04756bb743.png