kafka学习 -- 服务端命令学习

短命女 2023-02-21 06:10 4阅读 0赞

### kafka学习 -- 服务端命令学习 ###

*   *  写在前面
     *  一、服务端所有脚本
     *  二、常用的脚本
     *   *  2.1、服务端启动配置
         *  2.2、broker 节点
         *  2.3、Topic
         *  2.4、producer
         *  2.5、consumer 和 group
         *  2.6、zookeeper
         *  2.7、性能测试
         *  2.8、日志查询
     *  三、更多Kafka,，客户端、服务端，监控可参考下文

## 写在前面 ##

这里 介绍一下 kafka\_2.11-1.0.2 版本服务端命令

## 一、服务端所有脚本 ##

connect-distributed.sh  
kafka-consumer-perf-test.sh  
kafka-replica-verification.sh  
kafka-verifiable-producer.sh

connect-standalone.sh  
kafka-delete-records.sh  
kafka-run-class.sh  
trogdor.sh

kafka-acls.sh  
kafka-log-dirs.sh  
kafka-server-start.sh  
windows  
kafka-broker-api-versions.sh  
kafka-mirror-maker.sh  
kafka-server-stop.sh  
zookeeper-security-migration.sh

kafka-configs.sh  
kafka-preferred-replica-election.sh  
kafka-simple-consumer-shell.sh  
zookeeper-server-start.sh

kafka-console-consumer.sh  
kafka-producer-perf-test.sh  
kafka-streams-application-reset.sh  
zookeeper-server-stop.sh

kafka-console-producer.sh  
kafka-reassign-partitions.sh  
kafka-topics.sh  
zookeeper-shell.sh

kafka-consumer-groups.sh  
kafka-replay-log-producer.sh  
kafka-verifiable-consumer.sh

## 二、常用的脚本 ##

### 2.1、服务端启动配置 ###

bin/zookeeper-server-start.sh config/zookeeper.properties
    
    bin/kafka-server-start.sh config/server.properties
    bin/kafka-server-start.sh config/server1.properties
    bin/kafka-server-start.sh config/server2.properties

### 2.2、broker 节点 ###

在这里插入代码片

### 2.3、Topic ###

bin/kafka-topics.sh --create --zookeeper 192.168.1.74:2181 --replication-factor 3 --partitions 1 --topic topic2
    bin/kafka-topics.sh  --delete --zookeeper 192.168.1.74:2181  --topic topic1
    
    bin/kafka-topics.sh --list --zookeeper 192.168.1.74:2181
    
    bin/kafka-topics.sh --describe --zookeeper 192.168.1.74:2181 --topic topic1

### 2.4、producer ###

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic topic1

### 2.5、consumer 和 group ###

consumer start

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic topic1
    bin/kafka-console-consumer.sh --zookeeper 192.168.1.74:2181 --from-beginning --topic topic1

group命令，这里可看到 消费者组的Topic 管理（分区）

bin/kafka-consumer-groups.sh --zookeeper 192.168.1.74:2181 --list
    bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list
    
    
    bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group dltGroup --describe
    bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group program --describe
    bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group fooGroup --describe
    bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group multiGroup --describe

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyMTA1NjI5_size_16_color_FFFFFF_t_70]

### 2.6、zookeeper ###

启动 zookeeper 客户端，管理 kafka

bin/zookeeper-shell.sh 192.168.1.74:2181
    ls /
    ls /brokers/topics

*  查看 节点

> ls /brokers/ids

*  查看 topic

> ls /brokers/topics

*  查看删除的消息

> ls /admin/delete\_topics

*  其他

> ls /log\_dir\_event\_notification  
> ls /consumers  
> ls /consumers/console-consumer-45795  
> ls /consumers/console-consumer-45795/offsets  
> ls /consumers/console-consumer-45795/offsets/topic2

> ls /consumers/console-consumer-97962/offsets/topic1  
> ls /consumers/console-consumer-97962/owners/topic1  
> ls /consumers/console-consumer-97962/ids

> ls /vod\_clip\_dev\_new  
> ls /latest\_producer\_id\_block  
> ls /controller\_epoch  
> ls /isr\_change\_notification  
> ls /admin  
> ls /admin/delete\_topics  
> ls /zookeeper  
> ls /zookeeper/quota  
> ls /config  
> ls /cluster  
> ls /cluster/id  
> ls /controller  
> ls /brokers  
> ls /dubbo-dev

### 2.7、性能测试 ###

bin/kafka-consumer-perf-test.sh --broker-list localhost:9092 --group fooGroup --topic topic1 --messages 1024

![在这里插入图片描述][20200629180139653.png]

### 2.8、日志查询 ###

bin/kafka-log-dirs.sh --describe --bootstrap-server 127.0.0.1:9092 --topic-list foos

结果

{ 
      "version": 1,
      "brokers": [
        { 
          "broker": 0,
          "logDirs": [
            { 
              "logDir": "/kafka/kafka-logs-3d0b0150932f",
              "error": null,
              "partitions": [
                { 
                  "partition": "foos-0",
                  "size": 384,
                  "offsetLag": 0,
                  "isFuture": false
                }
              ]
            }
          ]
        }
      ]
    }

然后，查看日志

ls  /kafka/kafka-logs-3d0b0150932f/foos-0
    
    tail -f /kafka/kafka-logs-3d0b0150932f/foos-0/00000000000000000000.log

## 三、更多Kafka,，客户端、服务端，监控可参考下文 ##

*  [kafka学习 – kafka connect][kafka_ _ kafka connect]
 *  [kafka学习 – 客户端Consumer API学习][kafka_ _ _Consumer API]
 *  [kafka学习 – 客户端Producer API学习][kafka_ _ _Producer API]
 *  [kafka学习 – 服务端命令学习][kafka_ _]
 *  [Springboot集成Kafka][Springboot_Kafka]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyMTA1NjI5_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/2020063009483596.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyMTA1NjI5,size_16,color_FFFFFF,t_70
[20200629180139653.png]: https://img-blog.csdnimg.cn/20200629180139653.png
[kafka_ _ kafka connect]: https://blog.csdn.net/qq_42105629/article/details/107077881
[kafka_ _ _Consumer API]: https://blog.csdn.net/qq_42105629/article/details/107073062
[kafka_ _ _Producer API]: https://blog.csdn.net/qq_42105629/article/details/107072983
[kafka_ _]: https://blog.csdn.net/qq_42105629/article/details/107008616
[Springboot_Kafka]: https://blog.csdn.net/qq_42105629/article/details/100569364