七. kafka性能测试

喜欢ヅ旅行 2022-07-14 18:30 174阅读 0赞

生产者调优，主要可以利用kafka-producer-perf-test.sh(org.apache.kafka.tools.ProducerPerformance)。通过测试不同的配置来对比发送效率。

使用方法例子：

./kafka-producer-perf-test.sh   --topic test  --num-records 100000  --throughput 10000 --producer-props bootstrap.servers=app206:9092,app205:9092,app204:9092  --record-size 1000

--topic TOPIC          produce messages to this topic  
  --num-records NUM-RECORDS  
                         number of messages to produce  
  --payload-delimiter PAYLOAD-DELIMITER  
                         provides delimiter to be used when --payload-file is provided.  Defaults  to  new  line.  Note that this parameter will be ignored if --  
                         payload-file is not provided. (default: \\n)  
  --throughput THROUGHPUT  
                         throttle maximum message throughput to \*approximately\* THROUGHPUT messages/sec  
  --producer-props PROP-NAME=PROP-VALUE \[PROP-NAME=PROP-VALUE ...\]  
                         kafka producer related configuration properties like bootstrap.servers,client.id  etc.  These  configs take precedence over those passed  
                         via --producer.config.  
  --producer.config CONFIG-FILE  
                         producer config properties file.  
  
  
  either --record-size or --payload-file must be specified but not both.  
  
  
  --record-size RECORD-SIZE  
                         message size in bytes. Note that you must provide exactly one of --record-size or --payload-file.  
  --payload-file PAYLOAD-FILE  
                         file to read the message payloads from. This works  only  for  UTF-8  encoded  text  files.  Payloads  will be read from this file and a  
                         payload will be randomly selected when sending messages. Note that you must provide exactly one of --record-size or --payload-file.

1.  messages                生产者发送总的消息数量  
    
2.  message-size                每条消息大小  
    
3.  batch-size                每次批量发送消息的数量  
    
4.  topics                        生产者发送的topic

消费者

/kafka-consumer-perf-test.sh