使用Docker(k8s)安装Kafka并使用宿主机连接

悠悠 2023-01-05 05:09 175阅读 0赞

### 使用Docker（k8s）安装Kafka并使用宿主机连接 ###

1.  安装Docker及docker-compose  
    具体安装方法可以去[官网][Link 1]看教程  
    检查docker-compose是否安装成功  
    ![在这里插入图片描述][20210111171535401.png]
2.  创建 docker-compose.yml 文件

version: '2'
    services:
      zookeeper:
        image: "zookeeper"
        hostname: "zookeeper.local"
        container_name: "zookeeper"
        #设置网络别名 可随意取
        networks:
          local:
            aliases:
              - "zookeeper.local"
      kafka:
        image: "wurstmeister/kafka"
        hostname: "kafka.local"
        container_name: "kafka"
        ports:
          - "9092:9092"
        networks:
          local:
            aliases:
              - "kafka.local"
        environment:
          KAFKA_ADVERTISED_HOST_NAME: kafka.local
          KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    #设置网络，名为local模式
    networks:
      local:
        driver: bridge

1.  进入Kafka容器

docker exec -it kafka /bin/bash

1.  创建Topic

/opt/kafka_2.13-2.7.0/bin/kafka-topics.sh --create --zookeeper zookeeper:2181 -replication-factor 1 --partitions 1 --topic test_topic

1.  测试命令行生产消费

*  生产者

/opt/kafka_2.13-2.7.0/bin/kafka-console-producer.sh --broker-list kafka.local:9092 --topic test_topic

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L00yODM1OTIzMzg_size_16_color_FFFFFF_t_70]

*  消费者

/opt/kafka_2.13-2.7.0/bin/kafka-console-consumer.sh --bootstrap-server kafka.local:9092 --topic test_top
    ic --from-beginning

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L00yODM1OTIzMzg_size_16_color_FFFFFF_t_70 1]

1.  从宿主机使用代码连接Kafka  
    6.1 进入Zookeeper容器查看brokers注册信息
    
        # 进入容器
        docker exec -it zookeeper /bin/bash
        
        # 进入zookeeper命令行
        bin/zkCli.sh
    
    6.2 查看brokers注册信息
    
        get /brokers/ids/1001
    
    ![在这里插入图片描述][20210111175058923.png]  
    6.3 配置宿主机hosts
    
        # 添加
        127.0.0.1	kafka.local
    
    6.4 使用Java代码连接Kafka
    
        public class KafkaConsumerDemo { 
        public static void main(String[] args) { 
        	//1.配置消费者连接属性
        	Properties props = new Properties();
        	props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "kafka.local:9092");
        	props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        	props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        	props.put(ConsumerConfig.GROUP_ID_CONFIG, "test_demo");
        
        
        	//2.创建Kafka消费者
        	KafkaConsumer<String, String> consumer = new KafkaConsumer<String, String>(props);
        
        	//3.订阅topics
        	consumer.subscribe(Arrays.asList("test_topic"));
        // consumer.assign(Arrays.asList(new TopicPartition("test_topic", 0)));
        // consumer.seekToBeginning(Arrays.asList(new TopicPartition("test_topic", 0)));//不改变当前offset
        
        	//4.死循环读取消息
        	for (; ; ) { 
        		ConsumerRecords<String, String> records = consumer.poll(Duration.ofSeconds(1));
        
        		if (records != null && !records.isEmpty()) { 
        			records.forEach(r -> { 
        				System.out.println("key:" + r.key() + "----value:" + r.value());
        			});
        		}
        	}
          }
        }

测试成功  
![在这里插入图片描述][2021011117565910.png]

[Link 1]: https://www.docker.com/
[20210111171535401.png]: /images/20221119/558bf482a25d49cd863270aab665a41a.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L00yODM1OTIzMzg_size_16_color_FFFFFF_t_70]: /images/20221119/605c5ab611f647c1889a43442478fe04.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L00yODM1OTIzMzg_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20210111174513506.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L00yODM1OTIzMzg=,size_16,color_FFFFFF,t_70
[20210111175058923.png]: https://img-blog.csdnimg.cn/20210111175058923.png
[2021011117565910.png]: https://img-blog.csdnimg.cn/2021011117565910.png