Kafka 伪集群模式安装

ゝ一纸荒年。 2022-06-12 08:49 213阅读 0赞

### 一、安装准备 ###

*  **下载`Kafka`：**[CentOS 7 下载 安装 Kafka][CentOS 7 _ _ Kafka]
 *  **安装`JDK`：**[CentOS 7 下载 安装 JDK][CentOS 7 _ _ JDK]  
    ![在这里插入图片描述][2019092610062578.png]
 *  **安装`Zookeeper`：**[Zookeeper 单机模式 安装][Zookeeper _]  
    ![在这里插入图片描述][20191015140946394.png]

--------------------

### 二、`Kafka`伪集群模式 ###

#### A、创建`log`目录： ####

mkdir -p /data/data/kafka/log_1
    mkdir -p /data/data/kafka/log_2
    mkdir -p /data/data/kafka/log_3

![在这里插入图片描述][20191017134616169.png]

#### B、拷贝多份配置文件 ####

cd /data/opt/kafka_2.12-2.3.0
    
    cp ./config/server.properties ./config/server_1.properties
    cp ./config/server.properties ./config/server_1.properties
    cp ./config/server.properties ./config/server_1.properties

*  **`/data/opt/kafka_2.12-2.3.0/config/server_1.properties`**
    
        # kafka集群唯一标识
        broker.id=1
        
        # broker监听IP和端口也可以是域名
        listeners=PLAINTEXT://127.0.0.1:9092
        
        # 端口号
        port=9092
        # 日志存放
        log.dirs=/data/data/kafka/log_1
        
        # zookeeper 地址和端口,多个地址 , 隔开
        # zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182
        zookeeper.connect=127.0.0.1:2181
        
        # 可删除 topic
        delete.topic.enable=true
 *  **`/data/opt/kafka_2.12-2.3.0/config/server_2.properties`**
    
        # kafka集群唯一标识
        broker.id=2
        
        # broker监听IP和端口也可以是域名
        listeners=PLAINTEXT://127.0.0.1:9093
        
        # 端口号
        port=9093
        # 日志存放
        log.dirs=/data/data/kafka/log_2
        
        # zookeeper 地址和端口,多个地址 , 隔开
        # zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182
        zookeeper.connect=127.0.0.1:2181
        
        # 可删除 topic
        delete.topic.enable=true
 *  **`/data/opt/kafka_2.12-2.3.0/config/server_3.properties`**
    
        # kafka集群唯一标识
        broker.id=3
        
        # broker监听IP和端口也可以是域名
        listeners=PLAINTEXT://127.0.0.1:9094
        
        # 端口号
        port=9094
        # 日志存放
        log.dirs=/data/data/kafka/log_3
        
        # zookeeper 地址和端口,多个地址 , 隔开
        # zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182
        zookeeper.connect=127.0.0.1:2181
        
        # 可删除 topic
        delete.topic.enable=true

#### C、配置`Kafka`环境变量 ####

vim  /etc/profile

# Kafka环境变量
    export KAFKA_HOME=/data/opt/kafka_2.12-2.3.0
    export PATH=$KAFKA_HOME/bin:$PATH

#### D、`Kafka`环境变量生效 ####

source /etc/profile

#### E、运行`Kafka`伪集群 ####

cd /data/opt/kafka_2.12-2.3.0
    
    ./bin/kafka-server-start.sh ./config/server_1.properties &
    ./bin/kafka-server-start.sh ./config/server_2.properties &
    ./bin/kafka-server-start.sh ./config/server_3.properties &

#### F、测试`Kafka`伪集群 ####

*  **创建`topic`**
    
        cd /data/opt/kafka_2.12-2.3.0
        
        # 先创建一个topic  现在我们创建一个具有三份副本、两个划分partitions的主题
        ./bin/kafka-topics.sh --create --zookeeper 127.0.0.1:2181 --replication-factor 3 --partitions 2 --topic tmpTopic
    
    ![在这里插入图片描述][20191024110030513.png]
 *  **显示`topic`信息**
    
        ./bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic testTopic
    
    ![在这里插入图片描述][20191024110151833.png]
 *  **查看`topic`列表**
    
        ./bin/kafka-topics.sh --list --zookeeper localhost:2181
    
    ![在这里插入图片描述][20191024110236632.png]
 *  **查看`Zookeeper`节点信息**
    
        ls /brokers/ids
    
    ![在这里插入图片描述][20191024110445633.png]
 *  **启动生产者**
    
        ./bin/kafka-console-producer.sh --broker-list 127.0.0.1:9092,127.0.0.1:9093 --topic tmpTopic
    
    ![在这里插入图片描述][2019101614025831.png]
 *  **启动消费者**
    
        ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic testTopic
    
    ![在这里插入图片描述][20191016140328719.png]
 *  **测试生产者消费者**  
    ![在这里插入图片描述][20191016140403171.png]  
    ![在这里插入图片描述][2019101614043299.png]

#### G、关闭`Kafka`伪集群 ####

--------------------

https://blog.csdn.net/hudyang/article/details/80419214

4.3删除节点  
bin/kafka-topics.sh --delete --zookeeper 192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183 --topic test

(四)关闭集群服务

# 删除topic #

bin/kafka-topics.sh --delete --zookeeper 192.168.94.132:2181,192.168.94.132:2182,192.168.94.132:2183 --topic test

# 关闭kafka #

\[root@worker2 kafka\_2.12-1.1.0\]$ bin/kafka-server-stop.sh conf/server-1.properties

\[root@worker2 kafka\_2.12-1.1.0\]$ bin/kafka-server-stop.sh conf/server-2.properties

\[root@worker2 kafka\_2.12-1.1.0\]$ bin/kafka-server-stop.sh conf/server-3.properties

# 关闭zookeeper #

\[root@master zookeeper-3.4.11\]$ bin/zkServer.sh stop conf/zoo1.cfg

\[root@worker1 zookeeper-3.4.11\]$ bin/zkServer.shstop conf/zoo2.cfg

\[root@worker2 zookeeper-3.4.11\]$ bin/zkServer.shstop conf/zoo3.cfg=

[CentOS 7 _ _ Kafka]: https://blog.csdn.net/yhj19920417/article/details/60602881
[CentOS 7 _ _ JDK]: https://blog.csdn.net/yhj19920417/article/details/46547181
[2019092610062578.png]: /images/20220612/fc16a1fa478a4a49bee24ee0c9c46c12.png
[Zookeeper _]: https://blog.csdn.net/yhj19920417/article/details/68926984
[20191015140946394.png]: /images/20220612/bd07a10d33194ad6b0c0edd1e1503a3d.png
[20191017134616169.png]: /images/20220612/1121dd0483024bf58e35bc89a1d31939.png
[20191024110030513.png]: /images/20220612/304fb0fa36c142169a49d0f58afffad0.png
[20191024110151833.png]: /images/20220612/874ae868903d4a27ba00e0a2c13848c9.png
[20191024110236632.png]: /images/20220612/d3cb4485884446dc9b9066dd6dbe7c19.png
[20191024110445633.png]: /images/20220612/5773b7a690c247cb98373c9e2b7963c3.png
[2019101614025831.png]: /images/20220612/91148e90a5ee4328a840ccde7c577307.png
[20191016140328719.png]: /images/20220612/4e95ae7852d4418f87bdac3beb65a334.png
[20191016140403171.png]: /images/20220612/07358c1b518144bfad4abbea466ba7a3.png
[2019101614043299.png]: /images/20220612/3139f995efd2487dac8dc6d8af9bcbba.png