[Hadoop] CentOS7 部署Kafka0.10.2.1（单节点）

以你之姓@ 2022-05-09 01:50 181阅读 0赞

## 1. 软件版本 ##

*  操作系统：CentOS7.3
 *  Scala版本：2.11.12
 *  Zookeeper：zookeeper-3.4.5-cdh5.7.0
 *  Kafka版本：0.10.2.1（版本选择收spark straming+scala版本限制）

当前系统已经安装好scala和zookeeper。

## 2. Kafka安装 ##

下载解压

[hadoop@hadoop01 software]# wget http://apache.mirrors.hoobly.com/kafka/0.10.2.1/kafka_2.11-0.10.2.1.tgz
    [hadoop@hadoop01 software]$ tar -zxvf kafka_2.11-0.10.2.1.tgz -C ~/app/
    [hadoop@hadoop01 software]$ cd ~/app/
    [hadoop@hadoop01 app]$ ln -s kafka_2.11-0.10.2.1/ kafka

配置环境变量

[root@hadoop01 ~]# vi /etc/profile
    export KAFKA_HOME=/home/hadoop/app/kafka
    export PATH=$KAFKA_HOME/bin:$PATH
    
    [root@hadoop01 ~]# source /etc/profile

修改kafka配置文件

[hadoop@hadoop01 kafka]$ vi config/server.properties 
    broker.id=1
    log.dirs=/data/kafka-logs
    listeners=PLAINTEXT://192.168.1.8:9092
    advertised.listeners=PLAINTEXT://192.168.1.8:9092
    zookeeper.connect=192.168.1.8:2181

启动kafka

[hadoop@hadoop01 kafka]$ nohup kafka-server-start.sh config/server.properties &
    
    [hadoop@hadoop01 kafka]$ netstat -nltp | grep 9092
    (Not all processes could be identified, non-owned process info
     will not be shown, you would have to be root to see it all.)
    tcp6       0      0 192.168.1.8:9092       :::*                    LISTEN      5241/java           
    [hadoop@hadoop01 kafka]$ 
    
    # 停止使用命令
    [hadoop@hadoop01 kafka]$ bin/kafka-server-stop.sh

## 3.Kafka基本操作 ##

创建topic

[hadoop@hadoop01 kafka]$ bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
    Created topic "test".
    [hadoop@hadoop01 kafka]$

查看topic

[hadoop@hadoop01 kafka]$ bin/kafka-topics.sh --list --zookeeper localhost:2181
    test
    [hadoop@hadoop01 kafka]$

产生消息

[hadoop@hadoop01 kafka]$ bin/kafka-console-producer.sh --broker-list 192.168.1.8:9092 --topic test
    hello kafka
    hello zookeeper

消费消息

[hadoop@hadoop01 kafka]$ bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
    Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
    hello kafka
    hello zookeeper

查看topic描述信息

[hadoop@hadoop01 kafka]$ bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic test
    Topic:test	PartitionCount:1	ReplicationFactor:1	Configs:
    	Topic: test	Partition: 0	Leader: 1	Replicas: 1	Isr: 1
    [hadoop@hadoop01 kafka]$

删除topic

[hadoop@hadoop01 kafka]$ bin/kafka-topics.sh --delete --zookeeper localhost:2181 --topic test

注意：kafka默认删除topic操作是disable掉的，要想enable此操作，要在kafka配置文件里面修改参数：

[hadoop@hadoop01 kafka]$ vi config/server.properties 
    delete.topic.enable=true