elasticsearch filebeat+logstash+elasticsearch加密传输测试

左手的ㄟ右手 2023-07-01 08:57 13阅读 0赞

--------------------

### elasticsearch filebeat+logstash+elasticsearch加密传输测试 ###

## ##

### \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* ###

**配置文件**

**filebeat**

filebeat.inputs:
    - type: log
      enabled: true
      paths:
        - /usr/share/filebeat/logs/*.log
     
    output.logstash:
      hosts: ["172.18.0.32:5044"]

**logstash**

管道配置文件:logstash.conf
    
    input {
      beats {
        port => 5044
      }
    }
    
    filter {
      grok {
        match => { "message" => "%{NUMBER:document_id}\s+%{GREEDYDATA:info}" }
      }
    
      mutate {
        remove_field => ["host","agent","message","log","tags","input","ecs"]
      }
    }
    
    output {
    
      stdout { }
    
      elasticsearch { 
        hosts => ["172.18.0.33:9200"]
        user => "logstash_system"
        password => "123456"
        index => "info-%{+yyyy.MM.dd}"
        document_id => "%{document_id}"
      }
    }
    
    
    ************************
    配置文件logstash.yml
    
    http.host: "0.0.0.0"
    xpack.monitoring.enabled: true
    xpack.monitoring.elasticsearch.hosts: [ "http://172.18.0.33:9200" ]
    xpack.monitoring.elasticsearch.username: "logstash_system"
    xpack.monitoring.elasticsearch.password: "123456"

**elasticsearch**

http.host: 0.0.0.0
    network.host: 0.0.0.0
    
    discovery.type: single-node
    discovery.seed_hosts: ["172.18.0.33"]
    
    http.cors.enabled: true
    http.cors.allow-origin: "*"
    
    xpack.security.enabled: true

### \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* ###

**创建容器**

**filebeat**

docker run -it -d --net fixed --ip 172.18.0.31 \
    -v /usr/elasticsearch/elk/filebeat/config/filebeat.yml:/usr/share/filebeat/filebeat.yml \
    -v /usr/elasticsearch/elk/filebeat/logs:/usr/share/filebeat/logs \
    --name filebeat docker.elastic.co/beats/filebeat:7.5.1

**logstash**

docker run -it --net fixed --ip 172.18.0.32 -p 5044:5044 \
    -v /usr/elasticsearch/elk/logstash/config/logstash.conf:/usr/share/logstash/pipeline/logstash.conf \
    -v /usr/elasticsearch/elk/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml \
    --name logstash docker.elastic.co/logstash/logstash:7.5.1

**elasticsearch**

docker run -it --net fixed --ip 172.18.0.33 -p 9202:9200 -p 9302:9300  \
    -e ES_JAVA_OPTS="-Xms512m -Xmx512m"  \
    -v /usr/elasticsearch/elk/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml \
    --name es-single elasticsearch:7.5.1

**说明：先启动elasticsearch，设置好内置用户的密码后再启动logstash、filebeat**

### \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* ###

**相关输出**

**logstash控制台输出**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzkzMTYyNQ_size_16_color_FFFFFF_t_70][]

**查看elasticsearch文档**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzkzMTYyNQ_size_16_color_FFFFFF_t_70 1][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzkzMTYyNQ_size_16_color_FFFFFF_t_70 2][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzkzMTYyNQ_size_16_color_FFFFFF_t_70 3][]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzkzMTYyNQ_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200121193118989.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzkzMTYyNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzkzMTYyNQ_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20200121192920141.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzkzMTYyNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzkzMTYyNQ_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20200121192938241.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzkzMTYyNQ==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzkzMTYyNQ_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20200121192958395.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MzkzMTYyNQ==,size_16,color_FFFFFF,t_70