linux下使用docker-compose搭建elk

小灰灰 2023-01-18 04:11 217阅读 0赞

记录下在linux下简单的搭建单节点的elk

话不多说，直接上代码

docker-compose-elk.yml

version: '3'
    services:
       elasticsearch:
          image: elasticsearch:7.5.0
          container_name: es7.5.0
          environment:
             - discovery.type=single-node #配置es启动单节点
             - cluster.name=robin #配置es集群名称
             - ES_JAVA_OPTS=-Xms512m -Xmx512m #设置使用jvm内存大小
             - ELASTIC_PASSWORD=123456
             - xpack.security.enabled=true
             - xpack.security.transport.ssl.enabled=true
             - xpack.license.self_generated.type=basic
             - TZ=Asia/Shanghai
          volumes:
             - /mydata/elasticsearch/plugins:/usr/share/elasticsearch/plugins #插件文件挂载
             - /mydata/elasticsearch/data:/usr/share/elasticsearch/data #数据文件挂载
          ports:
             - 9200:9200
             - 9300:9300
    
       kibana:
          image: kibana:7.5.0
          container_name: kibana7.5.0
          depends_on:
             - elasticsearch #kibana在elasticsearch启动之后再启动
          environment:
             - elasticsearch.hosts=http://ip:9200 #设置访问elasticsearch的地址
             - TZ=Asia/Shanghai
          #挂载文件
          volumes:
             - /mydocker/conf/kibana.yml:/usr/share/kibana/config/kibana.yml
          ports:
             - 5601:5601
          restart: always
    
       logstash:
          image: logstash:7.5.0
          container_name: logstash7.5.0
          environment:
             - xpack.monitoring.enabled=true
             - xpack.monitoring.elasticsearch.username=elastic
             - xpack.monitoring.elasticsearch.password=123456
             - TZ=Asia/Shanghai
          volumes:
             - /mydocker/conf/logstash.conf:/usr/share/logstash/pipeline/logstash.conf #挂载logstash的配置文件
          depends_on:
             - elasticsearch  #kibana在elasticsearch启动之后再启动
          ports:
             - 9600:9600
             - 5044:5044

kibana.yml

elasticsearch.hosts: [ "http://ip:9200" ]
    server.host: "0.0.0.0"
    server.name: kibana
    xpack.monitoring.ui.container.elasticsearch.enabled: true
    i18n.locale: zh-CN #中文
    elasticsearch.username: 'elastic'
    elasticsearch.password: '123456'

logstash.conf

input {
            tcp {
                    mode => "server"
                    host => "0.0.0.0"
                    port => 5044
                    codec => json_lines
        }
    }
    output {
            elasticsearch {
                    hosts => ["http://ip:9200"]  # ES 地址和端口
                    index => "%{[appname]}-%{+YYYY.MM.dd}"  # 索引名称, 按天自动创建索引
                    user => "elastic"  # 用户名
                    password => "123456"  # 密码
            }
            stdout {
                    codec => rubydebug
            }
    }

运行 docker-compose -f docker-compose-elk.yml up -d

![20210430165516987.png][]

查看

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xfYnVfdGluZ19odWE_size_16_color_FFFFFF_t_70][]

查看kibana

输入地址 http://ip:5601/ 输入配置的账号密码

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xfYnVfdGluZ19odWE_size_16_color_FFFFFF_t_70 1][]

==============================================

springboot项目中使用logstash

引入依赖

<dependency>
        <groupId>net.logstash.logback</groupId>
        <artifactId>logstash-logback-encoder</artifactId>
        <version>6.0</version>
    </dependency>

添加配置文件 logback-spring.xml

<?xml version="1.0" encoding="UTF-8" ?>
    <configuration>
        <appender name="consoleLog" class="ch.qos.logback.core.ConsoleAppender">
            <layout class="ch.qos.logback.classic.PatternLayout">
                <pattern> %d{yyyy-MM-dd HH:mm:ss} %contextName [%thread] %-5level %logger{36} : %msg%n </pattern>
            </layout>
        </appender>
        <appender name="fileInfoLog" class="ch.qos.logback.core.rolling.RollingFileAppender">
            <filter class="ch.qos.logback.classic.filter.LevelFilter">
                <level>ERROR</level>
                <onMatch>DENY</onMatch>
                <onMismatch>ACCEPT</onMismatch>
            </filter>
            <encoder>
                <pattern> %d{yyyy-MM-dd HH:mm:ss} %contextName [%thread] %-5level %logger{36} : %msg%n </pattern>
            </encoder> 
            <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
                 <fileNamePattern>/log/info/usso/info.%d.log</fileNamePattern>
            </rollingPolicy>
        </appender>
        <appender name="fileErrorLog" class="ch.qos.logback.core.rolling.RollingFileAppender">
            <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
                <level>ERROR</level>
            </filter>
            <encoder>
                <pattern> %d{yyyy-MM-dd HH:mm:ss} %contextName [%thread] %-5level %logger{36} : %msg%n </pattern>
            </encoder> 
            <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
                 <fileNamePattern>/log/error/usso/error.%d.log</fileNamePattern>
            </rollingPolicy>
        </appender>
        
        <appender name="logStash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
            
            <destination>服务器ip:5044</destination>
            
            <encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder">
                <customFields>{"appname":"uaa"}</customFields>
            </encoder>
        </appender>
        <root level="info">
            <appender-ref ref="consoleLog" />
            <appender-ref ref="fileInfoLog" />
            <appender-ref ref="fileErrorLog" />
            <appender-ref ref="logStash" />
        </root>
    </configuration>

运行项目，即可在kibana查看

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xfYnVfdGluZ19odWE_size_16_color_FFFFFF_t_70 2][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xfYnVfdGluZ19odWE_size_16_color_FFFFFF_t_70 3][]

按操作创建成功后，点击下图图标即可查看

![20210430170958945.png][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xfYnVfdGluZ19odWE_size_16_color_FFFFFF_t_70 4][]

参考：

[https://github.com/deviantony/docker-elk][https_github.com_deviantony_docker-elk]

[https://doc.yonyoucloud.com/doc/logstash-best-practice-cn/index.html][https_doc.yonyoucloud.com_doc_logstash-best-practice-cn_index.html]

当然也有现成的，类似腾讯云，直接购买即可

[https://cloud.tencent.com/document/product/845/17153][https_cloud.tencent.com_document_product_845_17153]

[20210430165516987.png]: /images/20221021/21d0dac51ab4463e8a389248ae8ef537.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xfYnVfdGluZ19odWE_size_16_color_FFFFFF_t_70]: /images/20221021/f36438ec98ed4825b1800ee2d55be629.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xfYnVfdGluZ19odWE_size_16_color_FFFFFF_t_70 1]: /images/20221021/8ccc642ebe44421aa3989396abda3739.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xfYnVfdGluZ19odWE_size_16_color_FFFFFF_t_70 2]: /images/20221021/2b6b2361521045e0b572de3d8e0b5dc2.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xfYnVfdGluZ19odWE_size_16_color_FFFFFF_t_70 3]: /images/20221021/03c2b62399924e81995ce40dce438111.png
[20210430170958945.png]: /images/20221021/1cfb8043ead64613b2b1361553e818e5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xfYnVfdGluZ19odWE_size_16_color_FFFFFF_t_70 4]: /images/20221021/b85fb07eff834d56a69ac0d007660807.png
[https_github.com_deviantony_docker-elk]: https://github.com/deviantony/docker-elk
[https_doc.yonyoucloud.com_doc_logstash-best-practice-cn_index.html]: https://doc.yonyoucloud.com/doc/logstash-best-practice-cn/index.html
[https_cloud.tencent.com_document_product_845_17153]: https://cloud.tencent.com/document/product/845/17153