记录logback + ELK日志-蒲公英云

记录logback + ELK日志

logback + ELK日志

安装es

略

安装logstash

https://www.elastic.co/guide/en/logstash/current/index.html

新建一个简单的配置文件，只设置logstash的输入和输出，输入为tcp监听的端口，输出为es

input {

tcp {
    host => "10.1.1.1"
    port => 4567
    mode => "server"
    codec => json_lines
}

}

output {

 elasticsearch {
    action => "index"
    hosts => ["10.1.1.1:9200"]
    index => "%{[appName]}"
 }

}

使用命令行指定配置文件的方式启动logstash，可以nohup后台启动，我这里示例前台启动，启动参数详细介绍：https://www.elastic.co/guide/en/logstash/current/running-logstash-command-line.html

~/su/logstash-7.3.0# ./bin/logstash -f ./config/logstash-es.conf

看到如下日志打印就是启动成功

[2020-08-21T14:23:05,840][INFO ][logstash.inputs.tcp      ] Starting tcp input listener {:address=>"10.35.96.110:4567", :ssl_enable=>"false"}
[2020-08-21T14:23:05,986][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2020-08-21T14:23:06,946][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

安装kibana

kibana和的版本和es的版本必须一样，否则启动报错

修改配置文件，打开必要的配置项的注释

# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601
# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "10.1.1.1"
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["http://10.1.1.1:9200"]
# Kibana uses an index in Elasticsearch to store saved searches, visualizations and
# dashboards. Kibana creates a new index if the index doesn't already exist.
kibana.index: ".kibana"

启动kibana

~/su/kibana-6.8.1-linux-x86_64/bin# ./kibana

看到如下日志就是启动成功

log   [10:55:32.074] [info][migrations] Creating index .kibana_1.
  log   [10:55:32.230] [info][migrations] Pointing alias .kibana to .kibana_1.
  log   [10:55:32.355] [info][migrations] Finished in 281ms.
  log   [10:55:32.359] [info][listening] Server running at http://10.35.96.110:5601
  log   [10:55:32.664] [info][status][plugin:spaces@6.8.1] Status changed from yellow to green - Ready

到浏览器访问即可打开kibana的页面http://10.1.1.1:5601。

服务logback配置

<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="false" scan="true" scanPeriod="10 seconds">
    <property name="LogHome" value="logs/" />
    <property name="LogPattern" value="[%d{yyyy-MM-dd HH:mm:ss.SSS}][%p][App:%t][%C{0}:%M:%L]%X{requestId}%X{errorId}, %m%n" />
    <!-- 控制台 -->
    <appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>${LogPattern}</pattern>
            <charset>utf-8</charset>
        </encoder>
    </appender>
    <springProperty scop="context" name="destination" source="log.destination" defaultValue="127.0.0.1:4567"/>
    <springProperty scop="context" name="appName" source="spring.application.name" defaultValue="appLog"/>
    <appender name="logstash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
        <param name="Encoding" value="UTF-8"/>
        <destination>${destination}</destination>
        <!-- <filter class="com.program.interceptor.ELKFilter"/>-->
        <!-- encoder is required -->
        <encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder" >
            <customFields>{"appName":"${appName}"}</customFields>
        </encoder>
    </appender>
    <root level="INFO" additivity="false">
        <appender-ref ref="stdout" />
        <appender-ref ref="infoAppender" />
        <appender-ref ref="logstash" />
    </root>
</configuration>

其中的是在application.properties文件中获取的

server.port=8080
# ......
spring.application.name=es-log
log.destination=10.1.1.1:4567

查看日志

启动应用让应用打印日志后，通过head插件查看es，成功创建了记录日志的索引，这里我们设置的是es-log，并且记录了日志。
也可直接到kibana页面Management, Create index pattern, 可以找到我们记录日志的索引，添加后可在Discover菜单查看日志
还可使用字段：值的方式搜索过滤日志，e.g. status:200 AND extension:PHP)