elasticsearch整合logstash日志收集

你的名字 2022-04-13 12:54 292阅读 0赞

Elasticsearch+Logstash+Kiabana环境安装  虚拟机建议内存设置为2G

# Elasticsearch环境安装 #

[https://blog.csdn.net/qq\_38270106/article/details/84309702][https_blog.csdn.net_qq_38270106_article_details_84309702]

# Logstash环境安装 #

**1****、下载logstash****安装包**

[https://pan.baidu.com/s/1-q0eJW1MZZqlqiUERdrzcQ][https_pan.baidu.com_s_1-q0eJW1MZZqlqiUERdrzcQ]

**2、上传、解压**

> cd /usr/local
> 
> tar –zxvf  logstash-6.4.3.tar.gz

**3****、在config****目录下新建logstash.conf** **读入并且读出日志信息**

> cd /usr/local/logstash-6.4.3/config
> 
> vi logstash.conf

标准打印，在文件中加入如下内容

input {
        # 从文件读取日志信息 输送到控制台
        file {
            # 日志文件的路径 这里以es的log路径为例
            path => "/usr/local/elasticsearch-6.4.3/logs/myes.log"
            codec => "json" ## 以JSON格式读取日志
            type => "elasticsearch"
            start_position => "beginning"
        }
    }
    
    # filter {
    #
    # }
    
    output {
        # 标准输出 
        # stdout {}
        # 输出进行格式化，采用Ruby库来解析日志   
         stdout { codec => rubydebug }
    }

# 启动logstash #

> cd /usr/local/logstash-6.4.3/bin 
> 
> ./logstash -f ../config/logstash.conf

#  启动ES #

> cd /usr/local/elasticsearch-6.4.3/bin
> 
> su liuwqboy
> 
> ./elasticsearch

日志已经被logstash收集起来了

# ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM4MjcwMTA2_size_16_color_FFFFFF_t_70][] logstash收集日志，并持久化到ES服务器 #

**在config****目录下新建logstash2.conf** **读入并且读出日志信息**

> cd /usr/local/logstash-6.4.3/config
> 
> vi logstash2.conf

input {
        # 从文件读取日志信息 输送到控制台
        file {
            path => "/usr/local/elasticsearch-6.4.3/logs/my-application.log"
            codec => "json" ## 以JSON格式读取日志
            type => "elasticsearch"
            start_position => "beginning"
        }
    }
    
    # filter {
    #
    # }
    
    output {
        # 标准输出 
        # stdout {}
        # 输出进行格式化，采用Ruby库来解析日志   
         stdout { codec => rubydebug }
        # 把收集的日志持久化到ES服务器
    	 elasticsearch {
            hosts => ["192.168.33.129:9200"]
            index => "es-%{+YYYY.MM.dd}"
        }	 
    }

# 分别启动logstash和ES  #

注意：logstash要指定logstash2.conf文件启动

# **Kibana环境安装** #

[https://blog.csdn.net/qq\_38270106/article/details/84311168][https_blog.csdn.net_qq_38270106_article_details_84311168]

**启动kibana**

>  cd /usr/local/kibana-6.4.3-linux-x86\_64/bin
> 
> ./kibana

访问：[http://192.168.128.156:5601][http_192.168.128.156_5601]

日志已经持久化到了ES服务器了

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM4MjcwMTA2_size_16_color_FFFFFF_t_70 1][]

[https_blog.csdn.net_qq_38270106_article_details_84309702]: https://blog.csdn.net/qq_38270106/article/details/84309702
[https_pan.baidu.com_s_1-q0eJW1MZZqlqiUERdrzcQ]: https://pan.baidu.com/s/1-q0eJW1MZZqlqiUERdrzcQ
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM4MjcwMTA2_size_16_color_FFFFFF_t_70]: /images/20220413/7f4c03b7006f4842945e92807db40272.png
[https_blog.csdn.net_qq_38270106_article_details_84311168]: https://blog.csdn.net/qq_38270106/article/details/84311168
[http_192.168.128.156_5601]: http://192.168.128.156:5601
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM4MjcwMTA2_size_16_color_FFFFFF_t_70 1]: /images/20220413/289c254faeee481fada7948a32e3a724.png