Elasticsearch7.3.0导入测试数据

Love The Way You Lie 2021-11-06 00:22 249阅读 0赞

下载最MovieLens最小测试数据集：[https://grouplens.org/datasets/movielens/][https_grouplens.org_datasets_movielens]  
Logstash下载：[https://www.elastic.co/cn/downloads/logstash][https_www.elastic.co_cn_downloads_logstash]  
Logstash参考文档：[https://www.elastic.co/guide/en/logstash/current/index.html][https_www.elastic.co_guide_en_logstash_current_index.html]  
配置logstash.conf注意的是E:/ELK/ml-latest-small/movies.csv一定要是这个斜杠，就这个问题搞了一天，日志完全不报错。  
logstash.conf:  
input \{  
file \{  
path => “E:/ELK/ml-latest-small/movies.csv”  
start\_position => “beginning”  
sincedb\_path => “null”  
\}  
\}

filter \{  
csv \{  
separator => “,”  
columns => \[“id”,“content”,“genre”\]  
\}

mutate \{  
split => \{ “genre” => “|” \}  
remove\_field => \[“path”, “host”,"@timestamp",“message”\]  
\}  
mutate \{  
split => \[“content”, “(”\]  
add\_field => \{ “title” => “%\{\[content\]\[0\]\}”\}  
add\_field => \{ “year” => “%\{\[content\]\[1\]\}”\}  
\}  
mutate \{  
convert => \{  
“year” => “integer”  
\}  
strip => \[“title”\]  
remove\_field => \[“path”, “host”,"@timestamp",“message”,“content”\]  
\}  
\}  
output \{  
elasticsearch \{  
hosts => “[http://localhost:9200][http_localhost_9200]”  
index => “movies”  
document\_id => “%\{id\}”  
\}  
stdout \{\}  
\}  
E:\\logstash-7.3.0\\bin\\logstash.bat -f E:\\logstash-7.3.0\\config\\logstash.conf

[https_grouplens.org_datasets_movielens]: https://grouplens.org/datasets/movielens/
[https_www.elastic.co_cn_downloads_logstash]: https://www.elastic.co/cn/downloads/logstash
[https_www.elastic.co_guide_en_logstash_current_index.html]: https://www.elastic.co/guide/en/logstash/current/index.html
[http_localhost_9200]: http://localhost:9200