我想在Elasticsearch中导入一个文本文件。文本文件每行包含3个值。经过数小时的奋斗,我没有完成它。非常感谢您的帮助。
安装了Logstash的Elasticsearch 5.4.0。
样本数据:
username email hash username email hash username email hash username email hash username email hash
还构建了一个python脚本,但是它太慢了:
import requests import json from elasticsearch import Elasticsearch es = Elasticsearch([{'host': 'localhost', 'port': 9200}]) i = 1 with open("my2") as fileobject: for line in fileobject: username, email, hash = line.strip('\n').split(' ') body = {"username": username, "email": email, "password": hash} es.index(index='dbs', doc_type='db1', id=i, body=body) i += 1
编辑:感谢它的工作,但我想我的筛选器很烂,因为我希望它看起来像这样:
{ "_index": "logstash-2017.06.01", "_type": "db", "_id": "AVxinqK5XRvft8kN7Q6M", "_version": 1, "_score": null, "_source": { "username": "Marlb0ro", "email": "Marlb0ro@site.com", "hash": "123456", }
然后将数据如下所示:
{ "_index": "logstash-2017.06.01", "_type": "logs", "_id": "AVxinqK5XRvft8kN7Q6M", "_version": 1, "_score": null, "_source": { "path": "C:/Users/user/Desktop/user/log.txt", "@timestamp": "2017-06-01T07:46:22.488Z", "@version": "1", "host": "DESKTOP-FNGSJ6C", "message": "username email password", "tags": [ "_grokparsefailure" ] }, "fields": { "@timestamp": [ 1496303182488 ] }, "sort": [ 1496303182488 ] }
只需将其放入一个名为grok.conf:
grok.conf
input { file { path => "/path/to/your/file.log" start_position => beginning sincedb_path => "/dev/null" } } filter { grok { match => {"message" => "%{WORD:username} %{WORD:email} %{WORD:hash}" } } } output { elasticsearch { hosts => ["localhost:9200"] } }
然后使用运行Logstash,bin/logstash -f grok.conf您应该可以。
bin/logstash -f grok.conf