在我的场景中,Logstash接收的syslog行的“时间戳”是UTC,我们在Elasticsearch输出中使用事件“ timestamp”:
output { elasticsearch { embedded => false host => localhost port => 9200 protocol => http cluster => 'elasticsearch' index => "syslog-%{+YYYY.MM.dd}" } }
我的问题是,在UTC午夜,Logstash会在一天结束之前在时区外(GMT-4 => America / Montreal)将日志发送到其他索引,并且由于“时间戳记”,因此在20h(8h PM)之后索引没有日志就是UTC。
我们已经完成了围绕时区的转换工作,但是性能明显下降:
filter { mutate { add_field => { # Create a new field with string value of the UTC event date "timestamp_zoned" => "%{@timestamp}" } } date { # Parse UTC string value and convert it to my timezone into a new field match => [ "timestamp_zoned", "yyyy-MM-dd HH:mm:ss Z" ] timezone => "America/Montreal" locale => "en" remove_field => [ "timestamp_zoned" ] target => "timestamp_zoned_obj" } ruby { # Output the zoned date to a new field code => "event['index_day'] = event['timestamp_zoned_obj'].strftime('%Y.%m.%d')" remove_field => [ "timestamp_zoned_obj" ] } } output { elasticsearch { embedded => false host => localhost port => 9200 protocol => http cluster => 'elasticsearch' # Use of the string value index => "syslog-%{index_day}" } }
有没有一种方法可以优化此配置?
这是优化配置,请尝试并测试性能。
您无需使用mutate和date插件。ruby直接使用插件。
mutate
date
ruby
input { stdin { } } filter { ruby { code => " event['index_day'] = event['@timestamp'].localtime.strftime('%Y.%m.%d') " } } output { stdout { codec => rubydebug } }
输出示例:
{ "message" => "test", "@version" => "1", "@timestamp" => "2015-03-30T05:27:06.310Z", "host" => "BEN_LIM", "index_day" => "2015.03.29" }