我使用Bulk API将数据导入到我的ELK堆栈中。
{"index":{"_index":"waf","_type":"logs","_id":"325d05bb6900440e"}} {"id":"325d05bb6900440e","country":"US","ip":"1.1.1.1","protocol":"HTTP/1.1","method":"GET","host":"xxxxx","user_agent":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36","uri":"/?a=><script>alert(1)</script>","request_duration":1999872,"triggered_rule_ids":["100030"],"action":"challenge","cloudflare_location":"unknown","occurred_at":"2017-01-23T17:38:58.46Z","rule_detail":[{"id":"","description":"ARGS:A"}],"rule_message":"Generic XSS Probing","type":"waf","rule_id":"100030"}
我在数据中有一个ip,我想使用GEOIP插件将其转为经度和纬度。
我创建了一个小人妖:
PUT _ingest/pipeline/geoip-info { "description": "Add geoip info", "processors": [ { "geoip": { "field": "ip", "target_field": "client_geoip", "properties": ["location"], "ignore_failure": true } } ] }`
但是,当我导入数据时,管道会被忽略,有人可以解释我如何修改批量API,以通过管道传递信息,以便为我添加创建地图所需的时长。
谢谢
在批量通话中,您缺少管道名称
here | V {"index":{"_index":"waf","_type":"logs","_id":"325d05bb6900440e", "pipeline": "geoip-info"}} {"id":"325d05bb6900440e","country":"US","ip":"1.1.1.1","protocol":"HTTP/1.1","method":"GET","host":"xxxxx","user_agent":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36","uri":"/?a=><script>alert(1)</script>","request_duration":1999872,"triggered_rule_ids":["100030"],"action":"challenge","cloudflare_location":"unknown","occurred_at":"2017-01-23T17:38:58.46Z","rule_detail":[{"id":"","description":"ARGS:A"}],"rule_message":"Generic XSS Probing","type":"waf","rule_id":"100030"}
或者您也可以在批量URL中进行设置
POST _bulk?pipeline=geoip-info