Skip to content

Commit

Permalink
Move dissect tokenizing from Beat to ES in Traefik Filebeat Access Fi…
Browse files Browse the repository at this point in the history
…leset (#10442)

Traefik access Metricset were parsing incoming log lines by using a mixed of dissect processing on the Beat and a Grok processing on ES. This PR moves all parsing to ES
  • Loading branch information
sayden committed Feb 1, 2019
1 parent 19d8016 commit c86f937
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 9 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Migrate Elasticsearch audit logs fields to ECS {pull}10352[10352]
- Several text fields in the Logstash module are now indexed as `keyword` fields with `text` multi-fields (ECS). {pull}10417[10417]
- Several text fields in the Elasticsearch module are now indexed as `keyword` fields with `text` multi-fields (ECS). {pull}10414[10414]
- Move dissect pattern for traefik.access fileset from Filbeat to Elasticsearch. {pull}10442[10442]

*Heartbeat*

Expand Down
9 changes: 0 additions & 9 deletions filebeat/module/traefik/access/config/traefik-access.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,3 @@ paths:
- {{$path}}
{{ end }}
exclude_files: [".gz$"]

processors:
- dissect:
tokenizer: '%{source.address} %{traefik.access.user_identifier} %{user.name} [%{traefik.access.time}]
"%{http.request.method} %{url.original} HTTP/%{http.version}"
%{http.response.status_code} %{traefik.access.message}'

field: "message"
target_prefix: ""
6 changes: 6 additions & 0 deletions filebeat/module/traefik/access/ingest/pipeline.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
{
"description": "Pipeline for parsing Traefik access logs. Requires the geoip and user_agent plugins.",
"processors": [
{
"dissect": {
"field": "message",
"pattern": "%{source.address} %{traefik.access.user_identifier} %{user.name} [%{traefik.access.time}] \"%{http.request.method} %{url.original} HTTP/%{http.version}\" %{http.response.status_code} %{traefik.access.message}"
}
},
{
"grok": {
"field": "traefik.access.message",
Expand Down

0 comments on commit c86f937

Please sign in to comment.