I'm using filebeat to retrieve log files to Elastic cloud. I'd like to put errors and java exception in one document instead of multiple for each line. I managed to make it work on text log files but it does not seem to work on containers logs.
Here's the configuration for docker on my filebeat.yml :
...
filebeat.inputs:
- type: container
enabled: true
paths:
- '/var/lib/docker/containers/*/*.log'
pipeline: docker
parsers:
- multiline:
type: pattern
pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
negate: true
match: after
...
processors:
- add_host_metadata:
when.not.contains.tags: forwarded
- add_cloud_metadata: ~
- add_docker_metadata:
host: "unix:///var/run/docker.sock"
- add_kubernetes_metadata: ~
- decode_json_fields:
fields: ["message"]
target: "json"
overwrite_keys: true
A usual log message would look like this :
2023-05-10 10:18:31.990 INFO 1 --- XXXX
