I print the logs as follows. I want to index for fields such as traceid, guid. How can I do this with a configuration in fluentd and elasticseeach?
Java logger
stringBuilder.append("guid=[").append((guid == null ? "" : guid)).append("] ");
stringBuilder.append("traceid=[").append(tracer.currentSpan().context().traceId()).append("] ");
stringBuilder.append("spanid=[").append(tracer.currentSpan().context().spanId()).append("] ");
stringBuilder.append("method=[").append(request.getMethod()).append("] ");
stringBuilder.append("path=[").append(request.getRequestURI()).append("] ");
logger.info(stringBuilder.toString());
fluentd configuration file
apiVersion: v1
data:
01_sources.conf: |-
## logs from podman
<source>
@type tail
@id in_tail_container_logs
@label @KUBERNETES
path /var/log/containers/*auth*.log
pos_file /var/log/fluentd-containers.log.pos
tag kubernetes.*
read_from_head true
<parse>
@type multi_format
<pattern>
format json
time_key time
time_type string
time_format "%Y-%m-%dT%H:%M:%S.%NZ"
keep_time_key false
</pattern>
<pattern>
format regexp
expression /^(?<time>.+) (?<stream>stdout|stderr)( (.))? (?<log>.*)$/
time_format '%Y-%m-%dT%H:%M:%S.%NZ'
keep_time_key false
</pattern>
</parse>
emit_unmatched_lines true
</source>
04_outputs.conf: |-
<label @OUTPUT>
<match **>
@type elasticsearch
host "elasticsearch-master"
port 9200
user elastic
index_name fluentd-${time.strftime('%Y.%m.%d')}
logstash_format true
logstash_prefix fluentd
logstash_dateformat %Y%m%d
</match>
</label>
How can I do?