I have k8s cluster with сonfigured fluent-bit logger. Configuration of fluent-bit:
custom_parsers.conf: |
[PARSER]
Name docker_no_time
Format json
Time_Keep Off
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
fluent-bit.conf: |
[SERVICE]
Daemon Off
Flush 1
Log_Level info
Parsers_File parsers.conf
Parsers_File custom_parsers.conf
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_Port 2020
Health_Check On
[INPUT]
Name tail
Path /var/log/containers/*.log
Tag kube.*
DB /var/log/fluent-bit-kube.sqlite
Mem_Buf_Limit 5MB
Skip_Long_Lines On
[FILTER]
Name kubernetes
Match kube.*
Merge_Log On
Keep_Log Off
K8S-Logging.Parser On
K8S-Logging.Exclude On
[FILTER]
Name modify
Match kube.*
Add kube_cluster_name dev-k8s
[OUTPUT]
Name gelf
Match kube.*
Host log.my-graylog.ru
Port 12201
Mode udp
Gelf_Short_Message_Key log
Gelf_Host_Key dev.k8s
Compress false
But! Single pod(container) has additional field: MachineName . All messages for this pod is correct, correct multiline. Other pods doesn't contains this field. In *.log files inside /var/log/containers/
this field is absent for all containers, including that special pod (with MachineName
).
If look at all the logs from cluster in graylog can be seen field kubernetes_container_name. But messages incorrect, breaked multiline.
What other settings can modify log messages ? What else can I check to find the answer?
My goal is to add MachineName field for all pods in cluster