Disable/block the logs sent from fluentd sidecar container to Log Explorer in GCP

Question

Holla amigos, In Google Cloud GKE I have 3 containers inside a pod. The first one is the application, the second is the istio-proxy sidecar, and the third one is the fluentd sidecar. The scenario is simple where I would like to block/stop the logs that are being sent from the fluentd container to log explorer (GCP logging console). In the meantime, I would still like my fluentd store the logs inside the pod, so that I can check the logs manually using gke exec command. Please let me know if it is possible....

Answer 1

If your container is writing the logs at a specific path or on the Hostpath file system

you can use the fluentd to parse that specific application logs from path where the application container is writing. it depends on the application and your architecture.

You can also use the exclude_path to exclude

Here sharing one ref example for fluentd config

<source>
  @type tail
  path /var/log/app_name/*.log
  exclude_path ["/var/log/istio-*"]
  tag "kubernetes.*"
  refresh_interval 1s
  read_from_head true
  follow_inodes true
  <parse>
    @type json
    time_format %Y-%m-%dT%H:%M:%S.%NZ
    keep_time_key true
  </parse>
</source>

You can also set the filter as per requirement and push it further

<filter kubernetes.**>
  @type grep
  <regexp>
    key $.kubernetes.namespace_name
    pattern /^my-namespace$/
  </regexp>
  <regexp>
    key $['kubernetes']['labels']['example.com/collect']
    pattern /^yes$/
  </regexp>
</filter>