4

I am using this guide to run filebeat on a Kubernetes cluster. https://www.elastic.co/guide/en/beats/filebeat/master/running-on-kubernetes.html#_kubernetes_deploy_manifests

filebeat version: 6.6.0

I updated config file with:

filebeat.yml: |-
filebeat.config:
  inputs:
    # Mounted `filebeat-inputs` configmap:
    path: ${path.config}/inputs.d/*.yml
    # Reload inputs configs as they change:
    reload.enabled: false
  modules:
    path: ${path.config}/modules.d/*.yml
    # Reload module configs as they change:
    reload.enabled: false
# To enable hints based autodiscover, remove `filebeat.config.inputs` configuration and uncomment this:
#filebeat.autodiscover:
#  providers:
#    - type: kubernetes
#      hints.enabled: true

filebeat.modules:
- module: nginx
  access:
    enabled: true
    var.paths: ["/var/log/nginx/access.log*"]
- module: apache2
  access:
    enabled: true
    var.paths: ["/var/log/apache2/access.log*"]
  error:
    enabled: true
    var.paths: ["/var/log/apache2/error.log*"]

But, the logs from the PHP application (/var/log/apache2/error.log) are not being fetched by filebeat. I checked by execing into the filebeat pod and I see that apache2 and nginx modules are not enabled.

How can I set it up correctly in above yaml file.

UPDATE

I updated filebeat config file with below settings:

filebeat.autodiscover:
  providers:
    - type: kubernetes
      hints.enabled: true
      templates:
        - condition:
          config:
            - type: docker
              containers.ids:
                - "${data.kubernetes.container.id}"
              exclude_lines: ["^\\s+[\\-`('.|_]"]  # drop asciiart lines
        - condition:
            equals:
              kubernetes.labels.app: "my-apache-app"
          config:
            - module: apache2
              log:
                input:
                  type: docker
                  containers.ids:
                    - "${data.kubernetes.container.id}"

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-modules
  namespace: default
  labels:
    k8s-app: filebeat
data:
  apache2.yml: |-
    - module: apache2
      access:
    enabled: true
      error:
    enabled: true
  nginx.yml: |-
    - module: nginx
      access:
    enabled: true

Now, I am logging apache errors in /dev/stderr so that I can see it thru kubectl logs. Logs are fetching over kibana dashboard. But, apache module is still noe visible.

I tried checking with ./filebeat modules list:

Enabled:
apache2
nginx

Disabled:

Kibana Dashboard

enter image description here

RNK
  • 5,582
  • 11
  • 65
  • 133
  • 1
    There are multiple issues here that will prevent this from working. The first is that modules are setup to import from `${path.config}/inputs.d/*.yml` and not `filebeat.modules`. You need to look at how the `filebeat-inputs` ConfigMap map is setup and create one for your modules and then mount it at `/usr/share/filebeat/modules.d`. But even then, thosefile paths are not likely to be available to filebeat. Where are the log files written? Can you show the deployments for your applications? – Andy Shinn Jun 28 '19 at 22:04
  • @AndyShinn: I changed the way I am doing with `inputs.d` for `modules.d` as well. And I can see that module is now enabled. Log files are in a separate pod/container. It's a PHP application and log files are in that pod. I can see the logs by `kubectl exec -it application-pod-name -- /bin/bash` and `cat /var/log/apache2/error.log` I cannot see it when I do `kubectl logs application-pod-name`. As you said, filebeat is not fetching those log files. – RNK Jun 28 '19 at 22:33
  • 2
    Right, the files won't be available between pods. The more idiomatic way to do this is to get your application to log to stdout and then use the `docker` input type per https://raw.githubusercontent.com/elastic/beats/7.2/deploy/kubernetes/filebeat-kubernetes.yaml. – Andy Shinn Jun 28 '19 at 22:38
  • @AndyShinn: Then what's the point of having this module and giving the path of the error files? How can I configure so that `/var/log/apache2/error.log` can be linked with stdout and I can access it over `kubectl logs`? I tried by doing this `ln -sf /proc/self/fd/1 /var/log/apache2/error.log` and `ln -sf /dev/stdout /var/log/apache2/error.log` but none of the options is working. – RNK Jun 28 '19 at 22:43
  • That probably warrants a new question or at least modifying this question to be more "How to output Apache httpd logs to stdout". – Andy Shinn Jun 28 '19 at 22:45
  • @AndyShinn: Thanks. That, I will check later. But, can you please clarify that why do we need modules if we are capturing logs from stdout. – RNK Jun 28 '19 at 22:50
  • This is an advanced feature. But you can actually set a `docker` input for a module. It is documented a little bit in https://www.elastic.co/guide/en/beats/filebeat/master/filebeat-reference-yml.html. Instead of `var.paths` you can set `input` which would take the same configuration as `docker` input type. The autodiscover page at https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html talks a little bit about this and has an example for Redis. – Andy Shinn Jun 29 '19 at 21:18
  • @AndyShinn I changed the way you mentioned. Can you please see the updated question part? – RNK Jul 02 '19 at 21:26
  • If you are using GKE as provider you have to disable Stackdriver in other case filebeat won't work. – FL3SH Aug 06 '19 at 19:40

0 Answers0