Fluentbit record_modifier how to remove key with slash?

Question

I'm trying to remove not interested kubernetes metadata from logs what I'm parsing with fluentbit v. 2.0.5.

After couple of nest filters with lift action, I'm trying to delete keys with slash in name of a key, still see it in elasticsearch

Below filter and json of a log in in elasitcsearch. For example, keys like k8s_pod_id, k8s_docker_id were deleted, but keys with '/' still sthere

`

[FILTER]
    Name record_modifier
    Match kube.*
    Remove_key k8s_pod_id
    Remove_key k8s_docker_id
    Remove_key k8s_labels_pod-template-hash
    Remove_key k8s_container_hash
    Remove_key k8s_annotations_checksum/config
    Remove_key k8s_annotations_kubernetes_io/config_source
    Remove_key k8s_annotations_kubernetes_io/psp
    Remove_key k8s_labels_app_kubernetes_io/instance

{
  "_index": "kube-2022.12.10",
  "_id": "p79h_YQBhdSEPVFK5hZr",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2022-12-10T18:52:41.244Z",
    "level": "info",
    "msg": "kubernetes_sd_configs: added targets: 4, removed targets: 4; total targets: 87",
    "time": "2022-12-10T18:52:41.24443571Z",
    "region": "eu-west-1",
    "k8s_pod_name": "victoria-metrics-agent-68b4d79567-4p5jm",
    "k8s_namespace_name": "monitoring",
    "k8s_container_name": "victoria-metrics-agent",
    "k8s_container_image": "victoriametrics/vmagent:v1.81.2",
    "k8s_annotations_checksum/config": "3c9d197fd433243999f2700713e4ab61fc01e056a97ecbfe0f782bbaa931e6a7",
    "k8s_annotations_kubernetes_io/config_seen": "2022-12-07T10:26:57.838470108Z",
    "k8s_annotations_kubernetes_io/config_source": "api",
    "k8s_annotations_kubernetes_io/psp": "eks.privileged",
    "k8s_labels_app_kubernetes_io/instance": "victoria-metrics-agent",
    "k8s_labels_app_kubernetes_io/name": "victoria-metrics-agent"
  },

`

I have tried to escape '/' with backslash , also wrap key with qouts

hey @pluk, did you maybe find the solution for this? – confiq Jul 27 '23 at 14:46 — confiq, Jul 27 '23 at 14:46

score -1 · Answer 1 · answered Dec 12 '22 at 08:21

-1

One option is to remove all those annotations using the below filter

 Remove_key annotations

answered Dec 12 '22 at 08:21

P Ekambaram

15,499
7
34
59

1

I don't need to remove all annotations, only part of them, see the config: Remove_key k8s_annotations_checksum/config Remove_key k8s_annotations_kubernetes_io/config_source Remove_key k8s_annotations_kubernetes_io/psp Remove_key k8s_labels_app_kubernetes_io/instance – pluk Dec 12 '22 at 09:09

Fluentbit record_modifier how to remove key with slash?

1 Answers1