1

I'm having kafka cluster deployed in openshift in different namespace also opentelemetry is delpoyed in another namespace.

This is my yaml file for opentelemetry

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: demo-otlcol-otlp
  namespace: otlcol-demo
spec:
  config: |
    receivers:
      otlp:
        protocols:
          grpc:
          http:
    processors:
      batch:

    exporters:
      logging:
        loglevel: info

      kafka:
        brokers: 
          - dev-kafka-kafka-brokers.kafka.svc.cluster.local:9092
        protocol_version: 3.4.0

    service:
      pipelines:
        traces:
          receivers: [otlp]
          processors: [batch]
          exporters: [logging,kafka]
  mode: daemonset
  resources: {}
  targetAllocator: {}

The error i'm getting is,

Error: failed to get config: cannot unmarshal the configuration: 1 error(s) decoding:
* error decoding 'exporters': unknown type: "kafka" for id: "kafka" (valid values: [logging otlp otlphttp jaeger])
2023/05/31 09:24:46 collector server run finished with error: failed to get config: cannot unmarshal the configuration: 1 error(s) decoding:
* error decoding 'exporters': unknown type: "kafka" for id: "kafka" (valid values: [logging otlp otlphttp jaeger])

Can anyone explain how to use kafka in opentelemetry exporters

Mugi Coder
  • 11
  • 4
  • The error message means that the Kafka exporter is not supported in your collector distribution. You'd need to check back with Red Hat about details. I'm not familiar with their OpenTelemetry distro. Do you know what collector image the OpenShift setup pulls for the collector? – Michael Hausenblas May 31 '23 at 09:58

1 Answers1

2

I can confirm what Michael said. Kafka is atm not supported and not part of Red Hats otel distro. Here you see what otel components will be productized in the next release. To bypass your issue, you can create your own distro. The easiest way would be to extend Red Hats manifest.yaml with the Kafka exporter. For testing, you can also use the contrib image offered by the OpenTelemetry community. There is a section in the OTEL CRD to overwrite the collector image.

Example:

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: simplest
spec:
  image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.78.0
  config: ...
Frzifus
  • 21
  • 2