0
use serde::{Deserialize, Serialize};
use std::time::Duration;
use kafka::producer::{Producer, Record, RequiredAcks};
use opentelemetry::{global, KeyValue};

#[tokio::main]
async fn main() {
    let mut producer =
        Producer::from_hosts(vec!("localhost:29092".to_owned()))
            .with_ack_timeout(Duration::from_secs(1))
            .with_required_acks(RequiredAcks::One)
            .create()
            .unwrap();

    let meter = global::meter("test");
    let counter = meter.u64_counter("my_counter").init();
    counter.add(1, &[KeyValue::new("cpu", "80")]);
    counter.add(1, &[KeyValue::new("cpu", "90")]);

    let mut buf = String::with_capacity(2);
    for _ in 0..10 {
        // producer.send(&Record::from_value("KubeStatistics", serde_json::to_string("test").unwrap())).unwrap();

        /* How to pass metrics data over here */

        buf.clear();
    }
}

In the above Rust code, I am trying to create a counter metrics using opentelemetry and sending it to my kafka.

I have created a "counter", how am I supposed to pass this counter to the Kafka. Am I supposed to write a new exporter for Kafka ? If yes, is there any references available ? OR can I get an example snippet ?

Keval Bhogayata
  • 4,422
  • 3
  • 13
  • 36
  • I think the normal way of using opentelemetry metrics would be to use some kind of backend crate like `opentelementry-kubernetes`, and not pass metric values around manually. I assume it is possible to do so, opentelementry does [define](https://github.com/open-telemetry/opentelemetry-proto/blob/main/opentelemetry/proto/metrics/v1/metrics.proto) a binary format for metrics. But I have no idea how to actually do it, and it sounds odd. I'd be curious, why did you think of trying this? – Caesar Mar 31 '22 at 12:33
  • What will be actually be consuming from Kafka? Does that expect to have any particular format? If not, then just get the value directly from querying the counter and send it however you want – OneCricketeer Mar 31 '22 at 13:26

1 Answers1

3

Generally, it's preferable to export OpenTelemetry data via OTLP (see Rust docs here) to some intermediary like an OpenTelemetry Collector, then use the Kafka Exporter to write that data to a Kafka topic.

Austin Parker
  • 323
  • 2
  • 6