2
  1. Is it in epoch ms or in seconds?
  2. Does it represents when the message arvived to kafka? Or sent by the kafka-client-producer? Or when it was consumed?
  3. How can I know the difference clock time between my VM to the Kafka VM in Confluent Cloud?
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Stav Alfi
  • 13,139
  • 23
  • 99
  • 171

1 Answers1

1

In most Kafka clients the timestamp is set when the producer record is generated, before serialization, and it's usually epoch ms, but this is not consistent across all clients. For example, I've noticed python using only seconds

You shouldn't need to care about clock skew in Confluent Cloud

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • 1. I'm working with KafkaJs (node lib) and it gives me nanoseconds. Maybe it is related to a C# lib that is used to produce these kafka-messages. 2. I need to calculate (in prometheus) how much time it took from produce to consume, so I'm calculating: NOW - produce-time ===>> NOW - kafkaMsg.timestamp. Do you have a better alternative? – Stav Alfi May 30 '21 at 15:04
  • It's not consistent across all clients, so you'd have to look at the source code, or set the record timestamp yourself, if possible. Time between produce and consume wouldn't depend on the time of the brokers, so just note the timezone+timestamp of the producer record and subtract from the current time of the consumer – OneCricketeer May 30 '21 at 15:08
  • But if the broker and my vm are not synced, i may receive negative results – Stav Alfi May 30 '21 at 15:11
  • 1
    Why? The broker doesn't modify the record timestamp – OneCricketeer May 30 '21 at 15:14