0

In the official documentation of kafka-python library there is literally nothing about what value is assigned to a consumed record .timestamp attribute.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
mirakuru970
  • 1
  • 2

1 Answers1

3

The value of this Kafka timestamp depends on the log.message.timestamp.type broker configuration (see https://kafka.apache.org/documentation/#log.message.timestamp.type), which can be set to CreateTime (i.e client producer timestamp) or LogAppendTime (i.e broker timestamp).

Here's an extract of the producer client documentation that clarifies it:

If CreateTime is used by the topic, the timestamp will be the user provided timestamp or the record send time if the user did not specify a timestamp for the record. If LogAppendTime is used for the topic, the timestamp will be the Kafka broker local time when the message is appended.

Svend
  • 6,352
  • 1
  • 25
  • 38
  • Hi, thanks for clarification. Do you know by chance where/how i could change this parameter? Do i simply add it in the server.properties file? – mirakuru970 Jul 09 '22 at 13:17
  • Yes, to configure it globally for all topics of the Kafka cluster, you simply add it to the `server.properties` file. You can also set it for each specific topic by adding `--config "message.timestamp.type=..."` when you create it (or update it with `kafka-topics.sh`) – Svend Jul 10 '22 at 08:30
  • happy to help :) (please upvote and accept the answer if you find it useful) – Svend Jul 11 '22 at 11:53