I had a few questions about GoLang Kafka producer using librdkafka - These are based on the logs I am seeing in the producer log when I set debug: all. The producer spends some time in building message sets once the batch threshold is recached or linger.ms is passed. However, what happens almost all the time is - messages are moved from partition queue to xmit queue. I was trying to get some documentation on it, but could not find much, so wanted to check if I can get some help on the stack. My questions are following -
a) Does the application produce calls write to a partition specific queue(s)?
b) Are there one xmit queue and one partition queue per partition?
c) What triggers a transfer from partition queue to xmit queue? and why do we need two queues?
d) When the Kafka producer is creating messagesets for a partition - does it block all operations for the partition? (Like moving messages from partition queue to xmit queue)? In short, when message sets are being built for a partition, can new messages sneak in the xmit queue? Is it blocked?
e) How many threads work for creating message sets? Is it one per producer or one per partition?
