Stack Overflow
Stack Overflow

Questions tagged [kafka-topic]

202 questions
0
votes
1 answer

GDPR compliance via Kafka compacted topic

I would like to ask you some questions about apache kafka and compacted topics. We want to provide some PII Data over a kafka compacted topic. We want to delete the data on this topic via tombstone. There are currently multiple questions where we…
holzleube
  • 159
  • 1
  • 2
  • 10
0
votes
1 answer

How to update data in Kafka/Kafka stream?

Lets suppose there is Kafka topic orders. Data is stored in JSON format: { "order_id": 1, "status": 1 } Status defines status of order (pending - 1, completed - 2). How to change it on completed when it is finished? As I know Kafka topic…
user13763587
0
votes
1 answer

Consumer_failed_message in kafka stream: Records not pushed from topic

I have a flow where from IBM mainframe IIDR, I am sending records to Kafka topic. The value_format of the message coming to Kafka topic is AVRO and the key is in AVRO format too. The records are pushed into the Kafka topic. I have a stream…
Sanjay Nayak
  • 79
  • 2
  • 12
0
votes
3 answers

Kafka Topic Creation with --bootstrap-server gives timeout Exception (kafka version 2.5)

When trying to create topic using --bootstrap-server, I am getting exception "Error while executing Kafka topic command: Timed out waiting for a node" :- kafka-topics --bootstrap-server localhost:9092 --topic boottopic --replication-factor 3…
springcloudlearner
  • 457
  • 3
  • 20
0
votes
1 answer

What happens in Kafka if you create a new partition for a topic?

I read on some guides online that if you are using key ordering the new partition will eventually break that ordering, I really can't see how. Is this really what happens ?
GionJh
  • 2,742
  • 2
  • 29
  • 68
0
votes
3 answers

Counting Messages in Kafka Topics

I am trying to understand how to keep track of message ingestion with Kafka. The workflow we follow right now is purging all the messages in the topics, and then we re-ingest with code changes. I need to know how successful those code changes are.…
rysisnice
  • 9
  • 1
  • 2
0
votes
1 answer

Apache-Kafka: Issue in creating topic using windows cmd

I am new to Apache-Kafka.I am using: zookeeper-3.6.0 kafka-2.13-2.4.1 Windows-7 Earlier i was able to create and list topics on the same machine. After i restarted my system, i am unable to create and list topics. I am getting below…
user2152156
  • 1
  • 2
0
votes
1 answer

Spark Structured Streaming- Is it possible to write the offset twice

I am using spark structured streaming to consume data from kafka topic and write the data into another kafka sink. I want to store the offset twice - once when reading from the topic and stire the offset. Secondly- when writing the data onto…
Shivani
  • 1
0
votes
1 answer

Monitoring Kafka Topic by Jmeter

I am trying to monitor a Kafka topic through jmeter. I am sending 100 messages to the kafka topic though an external tool and the messages are passing perfectly as I have checked in the consumer window. Now I want to monitor the kafka topic using…
Afsana Khan
  • 57
  • 1
  • 2
  • 10
0
votes
1 answer

Topic deletion from Zookeeper

I have deleted topic directly from Zookeeper using the below command and did not execute deletion from Kafka before: zookeeper-shell.sh localhost:2181 rmr /brokers/topics/ Now what I see is that the topic shows up in the log.dirs of at…
Preethi Jahnavi
  • 187
  • 1
  • 1
  • 10
0
votes
1 answer

How to get the serialization format of a Kafka topic?

Apart from using Confluent Schema Registry, is there a way (via the built-in CLI tools) to view the serialization format of a topic's key and value?
Alexander Popov
  • 23,073
  • 19
  • 91
  • 130
0
votes
2 answers

number of kafka topic partitons and number of different keys in data

I want to use key/value pattern writing to Kafka in order to keep the same order of data writing while reading it. My question is should the number of partitions in the topic be equal to the number of different keys in the incoming data. I already…
scalacode
  • 1,096
  • 1
  • 16
  • 38
0
votes
1 answer

kafka producer - how to know which key will go to what partition?

I am aware of kafka console producer command and it is quite handy. I want to know is there a way to know for sure which key will go to which partition when we have key as well? Assuming we have 10 partitions in kafka topic, how will producer decide…
CuriousMind
  • 8,301
  • 22
  • 65
  • 134
0
votes
1 answer

kafka is not electiong new leader after 2 nodes are down

we test scalein in enviorment with 3 kafka nodes each one has a broker and zookeeper service on in the replica.factor=2 after the service went down on 2 nodes the leader of the partition stayed the old one and didnt movve to the only node that is…
NoamiA
  • 521
  • 4
  • 19
0
votes
1 answer

Streaming database data to Kafka topic without using a connector

I have a use case where I have to push all my MySQL database data to a Kafka topic. Now, I know I can get this up and running using a Kafka connector, but I want to understand how it all works internally without using a connector. In my spring boot…
Nithin Prasad
  • 554
  • 1
  • 9
  • 22
1 2 3
…
13 14