Stack Overflow
Stack Overflow

Questions tagged [ktable]

A KTable is continuously updating materialized view.

The KTable concepts is used by Kafka Streams and ksqlDB (both also used the concept of a KStream). A KTable is a materialized view that that continuously updated.

The data source of a KTable can either be a topic in Apache Kafka (i.e., each record in the topic is "upserted" into the KTable) or the KTable stores the result of a continuous query over an input KStream or KTable.

72 questions
0
votes
1 answer

How to Print Global Ktable in scala for kafka?

I am trying to see how GlobalKtable works in kafka and for that i am trying to write sample code. I have created globalKtable but i want to see that too I have tried peek function but its not available, now i am trying by view but its giving compile…
Wajih
  • 93
  • 1
  • 10
0
votes
1 answer

Reprocess enriched kstream data on insert/updates of KTable

Here is a hypothetical but analogous scenario to what I am trying to achieve using Kafka Streams. I have streaming data, sales, that I want to enrich with infrequently changing lookup data, say users & items, which I am planning to create KTable…
Vishal Shukla
  • 2,848
  • 2
  • 17
  • 19
0
votes
1 answer

What is the most efficient way to know that a Kafka event is visible in a K-Table?

We use Kafka topics as both events and a repository. Using the kafka-streams API we define a simple K-Table that represents all the events in the topic. In our use case we publish events to the topic and subsequently reference the K-Table as the…
Gerardo Villeda
  • 3
  • 1
0
votes
2 answers

Remove duplicates(considering key as flight number) and get only the latest record w.r.t timestamp

Can I upsert data, which is avro schema in Kafka? I want to pick record from topic and then, filter the flights (eg: consider two records have same flight number. We need to pick only latest one by considering time stamp as mentioned in Avro…
subrahmanyam b
  • 11
  • 3
0
votes
1 answer

How to reduce pair of records to 1 in KStream to KTable

This is the input Kafka topic which contains ConnectionEvent: ConnectionEvent("John", "123", "CONNECTED") ConnectionEvent("John", "123", "DISCONNECTED") ConnectionEvent("Anna", "222", "CONNECTED") ConnectionEvent("Rohan", "334",…
cppcoder
  • 22,227
  • 6
  • 56
  • 81
0
votes
1 answer

Maintain separate KTable

I have a topic which contains events of user connection and disconnection for each session. I would like to use Kafka stream to process this topic and update KTable based on some condition. Each record cannot update KTable. So I need to process…
cppcoder
  • 22,227
  • 6
  • 56
  • 81
0
votes
0 answers

How to make leftJoin for KTable Many To One?

I have two Ktable and I need doing leftJoin. This left join mast be ManyToOne. I have next code for leftJoin: KTable right = createKTable("topic_1"); KTable left =…
Леонид Дубравский
  • 123
  • 2
  • 10
0
votes
1 answer

Calculating Memory Footprint of GlobalKTables

I have a Kafka Streams application with GlobalKTables. I would like to compute the memory footprint of the same. The data in the underlying Kafka topics are compressed using SNAPPY. I couldn't find information about data stored on Ktables. Are…
guru
  • 409
  • 4
  • 21
0
votes
1 answer

Can I use topic more than once in Kafka Stream Topology?

Let's just hypothetically say groupby function was not available in kafka streams. Can I do the below to get the word count and build a KTable on top of it? Please note that i use "word-count-topic" twice in the topology. I have a use case where I…
Sarthak Sarathi
  • 27
  • 1
  • 7
0
votes
1 answer

Apache Kafka - KStream and KTable hard disk space requirements

I am trying to, better, understand what happens in the level of resources when you create a KStream and a KTable. Below, I wil mention some conclusions that I have come to, as I understand them (feel free to correct me). Firstly, every topic has a…
Chris_Gav
  • 112
  • 2
  • 8
0
votes
1 answer

Apache Kafka - Implementing a KTable

I am new to Kafka Streams API and I am trying to create a KTable. I have an input topic: s-order-topic, which is a json format message, as shown below. { "current_ts": "2019-12-24 13:16:40.316952", "primary_keys": ["ID"], "before": null, …
Chris_Gav
  • 112
  • 2
  • 8
0
votes
0 answers

KTable-KStream LeftJoin impacting the performance. Is there any alertnative to it

I have a use-case in which I am receiving the tweets on a topic, and user-details on other topic. I need to find username from the user-details and set it to tweets. Using following code I am able to get the expected outcome. KStream…
Suchita
  • 55
  • 1
  • 9
1 2 3 4
5