Stack Overflow
Stack Overflow

Questions tagged [ksqldb]

ksqlDB: an event streaming database for Apache Kafka.

ksqlDB is a source-available event streaming database for Apache Kafka® developed by Confluent. It is distributed, scalable, reliable, and real-time. ksqlDB combines the power of real-time stream processing with the approachable feel of a relational database through a familiar, lightweight SQL syntax.

You can download ksqlDB at https://github.com/confluentinc/ksql

There is a quickstart at: https://ksqldb.io/quickstart.html

926 questions
4
votes
1 answer

Kafka Connector to IBM DB2

I'm currently working in a Mainframe Technology where we store the data in IBM DB2. We got a new requirement to use scalable process to migrate the data to a new messaging platform including new database. For that we have identified Kafka is a…
Prasad Kintali
  • 41
  • 1
  • 3
4
votes
1 answer

How to join a KSQL table and a stream on a non row key column

I am using community edition of confluent Platform version 5.4.1. I did not find any CLI command to print the KSQL Server version but when I enter KSQL what I get to see can be found in the attached screenshot. I have a geofence table - CREATE…
Nitesh Ratnaparkhe
  • 605
  • 5
  • 20
4
votes
1 answer

Do we have a option to get data in KSQL streams from specific time-period/Timestamp

I know , in KSQL we can set offset to earliest or latest But can we get data from specific time period i.e I need to get data inserted to a topic from 06-May-2020 ?
Raghava Reddy
  • 313
  • 4
  • 11
4
votes
3 answers

Is it possible to get the latest value for a message key from kafka messages

Suppose I have different values for a same message key. For example: { userid: 1, email: user123@xyz.com } { userid: 1, email: user456@xyz.com } { userid: 1, email: user789@xyz.com } In this above case I want only the latest value updated by the…
Wilmetta
  • 51
  • 1
  • 3
4
votes
1 answer

How to connect to external Oracle database in Docker

I am trying to connect to Oracle database which is outside of docker container. I am new to Docker and not sure how to achieve the same. On my local machine, I can connect to specific database by providing Host, Service name and Port or just by…
Praveenks
  • 1,436
  • 9
  • 40
  • 79
4
votes
1 answer

Confluent Platform KSQL in headless mode

I've read through KSQL deployment options here https://www.confluent.jp/blog/deep-dive-ksql-deployment-options/. So it is recommended to use headless KSQL for production deployment. But I have not found any hints on how I can stop/change queries…
ALincoln
  • 431
  • 1
  • 4
  • 12
4
votes
1 answer

KSQL streams - Get data from Array of Struct

My JSON looks like: { "Obj1": { "a": "abc", "b": "def", "c": "ghi" }, "ArrayObj": [ { "key1": "1", "Key2": "2", "Key3": "3", }, { "key1": "4", "Key2": "5", "Key3": "6", }, …
Raghava Reddy
  • 313
  • 4
  • 11
4
votes
1 answer

How can I write KSQL queries in nodejs?

Could you provide me with a code sample in node js that runs KSQL queries on Kafka ? F.e. I would like to create and then retrieve a simple table.
Cap Barracudas
  • 2,347
  • 4
  • 23
  • 54
4
votes
1 answer

Can you specify a nested key for a table in KSQL?

I'm trying to create a table from a topic with an object key, but KSQL didn't recognize the variable. Any solution for this? create table csvexporttable (HEADER STRUCT, body…
wahib
  • 373
  • 1
  • 2
  • 12
4
votes
1 answer

KSQL STRUCT Fields For TIMESTAMP

is there are a way to make this work JSON DATA "Header": { "StoreID": 10225, "BusinessDate": "2019-05-03", "PeriodBusinessDate": "2019-05-03", "ProcessMode": "Partial" } I try this but is give me : No column with the provided…
wahib
  • 373
  • 1
  • 2
  • 12
4
votes
2 answers

Get latest values from a topic on consumer start, then continue normally

We have a Kafka producer that produces keyed messages in a very high frequency to topics whose retention time = 10 hours. These messages are real-time updates and the used key is the ID of the element whose value has changed. So the topic is acting…
Samer
  • 43
  • 4
4
votes
3 answers

Can I write sub query in KSQL?

I am new to ksql and have been using mysql for the longest time. I would like to know is there anyway to have subqueries in KSQL? This query works without any problem : SELECT a.executedate, count(a.pno), sum(a.amount)…
kurapika
  • 166
  • 1
  • 12
4
votes
2 answers

KSQL - Determining When a Table Is Loaded

How can I determine when KSQL has fully loaded my data from a Kafka topic into my table? GOAL: Take 2 Kafka topics, join them and write the results to a new Kafka topic. EXAMPLE: I am using Ksql's Rest API to issue the following commands. CREATE…
Hoppy
  • 136
  • 8
4
votes
1 answer

Update ksql stream with new topic schema

I write avro messages into kafka topic using schema registry. Then created stream based on the topic. The scream created with current schema. Then I add new field to the schema. The schema register updated, it's OK, but the stream stay with the…
Hersh
  • 630
  • 4
  • 16
4
votes
1 answer

KSQL table not showing data but Stream with same structure returning data

I have created a table in KSQL, while querying it's not returning any data. Then I created a stream on the same topic with same structure and I am able to query the data. What am I missing here. I need this as a table for joining with a…
Sniper007
  • 65
  • 6
1
2
3
…
61 62