Stack Overflow
Stack Overflow

Questions tagged [ksqldb]

ksqlDB: an event streaming database for Apache Kafka.

ksqlDB is a source-available event streaming database for Apache Kafka® developed by Confluent. It is distributed, scalable, reliable, and real-time. ksqlDB combines the power of real-time stream processing with the approachable feel of a relational database through a familiar, lightweight SQL syntax.

You can download ksqlDB at https://github.com/confluentinc/ksql

There is a quickstart at: https://ksqldb.io/quickstart.html

926 questions
2
votes
1 answer

How do i set a key field when creating table in KSQL?

I would like to ask can i set the key field when creating table? I have created a table by aggregation as below: CREATE TABLE withdrawal_less_than_5min AS SELECT executedate, status, count(*) as count FROM…
kurapika
  • 166
  • 1
  • 12
2
votes
2 answers

How to get partition key or other columns when "group by" is used in KSQL?

Basically I am trying get all columns when I am using group by expression in my query. Creating stream from a topic CREATE STREAM events_stream \ ( \ account VARCHAR, \ event_id VARCHAR, \ user_name VARCHAR, \ event_name…
Shakeel
  • 1,869
  • 15
  • 23
2
votes
2 answers

How to start ksql in windows?

how to start ksql in windows environment? we have installed confluent kafka using zip format. we have not found ksql-server-start.bat file in confluent-home/bin/windows directory, but we observed a file with same name under confluent-home/bin/…
sai krishna
  • 151
  • 2
  • 11
2
votes
1 answer

setup and Run KSQL on windows machine

I was just wondering if it is possible to run KSQL on windows machine. I downloaded from https://www.confluent.io/product/ksql/ and not sure how to set up and run. Thanks MR
user2726975
  • 1,285
  • 3
  • 17
  • 26
2
votes
1 answer

Get list of all Kafka KTables

How can I query the list of all KTables and KStreams that Kafka cluster has? Something like select * from all streams
Sergei Ledvanov
  • 2,131
  • 6
  • 29
  • 52
2
votes
2 answers

What is the simplest way to sync a Kafka KTable to a SQL database?

I have used KSQL to create a stream and an aggregated table off that stream. { "ksql":"DROP Stream IF EXISTS StreamLegacyNames; DROP Stream IF EXISTS StreamLegacy; CREATE Stream StreamLegacy (payload STRUCT)WITH…
John Bowyer
  • 1,213
  • 1
  • 15
  • 26
2
votes
1 answer

KTABLE not detecting same keys. ( Inserts a record instead of Update)

Use Case Goal is to identify the incoming events/rows to check if it is a new row or an update. New Row will go to a different topic and update row will go to a different topic. Approach: Have a lookup table (KTABLE) and do two join operations …
Krishnakanth Jc
  • 183
  • 1
  • 14
2
votes
1 answer

KSQL stream from topic with heterogeneous JSON structures

Is there a way to create a stream from a topic specifying that the entire record should be considered a VARCHAR so that I can create streams from it using extractjsonfield()? Sample records might looks something like: { "Header": { "RecType":…
Mike Cargal
  • 6,610
  • 3
  • 21
  • 27
2
votes
1 answer

Kafka Rest API KSQL Query waits forever and hangs

I am using Kafka REST API issued from curl POST to ksql If I do not use LIMIT20, it hangs. Also If I use it to query on a table again it hangs. I am running this from inside a python script Here I am querying from in between rowtime bcoz I can't get…
Nanobrains
  • 275
  • 1
  • 3
  • 11
2
votes
4 answers

How to DROP a table or stream in KSQL when topic was dropped first

Using KSQL (Confluent: Version: 5.0.1) I'm able to drop a table / stream normally (using DROP [TABLE|STREAM]) when the linked topic exist and when it is registered (Registered=true). However, if the topic is dropped first (Registered=false)…
Matthias
  • 178
  • 2
  • 6
2
votes
1 answer

How to fetch Kafka source connector schema based on connector name

I am using Confluent JDBC Kafka connector to publish messages into topic. The source connector will send data to topic along with schema on each poll. I want to retrieve this schema. Is it possible? How? Can anyone suggest me My intention is to…
Achaius
  • 5,904
  • 21
  • 65
  • 122
2
votes
1 answer

ksql json fieldname with a dash (-)

One of the field names in my JSON is "User-Agent". KSQL doesn't like the dash when I try to create a stream with that field name. I've tried single, double quotes, escaping it various ways--no luck. How do I get that field from json in a topic to…
medloh
  • 941
  • 12
  • 33
2
votes
1 answer

Can you run KSQL from a remote host?

I have confluent-ksql-server running on one of the nodes of my cluster . Can we make the ksql to be connected by a specific host/machine outside the kafka cluster ? PS- this is to provide ksql access to developers Thanks !
Tony
  • 671
  • 1
  • 9
  • 29
2
votes
2 answers

KSQL table get old and new value

Is it possible in KSQL to stream out the old and new values from a table? We'd like to use a table as a store of values and when one changes stream out a "reversal" value which is the previous one, tagged in some way, and the new value so that we…
Ben
  • 105
  • 3
2
votes
1 answer

How many connections/subscriptions can handle Kafka KSQL?

We are trying to migrate Kafka KSQL to our system and would like to share some problems that we could not solve during the process. We have 3 Kafka nodes in our cluster, each of the servers has: 8 CORE 50G+ RAM 100G ssd On each server we…
Rafik Avtoyan
  • 413
  • 3
  • 12
1 2 3
…
61 62