Stack Overflow
Stack Overflow

Questions tagged [ksqldb]

ksqlDB: an event streaming database for Apache Kafka.

ksqlDB is a source-available event streaming database for Apache Kafka® developed by Confluent. It is distributed, scalable, reliable, and real-time. ksqlDB combines the power of real-time stream processing with the approachable feel of a relational database through a familiar, lightweight SQL syntax.

You can download ksqlDB at https://github.com/confluentinc/ksql

There is a quickstart at: https://ksqldb.io/quickstart.html

926 questions
2
votes
1 answer

KSQL create table with multi-column aggregation

So essentially I'd like to group by two columns, like this: CREATE TABLE foo AS SELECT a, b, SUM(a) FROM whatever GROUP BY a, b whatever is a stream in Kafka format. When I issue the command, ksql returns: Key format does not support…
Richard Steinbrecht
  • 161
  • 2
  • 17
2
votes
1 answer

How to create a table in KAFKA using Confluent KSQLDB with a console / command line / cli?

I am trying to create a table in KAFKA using Confluent's KSQLDB. I have easily use the UX provided by Confluent but I am trying to automate this process and hence I need a command line scipting capability to execute this task. I have been able to…
Eric Broda
  • 6,701
  • 6
  • 48
  • 72
2
votes
0 answers

KSQL: EXPLODE in WHERE expression

I have an avro input_stream something like this: VERSION STRING, ID STRUCT< NAME STRING, STATUS STRING >, ROLES ARRAY< STRUCT< DESC STRING, TYPE STRING > > and I want to falatten it and do some filtering as well. This is…
user213546
  • 341
  • 1
  • 7
2
votes
0 answers

I am not getting results from ksql StreamQuery integrated with java. when I am printing log for client is showing not completed

I am using confluent kafka version 6.o downloaded from https://www.confluent.io/download/ I am referring https://docs.ksqldb.io/en/latest/developer-guide/ksqldb-clients/java-client/ acritical . https://www.youtube.com/watch?v=85udigshlNI with java…
raj
  • 31
  • 4
2
votes
1 answer

Is there a KSQL statement to update values in table?

I have a stream of data in topic that should be treated as ksql table (only last value of given key matters) and this data is about updates of some data's specific fields in other topic. Is there any way in KSQLDB to process stream that update…
Ferdynand Kiepski
  • 181
  • 3
  • 11
2
votes
1 answer

Create KSQL stream with default values for a column?

Is it possible to create a stream from Kafka topic with default values or fixed value for a few columns which are not present in the topic ? create stream test ( Balance BIGINT, Name STRING <---- want to specify a fix value ("NA" or "no name") for…
Nan-7
  • 71
  • 1
  • 8
2
votes
1 answer

Is there any version control tool for ksqldb queries?

I'm new to ksqldb, and I’m looking for a version control tool for ksqldb like flyway migrations for RDMBS in java. I read the page below and understand the way of upgrading queries, but I'd like to operate to control versions of streams and tables…
shota hizawa
  • 105
  • 2
  • 8
2
votes
0 answers

Is it possible to use custom deserializer in KSQLDB

Is there a way to configure a custom avro deserializer in KSQL, without storing schema in schema registry. I want to store schema in a file system rather than schema registry.
tzgar
  • 21
  • 1
2
votes
1 answer

"Transactional Id authorization failed" when trying to create a stream in Ksql

I have a KSQL server set up in OpenShift and connected to the on-premise Cloudera Kafka cluster (CDH6) (kerberized and with SSL). When I do a list topics- or print-command everything works fine. But as soon as I want to create a stream, I get the…
aireducs
  • 61
  • 1
  • 7
2
votes
0 answers

ksqlDB: Best way to create Tables from a Debezium source topics?

I would like to create Tables in ksqlDB from Debezium source topics, with the ultimate aim of performing a left join on these tables and efficiently outputting materialized views to a downstream database using the JDBC sink connector. The Debezium…
Zejji Zejji
  • 497
  • 3
  • 14
2
votes
1 answer

KSQL for distinct values

How do I check if there are more than two records in a stream that are from the same userId but with a different productId. Lets say we have stream (uid,pid,price) I want to emit if from the same uid there are multiple different pid. What is the…
Average Bear
  • 191
  • 1
  • 3
  • 13
2
votes
1 answer

How can I perform an aggregation on top of an aggregation over a time window in Kafka with ksql

I have a bunch of firewall data. I would like to: A) sum the bytes per IP per hour, and then B) calculate the min and max sums across all IPs in that hour I have been able to do A in Kafka, however, I cannot figure out how to do B. I've been poring…
jojo
  • 83
  • 7
2
votes
0 answers

KSQL Java Client app Received 404 response from server:

I have written KSQL Client Java app to fetch the topic message details from KSQL Tables. This is the code snippet to read the topic messages from KSQL table. But when i run this program getting below error. Please let me know how to resolve this…
Ramesh
  • 21
  • 1
2
votes
1 answer

How Create KSQLdb Stream fields from nested JSON Object

I have a topic over which i am sending json in the following format: { "schema": { "type": "string", "optional": true }, "payload": “CustomerData{version='1', customerId=‘76813432’, phone=‘76813432’}” } and I would like to create…
Roger Alkins
  • 125
  • 11
2
votes
1 answer

data in ksqlDB table are persistent?

We are using ksqlDB 0.11.0, and we want to persistently store data in ksqlDB's table. But we set log.retention.hours to 72, so i confused whether the table store the recent 72 hours data? create stream stream_ori_sacmes_pack_detail with (kafka_topic…
Jun Zhou
  • 33
  • 1
  • 4
1 2 3
…
61 62