Stack Overflow
Stack Overflow

Questions tagged [ksqldb]

ksqlDB: an event streaming database for Apache Kafka.

ksqlDB is a source-available event streaming database for Apache Kafka® developed by Confluent. It is distributed, scalable, reliable, and real-time. ksqlDB combines the power of real-time stream processing with the approachable feel of a relational database through a familiar, lightweight SQL syntax.

You can download ksqlDB at https://github.com/confluentinc/ksql

There is a quickstart at: https://ksqldb.io/quickstart.html

926 questions
2
votes
2 answers

Kafka to Snowflake connection issue

I am trying to connect from local standalone Confluent Kafka topics to Snowflake tables. I am using the following connector config via ksqldb. CREATE SINK CONNECTOR `snowflake_sink`…
Umesh K
  • 13,436
  • 25
  • 87
  • 129
2
votes
2 answers

Create table from topic and got some serialization exception about Size of data received by LongDeserializer is not 8

Environments (Docker): # 5.5.1 image: confluentinc/cp-zookeeper:latest # 2.13-2.6.0 image: wurstmeister/kafka:latest # 5.5.1 image: confluentinc/cp-schema-registry:latest # 5.5.1 image: confluentinc/cp-kafka-connect:latest # 0.11.0 image:…
chuck
  • 21
  • 4
2
votes
1 answer

ksql what is the difference between the primary key and with KEY

ksql. what is the difference between the primary key and with KEY then create ktable ? should they be applied simultaneously?
padavan
  • 714
  • 8
  • 22
2
votes
1 answer

Should KSQL tables be showing multiple rows per key for aggregates?

My understanding of KSQL tables is that they show an "as is" view of our data rather than all the data. So if I have a simple aggregating query and I SELECT from my table, I should see the data as it is at this point in time. My data…
NotNowJohn
  • 93
  • 8
2
votes
1 answer

KSQL Windowed Aggregation Stream, Session ending

I am grouping events coming from a kafka topic by one of its properties and over time using the KSQL Windowed Aggregation, specifically the Session Window. I have been able to create a stream of "session start signals" as described in this…
Filippo Vitale
  • 7,597
  • 3
  • 58
  • 64
2
votes
1 answer

DataException: Key must be a struct or map

I am trying to configure the Confluent Cassandra Sink Connector with Kafka Connect using KSQLdb. CREATE SINK CONNECTOR cassandra WITH( "name" = 'CASSANDRA', "connector.class" = 'io.confluent.connect.cassandra.CassandraSinkConnector', …
4it med
  • 181
  • 1
  • 7
2
votes
2 answers

JSON Array to JSON object using KSTREAM or KSQL

I have data coming into Kafka in the below format. {"WHS":[{"Character Set":"UTF-8","action":"finished","Update-Date-Time":"2020-04-11…
viprud
  • 25
  • 9
2
votes
1 answer

Kafka-Ksql Rekeying on stream results in data disappearing after few min

The rekeying works fine in the following scenarios 1. Base topic to stream then rekey - Good 2. Table to stream then rekey - Good But when trying with Table - Table join results in new table, create stream on that resulting table, then rekey stream…
Kotharu
  • 21
  • 2
2
votes
0 answers

why No new messages seen in ksql even messages exist?

I create table in ksql like this: CREATE TABLE ORDERS_T (id BIGINT, product VARCHAR, quantity BIGINT, price BIGINT) WITH (KAFKA_TOPIC='orders', VALUE_FORMAT='JSON'); In kafka I ve "orders" topic And when I select query tab in ksql even total…
CompEng
  • 7,161
  • 16
  • 68
  • 122
2
votes
1 answer

KSQL Hopping Window : any way to get only one record in response?

We are using KSQL to perform some aggregations / filtering on a real time data. One of the use case we have is, we need to perform some operation on last N days of a particular activity, this would be continuous operation. So this needs to be…
user12818876
  • 21
  • 2
2
votes
2 answers

Windows subsystem for Linux : Command Not Found Error

I have installed windows subsystem for Linux to run Ubuntu 16.04 on my windows 10 home platform. I have extracted all required directories to run KSQL on this platform. Now, when I am trying to run any command after navigating to the bin folder.…
Praveenks
  • 1,436
  • 9
  • 40
  • 79
2
votes
1 answer

Can I convert from Table to Stream in KSQL?

I am working in the kafka with KSQL. I would like to find out the last row within 5 min in different DEV_NAME(ROWKEY). Therefore, I have created the stream and aggregated table for further joining. By below KSQL, I have created the table for finding…
Hong
  • 365
  • 4
  • 14
2
votes
1 answer

Specify KSQL Stream Subject names explicitly

I have two KSQL topics my-topic-1 and my-topic-2, with messages serialised via AVRO. For historical reasons, the my-topic-1 schema is not in the recommended topic-value format, but is instead my-custom-subject-name. I want to move records from one…
Yair Halberstadt
  • 5,733
  • 28
  • 60
2
votes
1 answer

KSQL says it expects existing topic to have 2 partitions (topic has 1),

I am facing below error when trying to create Stream in KSQL from an existing Kafka topic. io.confluent.ksql.exception.KafkaTopicExistsException: A Kafka topic with the name 'test-data' already exists, with different partition/replica configuration…
srini
  • 21
  • 3
2
votes
2 answers

KSQL connection issues with kafka

Provide details of the setup you're running Let us know what version of KSQL you're running, what OS, etc, KSQL version: confluent-ksql-5.3.1-1. Outline your question Ask you question! KSQL not able to connect to the kafka, resulting in timeout…
gangadhar r
  • 111
  • 1
  • 9
1 2 3
…
61 62