Stack Overflow
Stack Overflow

Questions tagged [ksqldb]

ksqlDB: an event streaming database for Apache Kafka.

ksqlDB is a source-available event streaming database for Apache Kafka® developed by Confluent. It is distributed, scalable, reliable, and real-time. ksqlDB combines the power of real-time stream processing with the approachable feel of a relational database through a familiar, lightweight SQL syntax.

You can download ksqlDB at https://github.com/confluentinc/ksql

There is a quickstart at: https://ksqldb.io/quickstart.html

926 questions
3
votes
2 answers

Issue to access KSQL CLI on Docker

I'm trying access the KSQL CLI for first time on Windows terminal but I got the error: Unable to find image 'cp-ksql-cli:5.3.1' locally To do it I get docker images $ docker images REPOSITORY TAG …
Augusto
  • 3,825
  • 9
  • 45
  • 93
3
votes
0 answers

Data not syncing from mysql to elastic search after processing through Kafka

We are trying to send data from MySQL to elastic(ETL) though Kafka. In MySQL we have multiple tables which we need to aggregate in specific format than we can send it to elastic search. For that we used debezium to connect with Mysql and elastic and…
Rahul Kumawat
  • 165
  • 1
  • 11
3
votes
0 answers

How do I sync the geospatial data from kafka topic to postgres?

I have been trying to sync the geospatial data from kafka topic to postgres db with postgis enabled. So the geospatial data is in form of array [lat, lng] at the moment in kafka topic. I want to sync this field as (Point(lat,lng), SRID) in postgres…
py_trainee
  • 129
  • 8
3
votes
0 answers

Table-Table Join duplicate entries

we are using kafka in production and I try to push the adoption and usage of KSQL in the same direction. But I already failed with one simple table-table join. I’ve tried with our production data first and ran in an issue. So I thought I missed…
hammi
  • 161
  • 1
  • 8
3
votes
3 answers

Stream join example with Apache Kafka?

I was looking for an example using Kafka Streams on how to do this sort of thing, i.e. join a customers table with a addresses table and sink the data to ES:- Customers +------+------------+----------------+-----------------------+ | id |…
Gavin Gilmour
  • 6,833
  • 4
  • 40
  • 44
3
votes
1 answer

ksql rest api creation and getting output in json

I am unable to fetch the data from a katable to a pojo class ouput I have tried to call rest end point of confluent kafka but I am uanble to put the output in a pojo class. @PostMapping("/QueryForEquipment") @Consumes("application/json") …
Neha Chaturvedi
  • 41
  • 2
3
votes
1 answer

How to preserve the field case sensitivity in ksql while creating the stream form schema registry by specifying the topic name

created the stream using schema-registry without specifying the filed name refer below CREATE STREAM sample_avro WITH (KAFKA_TOPIC='test',VALUE_FORMAT='AVRO'); Topic name test contains a few fields and values when described the topic all the…
hepzi
  • 435
  • 4
  • 17
3
votes
1 answer

Create KSQL table with ROWKEY same as Kafka topic message key

I'm creating a KTable from a topic with JSON value format, the producers of the topic also include a key for each message at Kafka level and JSON message itself contains a JSON property called key which I use as ROWKEY in the following…
bithavoc
  • 1,539
  • 15
  • 20
3
votes
2 answers

What's the way of running KSQL from spring boot app

I have a spring boot application that is connected to a kafka cluster. How can I run KSQL from java code?
Sergei Ledvanov
  • 2,131
  • 6
  • 29
  • 52
3
votes
1 answer

How to write the query to extract all the field value along with condition satisfied values in KSQL

Created stream with the following field CREATE STREAM pageviews_original_string(view_time string, user_id varchar, pageid varchar) WITH (kafka_topic='pageviews',value_format='DELIMITED',KEY='pageid'); Changed the pageid into the uppercase along…
hepzi
  • 435
  • 4
  • 17
3
votes
2 answers

Data in ksql table not being persistent

We are using confluent platform on ubuntu. We have simple JSON data being sent through a cURL request to kafka-rest server on kafka topic named "UE_Context". A kafka stream named "UE_CONTEXT_STREAM" is created for this topic with the below…
Swaru
  • 144
  • 2
  • 8
3
votes
1 answer

how to make ksql print only latest records

I am running following query on ksql. but I want only last 20 records. SELECT MAX(ROWTIME),TIMESTAMPTOSTRING(ROWTIME, 'yyyy-MM-dd HH:mm:ss'),SERIAL,COUNT FROM MY_STREAM WHERE TIMESTAMPTOSTRING(ROWTIME, 'yyyy-MM-dd HH:mm:ss') >= DATE AND…
Nanobrains
  • 275
  • 1
  • 3
  • 11
3
votes
1 answer

How to update KSQL stream definition dynamically based on schema-registry

I created a KSQL stream based on schema-registry by following this post. The Kafka JDBC connector updates a latest schema in schema-registry. The new stream gets created with the latest schema, but existing stream sill in the oldest schema. I don't…
Achaius
  • 5,904
  • 21
  • 65
  • 122
3
votes
1 answer

Why doesn't a KSQL stream not return/show data from messages on topic in broker?

I've got a simple Kafka broker running with a topic raw_events. With kafka-console-producer --topic raw_events --broker-list kafka:29092 < event.json I'm adding events to that topic that successfully show up with kafka-console-consumer…
Jochem Schulenklopper
  • 6,452
  • 4
  • 44
  • 62
3
votes
1 answer

KSQL Join more than two streams

Is it possible to join more than two streams/tables in KSQL? Example: I have three streams: CREATE STREAM StreamA (id BIGINT, message VARCHAR) WITH (KAFKA_TOPIC='TopicA', VALUE_FORMAT='DELIMITED'); CREATE STREAM StreamB (id BIGINT, aid BIGINT,…
Tim Coombe
  • 33
  • 1
  • 4
1 2 3
…
61 62