Stack Overflow
Stack Overflow

Questions tagged [ksqldb]

ksqlDB: an event streaming database for Apache Kafka.

ksqlDB is a source-available event streaming database for Apache Kafka® developed by Confluent. It is distributed, scalable, reliable, and real-time. ksqlDB combines the power of real-time stream processing with the approachable feel of a relational database through a familiar, lightweight SQL syntax.

You can download ksqlDB at https://github.com/confluentinc/ksql

There is a quickstart at: https://ksqldb.io/quickstart.html

926 questions
2
votes
0 answers

Ksqldb Data Enrichment for nested array

I learn ksqldb Data Enrichment and there was a question: Can ksqldb enrich nested array? ktable users: { "userId": "1", "displayname": "Fred", "Tags": [2,5,6,7] }, { "userId": "2", "displayname": "Ben", …
padavan
  • 714
  • 8
  • 22
2
votes
2 answers

How to validate ksql script?

I would like to know if there is a way for checking if a .ksql script is syntactically correct? I know that you can send a POST request to the server, this however would also execute the containing ksql commands. I would love to have some kind of…
XH_P
  • 31
  • 1
  • 8
2
votes
0 answers

How to use Kafka Streams' Processor API with ksqlDB?

I am trying to find the better way to store our data. ksqlDB provides terminal, UI view to query streams and tables, which makes much easier to handle data. Whilst PAPI's persistence state store in saved all in memory and only way to view is through…
JustLift
  • 153
  • 1
  • 7
  • 17
2
votes
2 answers

KSQL DB collect_list and collect_set does not support multiple columns, struct or map

I am currently building a POC for my company on ConfluentCloud. The current version of KSQLDB does not yet support collect_list/collect_set on multiple columns, a struct or a map; therefore, I am trying to think of a workaround. I am consuming SQL…
tulde23
  • 398
  • 1
  • 6
2
votes
1 answer

Error "Invalid join condition: table-table joins require to join on the primary key of the right input table" on joining two tables on Kafka ksqlDB

I need to create a Kafka topic from a combination a nine other topics, all of them produced by Debezium PostgreSQL source connector, in AVRO format. To start, I'm trying (so far unsuccessfully) to combine fields from only two topics. So, first a…
dotSlashLeo
  • 21
  • 3
2
votes
0 answers

How to configure ksqldb BASIC authentication in docker

I tried to setup ksqldb Basic auth in the following way, based on configuring-listener-for-http-basic-authenticationauthorization version: '3.4' services: zookeeper: image: 'bitnami/zookeeper:latest' ports: - '2181:2181' …
Kubus
  • 677
  • 6
  • 18
2
votes
1 answer

With KSQL, why does my table keep data with older ROWTIME and discard updates with newer ROWTIME?

I have a process that feeds relatively simple vehicle data into a kafka topic. The records are keyd by registration and the values contain things like latitude/longitude etc + a value called DateTime which is a timestamp based on the sensor that…
Pieter Breed
  • 5,579
  • 5
  • 44
  • 60
2
votes
1 answer

KSQLDB Emit Final does not return any value

I've following query on ksqldb , if I use "EMIT CHANGES" it does work, but if change it to "EMIT FINAL" it does not return any value after the window end CREATE TABLE sspc_3536660_v4 as select sspc, >LATEST_BY_OFFSET(CASE WHEN metric…
user1925234
  • 21
  • 1
2
votes
1 answer

The stream created in ksqlDB shows NULL value

I am trying to create a stream in ksqlDB to get the data from the kafka topic and perform query on it. CREATE STREAM test_location ( id VARCHAR, name VARCHAR, location VARCHAR ) WITH (KAFKA_TOPIC='public.location', …
DebianUser
  • 23
  • 4
2
votes
2 answers

Mount (Volumes) not working when embedding Kafka connect into Ksqldb-server

The "volumes" mapping does not seem to work when trying to embed Kafka connector in Ksqldb Server Below is my docker file ksqldb-server: image: confluentinc/ksqldb-server:0.18.0 hostname: ksqldb-server container_name:…
Gautam T Goudar
  • 271
  • 3
  • 12
2
votes
1 answer

Connect to Kafka on host from Docker (ksqlDB)

I'm running ksqldb-server from a docker-compor found here https://ksqldb.io/quickstart.html#quickstart-content My kafka bootstrap server is running on the same VM in standard alone mode. I can see the messages in one topic with a console…
Thadeu Melo
  • 947
  • 14
  • 40
2
votes
0 answers

Limiting kafka ksqlDB query result for web usage

I need to limit the amount of lines returned in a ksql query. At the moment this works well : SELECT * FROM foo_view WHERE id = 123; And I need something to help me page the results to my web page. Something like: SELECT * FROM foo_view WHERE id =…
THIAGO PEREIRA MUNIZ
  • 21
  • 1
2
votes
0 answers

ksqlDB get message's value as string column from Json topic

There is a topic, containing plain JSON messages, trying to create a new stream by extracting few columns from JSON and another column as varchar with the message's value. Here is a sample message in a topic_json { "db": "mydb", "collection":…
Amrit Jangid
  • 168
  • 1
  • 9
2
votes
1 answer

ksqlDB shows wrong `yyyy-MM-dd HH:mm:ss.SSSSSS` format for timestamp

I used kafka-connect to stream a table from MySQL to a kafka topic. The table contained some columns with datetime(6) column type liked this 1611290740285818. When I converted this timestamp to string using ksqlDB using: SELECT…
Scorpioooooon21
  • 491
  • 5
  • 17
2
votes
0 answers

Why there's no SELECT DISTINCT in ksqDB

I'm trying to figure out why there's no SELECT DISTINCT syntax in ksqlDB. Is the absence caused by a conceptual factor? Or is it just missing for now? The reason I'm asking is because I want to have a table based on an existing stream. The only way…
yuranos
  • 8,799
  • 9
  • 56
  • 65
1 2 3
…
61 62