Stack Overflow
Stack Overflow

Questions tagged [ksqldb]

ksqlDB: an event streaming database for Apache Kafka.

ksqlDB is a source-available event streaming database for Apache Kafka® developed by Confluent. It is distributed, scalable, reliable, and real-time. ksqlDB combines the power of real-time stream processing with the approachable feel of a relational database through a familiar, lightweight SQL syntax.

You can download ksqlDB at https://github.com/confluentinc/ksql

There is a quickstart at: https://ksqldb.io/quickstart.html

926 questions
1
vote
1 answer

Unable to fetch the data from ksql using Java client

I'm developing a simple Java app, which tries to pull data from ksqldb using Java client. Below is the code, but StreamedQueryResult streamedQueryResult = client.streamQuery(pullQuery).get(); forever going in block state. public class ExampleApp { …
Jeff Cook
  • 7,956
  • 36
  • 115
  • 186
1
vote
1 answer

How to make ksql_processing_log and send error to kafka topic

I update log4j.properties in ksqldb and make auto create topic and stream in ksql.properties, but the error does not show in topic ksql_processing_log but shows in file ksql.log why? and this my log4j.properties log4j.rootLogger=INFO, stdout,…
Mujiburrahman
  • 13
  • 2
1
vote
1 answer

ksqlDB with Apicurio Registry

I’m trying to make ksqlDB work with Apicurio Registry (RedHat) and have had a few issues with compatibility with Debezium. So, Apicurio has 4 different APIs that can be used. The “official” and recommended api is called "/registry/v2" and it seems…
Propster
  • 19
  • 4
1
vote
1 answer

Why is my JDBC Kafka Connector not picking up new events or creating a topic after creating a source connector?

We currently have a connector up and running according to our ksqlDB Server: ENTITY_CHANGE | SOURCE | io.confluent.connect.jdbc.JdbcSourceConnector | RUNNING (1/1 tasks RUNNING) Though a jdbc_entity_change topic is not generated by our source…
Farhan Islam
  • 609
  • 2
  • 7
  • 21
1
vote
1 answer

ksqlDB migrations tool HTTP2 Error when trying to apply migrations via ksqldb-cli docker image

I am trying to use ksqlDB migrations via a Github Actions file like so: https://github.com/jzaralim/ksqldb-migrations-action/blob/main/.github/actions/ksqldb-migrations-apply/action.yaml. When running my migrations I receive this…
Farhan Islam
  • 609
  • 2
  • 7
  • 21
1
vote
0 answers

KSQLDB randomly stops working with Kafka cluster - "Error sending fetch request" and blocked thread warnings

We are running a KSQLDB with two nodes, which connects to a 16-node Kafka cluster. After about a week, the KSQLDB randomly stops working, and we are unable to use the KSQL CLI. The server status becomes "unknown," and we cannot perform queries.…
jakabl
  • 11
  • 1
1
vote
1 answer

Flux from CompleteableFuture>

I'm using a Java client that returns a CompleteableFuture (StreamedQueryResult extends Publisher). And I'm trying to convert this to a flux and emit values as they're received. The client connection will be infinite so I believe the issue might be…
andresmonc
  • 398
  • 7
  • 21
1
vote
1 answer

Receive message on KSQL window close

I have an initial event with some key. I want to understand, if there are no events with the same key happened within fixed time interval(let's assume 60 seconds) after initial event and do some actions in this case immediately. The first thought…
Andrey Kuchko
  • 11
  • 3
1
vote
1 answer

KSQLDB - how to add message with key to Kafka topic with using serializer JSON

I'm trying to add message with key to Kafka topic (running locally on DOCKER). I don't know why, JSON serializer works for value but not for key. Creating the topic and stream in KSQLDB CLI using the command: CREATE STREAM IF NOT EXISTS users_stream…
magda_lena
  • 13
  • 3
1
vote
1 answer

Is it possible to create a stream in ksqlDB to emit an ID that detects changes from 3 different tables that are joined together?

Is it possible to create a stream that detects changes on 3 different tables? For example, I have Table A which contains Ids for Table B and Table C. If I constructed my join query correctly. could I emit an event that contains Table A's id if there…
Farhan Islam
  • 609
  • 2
  • 7
  • 21
1
vote
0 answers

Why kafka topics receive messages and streams doesn't?

I've got a complete Confluent Platform on an AKS cluster. I'm trying to send messages inside a ksqldb stream, but the messages arrive only on the topic. This is how I created the stream "CREATE STREAM PROVA (ID STRING KEY, NUM INTEGER) WITH…
Luana Mantovan
  • 35
  • 5
1
vote
1 answer

Ksql Can't find any functions with the name 'COUNT_DISTINCT'

I am running ksql cli using docker run --net=host --interactive --tty confluentinc/cp-ksql-cli:latest http://localhost:8088 I have created a stream CREATE STREAM RAW (`nodeId` INT,`data` STRING) WITH (kafka_topic='raw-data',…
dev Joshi
  • 305
  • 2
  • 21
1
vote
1 answer

KSQLDB: Using CREATE STREAM AS SELECT with Differing KEY SCHEMAS

Here is the description of the problem statement: STREAM_SUMMARY: A stream with one of the value columns as an ARRAY-of-STRUCTS. Name : STREAM_SUMMARY Field |…
Krishnamurthy Hegde
  • 43
  • 6
1
vote
1 answer

Convert timestamp fields to ISO 8601 when push data to topic

I have a stream that pulls data from a table in postgres with the following definition: CREATE TABLE "user" ( "_uid" UUID NOT NULL DEFAULT gen_random_uuid() PRIMARY KEY, "_created" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3), "_updated"…
LeandroLuk
  • 154
  • 1
  • 4
  • 11
1
vote
0 answers

Error issuing GET to KSQL server. path:/info when I connect to ksql server

I try to use ksql in my local this is my docker-compose file kafka: image: confluentinc/cp-kafka:latest container_name: mvtds-transformer-kafka depends_on: - zookeeper ports: - "9092:9092" environment: KAFKA_BROKER_ID: 1 …
Yingjie Guan
  • 11
  • 3
1 2 3
…
61 62