Stack Overflow
Stack Overflow

Questions tagged [confluent-cloud]

154 questions
2
votes
1 answer

Configuring Spark Structured Streaming with authenticated Confluent Schema Registry

I’m using a Kafka Source in Spark Streaming to receive records generated using Datagen in Confluent Cloud. I intend to use Confluent Schema Registry, Currently, this is the exception I am facing : * Exception in thread…
Aditi Sinha
  • 29
  • 4
2
votes
1 answer

Kafka connector "Unable to connect to the server" - dockerized kafka-connect worker that connects to confluent cloud

I'm following similar example as in this blog post: https://rmoff.net/2019/11/12/running-dockerised-kafka-connect-worker-on-gcp/ Except that I'm not running kafka connect worker on GCP but locally. Everything is fine I run the docker-compose up and…
solujic
  • 924
  • 1
  • 18
  • 43
2
votes
1 answer

What is the kafka message "timestamp" represents?

Is it in epoch ms or in seconds? Does it represents when the message arvived to kafka? Or sent by the kafka-client-producer? Or when it was consumed? How can I know the difference clock time between my VM to the Kafka VM in Confluent Cloud?
Stav Alfi
  • 13,139
  • 23
  • 99
  • 171
2
votes
1 answer

Confluent cloud | Java Avro Serialization | junk characters in value

I am trying with basic Avro publisher in java which is available in examples https://github.com/confluentinc/examples/blob/5.5.1-post/clients/avro/src/main/java/io/confluent/examples/clients/basicavro/ProducerExample.java I was successfully able to…
Shivakumar ss
  • 653
  • 7
  • 19
2
votes
2 answers

No Schema Validation on Confluent Cloud

I am using confluent cloud and currently i want to prevent messages with bad schema to ingest in my kafka topic. For that i am using schema management feature of confluent where i want to validate if the message is not matching the specified schema…
vicky
  • 561
  • 1
  • 7
  • 17
2
votes
0 answers

Confluent KSQL "Register a stream or table for this topic" doesn't populate topic fields

I have 4 topics where I was able to successfully stream data to, and their schemas look good. But after clicking a topic > "Query in KSQL", I noticed only 1 of them populates the "Field(s) you'd like to include in your STREAM". The other 3 show up…
atkayla
  • 8,143
  • 17
  • 72
  • 132
2
votes
1 answer

Apache Kafka on Confluent Cloud - Incoherent offsets in partitioned topic and consumer lag

I am finding a strange behaviour when using Kafka on Confluent Cloud. I have created a topic with the default partitioning value: 6. My system consists of a Java Producer application that sends a message to that topic and a Kafka Streams application…
codependent
  • 23,193
  • 31
  • 166
  • 308
1
vote
1 answer

Confluent MySQL connector connection not established

I am trying to connect MySQL DB (installed in my laptop) from Confluent Cloud Remote access using MySQL Source connector. I am not able to establish the connection with MySQL Connector. Error: Unable to validate configuration. If an update was made…
Adhish
  • 65
  • 9
1
vote
1 answer

How to increase default max memory of message in Confluent Cloud?

I am new to Kafka. I have created a cluster and a topic in a Cloud Confluent Kafka (Basic plan). the default max limit of message is 8MB as per the…
Siddharth Sharma
  • 11
  • 2
1
vote
1 answer

go client example of a kafka consumer for confluent cloud with sasl.username and sasl.password

Does anyone have a go client example of a Kafka consumer for the confluent cloud with sasl.username and sasl.password? I am running into an error while trying to consume a message from the confluent cloud. Failed to connect to Kafka broker: kafka:…
xsqian
  • 199
  • 5
  • 13
1
vote
0 answers

How to integrate Camel Spring boot with Kafka in Confluent Cloud?

I have an application which uses camel Spring Boot with debeezium to listen to a mysql database and to publish in a Kafka topic. It was all working fine since I changed the kafka from local to use confluent cloud. I have some other applications…
Renan Geraldo
  • 605
  • 1
  • 6
  • 17
1
vote
1 answer

Terraform enable or disable a resource conditionally

My requirement is to create or delete a resource by specifying enable flag true or false (In case of false the resource should get deleted, in case of true the resource should get created) Kindly refer below code - here I am creating "confluent…
sHaShAnk K
  • 27
  • 3
1
vote
1 answer

Terraform for_each iteration with the file function

My requirement is to create a dynamic resource Confluent Schema. Below is the schema.tf file. Basically, need to include map type object and will be creating different Schema by passing name and file attributes. What changes to be done on below…
sHaShAnk K
  • 27
  • 3
1
vote
0 answers

Unable to connect to Confluent Kafka in Python

I am trying to connect to confluent cloud hosted in Azure and consume the messages but on running the python code it give me the below error which I am unable to gauge. 2022-12-12 11:37:16,249 ERROR FAIL [python-example-consumer#consumer-1]…
potterson11
  • 147
  • 7
1
vote
0 answers

PyFlink 1.11 not connecting to Confluent Cloud (Kafka) cluster

I am configuring PyFlink to connect to a confluent cloud kafka cluster. I am using SASL/PLAIN. Below is the code snippet: """ CREATE TABLE {0} ( `transaction_amt` BIGINT NOT NULL, `event_id` VARCHAR(64) NOT NULL, …
Ahmed Zamzam
  • 11
  • 1
1
2
3
…
10 11