0

This is a question on how to connect self managed ksqlDB / ksql server to confluent cloud.

I have a confluent basic cluster running in https://confluent.cloud/ in the GCP asia south.

enter image description here In this cluster i want to connect self managed ksqlDB to Confluent Cloud Control center.

enter image description here

Here is my configurations which i copied from confluent cloud and put in the managed ksqldb.

enter image description here

This self managed ksqldb is a single machine GCP compute unit.

The same configuration is present in following properties.

/home/confluent/confluent-5.5.1/etc/ksqldb/ksql-server.properties

and the ksql server was started using following commands.

nohup /home/confluent/confluent/confluent-5.5.1/bin/ksql-server-start /home/confluent/confluent/confluent-5.5.1/etc/ksqldb/ksql-server.properties &

Command line :

/home/confluent/confluent-5.5.1/bin/ksql

Couple of things were noted in ksql terminal :

  • STREAM was created successfully in the terminal but not available in the cloud.

  • On the command "show streams;" It is able to show the specific STREAM.

  • print {STREAM}; It doesnt show up data even while data is pushed to STREAM.

  • I have not set any host entries.

  • On show connectors following exception is generated in ksql terminal.

    ksql> show connectors; io.confluent.ksql.util.KsqlServerException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8083 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8083 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused) Caused by: Could not connect to the server. Caused by: Could not connect to the server.

I am expecting my ksqlDB shows up in the confluent cloud but unable to see though.

enter image description here

I dont know what more configurations are required, so that my self managed ksql server works and show up in confluent cloud.

Shivakumar ss
  • 653
  • 7
  • 19

1 Answers1

1

seems that you are confusing some terminology here, self-managed != managed.

Managed KSQLDB is the service available on your Confluent Cloud console (last image). In there you have to add applications which spin up a KSQLDB cluster for your queries.

For self-managed KSQLDB instance running in GCP, you can connect it to Confluent Cloud, but it is not going to appear on the list of KSQLDB Applications, as you'll have to operate it yourself.

Docs:

jeqo
  • 441
  • 3
  • 6
  • Oh my bad. I had mistakenly put self managed as managed ksqldb. just corrected it. Thanks for that. It would have been nice to see self managed ksql status in confluent cloud so that everything was in one place in control center. – Shivakumar ss Sep 08 '20 at 09:24