0

I'm trying to read data from GCP kafka through azure databricks but getting below warning and notebook is simply not completing. Any suggestion please?

  WARN NetworkClient: Consumer groupId Bootstrap broker rack disconnected

  Please note I've properly configured SSL properties of truststore etc and I'm able to read data from simple JAVA spring app locally.

Syed Mohammed Mehdi
  • 183
  • 2
  • 5
  • 15
  • 1
    Have you allowed Azure to connect to the GCP VPC? Please share code as a [mcve] – OneCricketeer Sep 17 '22 at 15:18
  • How to allow Azure to connect to GCP VPC? – Syed Mohammed Mehdi Sep 17 '22 at 18:02
  • Code is df = spark.readStream \ .format("kafka") \ .option("kafka.bootstrap.servers", ...) \ .option("kafka.security.protocol", "SSL") \ .option("kafka.ssl.truststore.location", ) \ .option("kafka.ssl.keystore.location", ) \ .option("kafka.ssl.keystore.password", dbutils.secrets.get(scope=,key=)) \ .option("kafka.ssl.truststore.password","") – Syed Mohammed Mehdi Sep 17 '22 at 18:09
  • Please [edit] your question and format the code. You go to your GCP console and edit the network configs to allow Azure connections. Or, you run Databricks in Google Cloud instead since that's where your kafka is – OneCricketeer Sep 17 '22 at 21:57

0 Answers0