0

I am new to gcloud and trying to leverage bigquery for accessing the data. I am trying to implement a connection pooling mechanism for bigquery and found that Hikari (one of the default datasource provided by spring boot) as a viable option. Could you please guide me in setting up the Bigquery credentials in the Hikari datasource as I am not able to figure out the way to give the driver, url and credentials.

I imported the bigquery api and hikari api in my pom dependencies

FYI: I am trying to make use of springboot

vijayakumarpsg587
  • 1,079
  • 3
  • 22
  • 39
  • For Bigquery authentication you can take a look into [this document](https://cloud.google.com/bigquery/docs/reference/libraries#setting_up_authentication). – enle lin May 18 '18 at 11:05
  • thank you. I think I found a way. I need to add JDBC drivers using Simba which is the official JDBC driver support for bigquery and then use that to develop a Hikari connection pool just using spring properties. – vijayakumarpsg587 May 25 '18 at 07:37
  • Great! If it worked to you, could you post it as an answer in order to help the community member who will have same problem =) – enle lin May 26 '18 at 10:16
  • @enlelin : sorry for the delay, i initially thought that we found a way to use connection pooling using Hikari using Simba but was continuously facing errors. Hence we shifted to bigquery API. Also I tried CDATA jdbc drivers as well and seem to have connectivity issues while deploying to my app engine. – vijayakumarpsg587 Jun 13 '18 at 09:37
  • I understand that you are having connection issue to the datasource from Bigquery API. Can you show the error? (consider to update your post and provide more information) – enle lin Jun 18 '18 at 14:43

1 Answers1

1

I have managed to establish connection from spring boot to bigquery using simba JDBC drivers and successfully queried it.

Firstly, you need to include GCP starter dependency in your boot.

       <artifactId>spring-cloud-gcp-dependencies</artifactId>

Dowload and add dependency jars of simba drivers in spring boot from the below link. Also refer installation guide in the link.

https://cloud.google.com/bigquery/docs/reference/odbc-jdbc-drivers

Secondly, create a service account that has access to bigquery and generate a key JSON file which is used to authenticate our spring boot application for connecting bigquery. Refer the below link.

https://cloud.google.com/bigquery/docs/reference/libraries#setting_up_authentication.

Lastly configure this key JSON file to be accessed by our spring book application using the simba driver JDBC installation guide. Create a simba driver datasource and use this data source to create JDBC template bean. ( JDBC url formation is given in the installation guide of simba driver) . Thats it!! Run the application and see the results.

If you face any difficulties feel free to ask the specifics.

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
loke
  • 53
  • 6
  • Thanks for the references provided. I have stopped working on the project. I will try these out later and let you know – vijayakumarpsg587 Jun 08 '21 at 05:53
  • When I started working the only thing I was able to find in stack overflow was your question but no answers... So after achieving it, I thought I will share my knowledge... I know it's 2 years late..but still... – loke Jun 09 '21 at 06:06
  • And I thank you for that actually :) Its alwalys a new learning for me and I greatly appreciate it , you taking time to put this together and answer it :) . It will be useful for everyone who takes this route – vijayakumarpsg587 Jun 09 '21 at 07:45