0

I need to modify the Google Data Studio - Google BigQuery Connector for the customized requirements.

https://support.google.com/datastudio/answer/6370296

First Question: How could I find the source code for this data connector?

Second question:

According to the guide, https://developers.google.com/datastudio/connector/reference, getData(), Returns the tabular data for the given request.

And the response is in this format

{
     "schema":[
        {
           "name":"OpportunityName",
           "dataType":"STRING"
        },
        {
           "name":"IsVerified",
           "dataType":"BOOLEAN"
        },
        {
           "name":"Created",
           "dataType":"STRING"
        },
        {
           "name":"Amount",
           "dataType":"NUMBER"
        }
     ],
     "rows":[
        {
           "values":[
              "Interesting",
              true,
              "2017-05-23",
              "120453.65"
           ]
        },
        {
           "values":[
              "SF",
              false,
              "2017-03-03",
              "362705286.92"
           ]
        },
        {
           "values":[
              "Spring Sale",
              true,
              "2017-04-21",
              "870.12"
           ]
        }
     ],
     "cachedData":true
  }

But BigQuery could have 100 millions records in the table. We don't care that it could be 100 millions records, we just give the response in this format anyway?

Thanks!

searain
  • 3,143
  • 6
  • 28
  • 60
  • If you could modify the connector, how would you get Data Studio to use it? (i.e. you can't install it). – elithrar Oct 13 '18 at 00:55
  • "Distribute the connector (Share or publish). Community connectors can be shared directly with any user and/or can be submitted for inclusion in both the in-product gallery and the public Data Studio Connector Gallery for any user to find and use." You can share your connector with certain group people, right? – searain Oct 13 '18 at 01:14

1 Answers1

2

The existing DS-BQ connector is not open source, hence you won't be able to modify its behavior.

With that said:

  • The DS-BQ connector has a "smarter" API contract than the open one - queries and filters will be passed down.

  • Feel free to create your own DS-BQ connector with whatever logic you might require! Community connectors would love your contributions.

Felipe Hoffa
  • 54,922
  • 16
  • 151
  • 325
  • When we use BigQuery data connector, the data source could have dozens millions records. Is that Google offered BigQuery data connector using the above format too. Listing millions records in json format? I am wondering, if we use the above response format, then our customized data connector written in Google apps script will query millions records from the BigQuery data source and list the million records in the above json format in the response. Would we have memory issue, or response too big issue or time out issue? – searain Oct 13 '18 at 04:49
  • Yeah - community connectors don't pass filters down, so you would need to deal with that. – Felipe Hoffa Oct 13 '18 at 06:42
  • so what is the data size limitation (size of the data source) for community connectors? – searain Oct 13 '18 at 07:01
  • URL Fetch response size 50MB / call? – searain Oct 13 '18 at 07:06