0

I am using the Firebase Extension for BigQuery. https://firebase.google.com/products/extensions/firestore-bigquery-export

The document data that shows up in big query is all crammed into one string field called data. This is not usable as I can't query on it.

If I import the data from a firestore export or backup it comes in as expected with each document property in its own field.

Any ideas on how to get the Firebase Extension for BigQuery to work the same as transferring data from an export?

Data from Extension

SELECT * FROM `qmtbc-dev.qmtbc_dev_export.members_raw_latest`

Data from Extension

Data from Export/Backup

SELECT * FROM `qmtbc-dev.qmtbc_dev_export.membersExport`

Data from Export/Backup

EDIT:

It would be good to know if this is by design or something I have not set up correctly in the configuration.

MadMac
  • 4,048
  • 6
  • 32
  • 69
  • @Rob It is a result set. How on earth am I meant to put the text from that in the question without it looking like rubbish? – MadMac Nov 18 '19 at 03:27
  • The data itself is not relevant to the question. – MadMac Nov 18 '19 at 03:28
  • 1
    Ok. As long we can understand it from the question without the image I guess it's OK. – Rob Nov 18 '19 at 03:28
  • 1
    It is a String but not any String it is a JSON, try with UNNEST https://cloud.google.com/bigquery/docs/reference/standard-sql/json_functions – cutiko Nov 18 '19 at 03:44
  • @cutiko Thanks that looks useful. I will give it a try if I don't get a solution. I would have thought the extension would have done this part automatically. This table is populated real time (trigger from firestore) so I would have to set up a job to run a script and export it to another table regularly. Seems like a lot of work that I would have expected handled already. – MadMac Nov 18 '19 at 04:26
  • @MadMac where you able to make it work? – Nahuel Varela Nov 19 '19 at 08:24
  • @Nahuel Varela Not yet I am still waiting to see what the stackoverflow net drags in. I made a functionality request here: https://github.com/firebase/extensions/issues/113#issuecomment-555537246 – MadMac Nov 19 '19 at 19:08
  • @MadMac was the information on the Github issue working for you? – Nahuel Varela Nov 22 '19 at 09:24
  • @NahuelVarela yes it seems to work. I will put a full answer up once I have time to test it further. – MadMac Nov 22 '19 at 21:19

0 Answers0