2

My high level purpose is to export mongo data to Bigquery so I could do data analysis.

I don't want to export as csv because doing so requires me to specify fields to export manually.

However, mongoexport to json will have these type data like

"registerTimestamp":{"$numberLong":"1429594506335"}

This tpye $numberLong really messes up my bigquery import. Error message like:

Errors: query: Illegal field name: $numberLong

I don't find a way to export mongo without type. How to solve this export to bigquery problem....

Lucas Shen
  • 327
  • 6
  • 14

3 Answers3

2

I think you can do something like this:

Create a script call command.js containing:

printjson( db.collection.find().toArray() )

Then execute the command as below:

mongo dbname command.js > output.json

This article is the source and will provide more details

Community
  • 1
  • 1
Brmmmm
  • 152
  • 7
1

I end up exporting CSV and take it from there.

Lucas Shen
  • 327
  • 6
  • 14
0

I think you could open the json file with vim then replace the type with an empty string using a regular expression like this:

  1. to NumberLong(***), use

:%s/NumberLong((.*))/\1/g

  1. to ObjectId(***), use

:%s/NumberLong((.*))/\1/g