MapReduce on gridfs file with mongodb

Question

I want to store bson documents in gridfs because they grow rapidly over 16MB. But i also have to do some mapreduce analytics on them. Is that possible or do i have to split the document in multiple documents to do that. Tutorials and other stuff are always talking about binary data like pictures, videos and so on but not about bson documents

Thanks.

score 0 · Accepted Answer · answered Feb 07 '13 at 23:31

0

GridFS is only meant to store binary files. It is not meant to split normal documents (which you call BSON documents). If your BSON documents are too large, you need to rethink your data schema. If you provide that schema, I can update my answer with hints and tips.

answered Feb 07 '13 at 23:31

Derick

35,169
5
76
99

I wanted to store one day of log data from one source in one bson document and retain them for some period of time. I would then have about 80 Mb in one doc. I splitted it to one doc per entry. I calculcate the statistics after import and drop the log data. Thanks and sorry for the delay – Finn Feb 25 '13 at 17:49

MapReduce on gridfs file with mongodb

1 Answers1