0

I have two collections, LogData and OptData. LogData is having just 600 records whereas OptData is having around 3 million records. To make some logical data, I am trying to merge both colletions. But merging simply by using mongo command create a document larger in size than 16 MB.

Can I merge files using GridFs?

My collections are like 

   LogData
 [{
  "SId": 10,
   "NoOfDaya" : 9
  }
 {
 "SId": 11,
"NoOfDaya" : 8
}]



 OptData 
 [ {
     "SId": 10,
    "CId": 12
    }

{
"SId": 10,
 "CId": 13
   }]

I want something like 

LogData
[{
  "SId": 10,
   "NoOfDaya" : 9
   "OptData" : [{
      "CId": 12
    },{
    "CId": 13
   }
   ]
  }
  {
      "SId": 11,
     "NoOfDaya" : 8,
     "OptData" : []
 }]

I am able to do this by using mongodb find command but it create document size larger than 16 mb.Is it possible to do this using mongodb GridFS?

sangita
  • 191
  • 5
  • 14
  • Whatever you're doing, if your document is even remotely close to the 16MB size limit, you're almost certainly doing it wrong. And no, you can't save huge structured documents as files and expect them to behave as if they weren't files now. GridFS is just a simple way to chunk up `byte[]` data. – mnemosyn Feb 24 '15 at 14:38

1 Answers1

1

You could try to store the whole document as a text string of JSON or binary string of BSON on GridFS, but this is likely not what you want, because when you store data in GridFS MongoDB treats it as a meaningless blob of data and can not perform any useful queries on it.

It would be better to just keep the data normalized in two collections for now. When this causes a specific problem for you, you could describe this problem in a new question. We might be able to suggest you a more practicable solution.

Philipp
  • 67,764
  • 9
  • 118
  • 153
  • In current scenario, I have two collections. Collection1 is OptData(It have around 90 million records). Collection2 LogData(It have only 600 records). We update regularly LogData based on some calculation. Problem here is that, I cannot apply join here since mongoDB doesn't allow it. I can't merge two collections since records are becoming more than 16 mb in size. – sangita Feb 25 '15 at 05:36
  • @sangita This problem is too complex to solve with a comment. Please create a new question and explain exactly what querries you need to perform. – Philipp Feb 25 '15 at 08:14
  • I have created another question [link](http://stackoverflow.com/questions/28714773/map-reduce-to-combine-data-mongodb). Let me know if any thing is still unclear to you. – sangita Feb 25 '15 at 08:56