1

I have a query that I need to run on a collection and load the entire output into memory as an array for further processing. The query is as below.

const votes = db.Vote.find(
     {
         "companyName" : "<companyName>",
         surveyId: "<surveyId>",
         completed:true
     },
     {
         postId: true,
         comment: true,
         options: true,
         userId: true,
     }
 )
 .batchSize(100000)
 .toArray()

This operation takes around 10-15 seconds to run. Is there any way I can optimize this operation? I can't take the output in chunks (using a forEach) and process that. I do need the entire output in-memory in the array. I've tried using cursor streams suggested in this answer but the performance is still the same.

Is this an inherent challenge in MongoDB and the only way out is some form of vertical scaling?

prasad_
  • 12,755
  • 2
  • 24
  • 36
Sriram C S
  • 11
  • 2
  • Have you used any indexes on the `find` filter fields (companyName, etc.)? In general, indexes are used for faster query performance. – prasad_ Feb 05 '20 at 10:17
  • 1
    Sorry for not mentioning this in the question. There are indexes already in place. The problem is not with the finding of the records as much as it is with the deserialization of the found records into an array. – Sriram C S Feb 06 '20 at 08:57

0 Answers0