6

I have javascript that I load to mongo to get some info. 

mongo --port=27017 script.js

I am passing ObjectIds from the file as a variable:

#script.js

db=connect('127.0.0.1:27017/dbname')
db.auth('user', 'pass') //get credentials


var file = cat('file_with_ids.txt');  // read the file
var objectId = file.split('\n');
for (var k =0, j = objectId.length-1; k<j; k++){
    someFunction(objectId[k])

}

It works with files with (500K rows), but I have files with 1.5 M and more records that I need to work on. I was not successful finding documentation on mongodb cat() and its internals. I know that I can split the file into chunks and feed one by one using bash script, but I do not want to go that route. Limitations: I can only use pure JS, no node (could have been easier). I have written python code to do it, it works but python (pymonogo) reorders the order of the documents.

Any suggestions? How to solve this in pure JS? Thanks

p u
  • 1,395
  • 1
  • 17
  • 30
oshaiken
  • 2,593
  • 1
  • 15
  • 25
  • 1
    The `mongo` shell provides a limited programming environment as compared to a full driver implementation like Python. It sounds like you are already aware of better approaches: use a normal driver (eg. PyMongo) or split your input into smaller batches and try to work around the limitations of the `mongo` shell. The Python approach would be much more robust if you want proper I/O and error handling. If your Python code isn't working as expected you should perhaps post a question with relevant code and what you are trying to achieve. – Stennie Jul 22 '17 at 22:39

0 Answers0