Adding new property to each document in a large collection

Question

Using the mongodb shell, I'm trying to add a new property to each document in a large collection. The collection (Listing) has an existing property called Address. I'm simply trying to add a new property called LowerCaseAddress which can be used for searching so that I don't need to use a case-insensitive regex for address matching, which is slow.

Here is the script I tried to use in the shell:

for( var c = db.Listing.find(); c.hasNext(); ) {
   var listing = c.next();
   db.Listing.update( { LowerCaseAddress: listing.Address.toLowerCase() });
}

It ran for ~6 hours and then my PC crashed. Is there a better way to add a new property to each documentin a large collection (~4 million records)?

It looks like you are missing some parameters on the update - you don't have a criteria so it might be trying to update every record on each iteration of the loop. (I am not actually sure how mongo behaves if you don't set a criteria on the update) — James Avery, May 31 '11 at 13:39
javascript here does not work. Take a look at Sid Burn's answer. Also, if you have to update lots and lots of documents, you probably want to do it in batches to avoid locking and to have a checkpoint for progress in case something fails. — Gates VP, Jun 01 '11 at 17:23

score 24 · Accepted Answer · answered May 31 '11 at 14:39

24

you JavaScript didn't work, but the code below works. But don't knew how long it takes for 4 Million records.

db.Listing.find().forEach(function(item){
    db.Listing.update({_id: item._id}, {$set: { LowerCaseAddress: item.Address.toLowerCase() }})
})

answered May 31 '11 at 14:39

David Raab

4,433
22
40

score 3 · Answer 2 · edited May 22 '18 at 10:45

3

You can use updateMany for this too:

try {
 db.<collection>.updateMany( {}, {$set: { LowerCaseAddress: 
 item.Address.toLowerCase() } } );
} catch(e) { 
  print(e);}

edited May 22 '18 at 10:45

Matthias Herrmann

2,650
5
32
66

answered Sep 12 '16 at 16:02

timelfelt

680
1
11
26

2

How is item not undefined? – Matthias Herrmann May 22 '18 at 10:34

Adding new property to each document in a large collection

2 Answers2