14

I have a collection in MongoDB called CrawlUser. It has a list called CrawlStatuses, which is a list of CrawlStatus objects. CrawlStatus has a property called LastErrorMessage which I want to remove from the collections.

I tried to do the following to remove it but it didn't work... No error message given, but the LastErrorMessage column is still there.

db.CrawlUser.update( {}, { $unset: { "CrawlStatuses.LastErrorMessage": 1 } }, false, true);

Any ideas what I'm doing wrong?

One other related question, if I do the $unset command on a column in a collection that is very large (millions of rows), mongodb uses up all of the ram on the server (as if it's trying to store the entire collection in memory), then the server crashes. Is there a better way to remove columns when you have large collections?

Justin
  • 17,670
  • 38
  • 132
  • 201

2 Answers2

13

The update with the empty parameter doesn't seem to work. I tried it in the mongo shell and mongoconsole. In the mongoconsole it gave an error about update expecting the first parameter to be an array or an object.

However, you can do the same thing using the $exists find query.

Try:

`db.CrawlUser.update( {CrawlStatuses:{$exists:true}}, { $unset: { "CrawlStatuses.LastErrorMessage": 1 } }, false, true);`

That worked for me.

Keep in mind that based on the docs, $exists doesn't use an index, so it will be slower. I suggest adding a parameter that you can add an index on and query it when doing the $unset.

Alexandru Petrescu
  • 3,449
  • 2
  • 23
  • 23
  • By the way, I had another case where I needed to do the same thing, $unset a property of a child collection, however the parent collection was much larger (3+ million records), and this approach did not work... It took half hour to execute, didn't throw any sort of error or give a success message, yet the property still exists after it completed. I hope they fix this issue soon.... – Justin Apr 24 '11 at 18:35
  • Unfortunately, it's over 2 years later and the ability to remove a field from a child collection is still not implemented. I even tried to remove a field using a forEach loop, but that was also unsuccessful. At this point it may require dumping the entire database, using sed and awk to remove the field, and then reimporting. – Aquarelle Nov 08 '13 at 20:25
0

Looks like you have a couple of issues here.

#1: The $unset command

As far as I can see, this should work just fine. I got the following output on my test:

MongoDB shell version: 1.6.5
connecting to: test
> db.foo.save( { _id : 1, status : { err : 'blah', y : 1 } } )
> db.foo.save( { _id : 2, status : { err : 'blahblah', y : 5 } } )
> db.foo.find()
{ "_id" : 1, "status" : { "err" : "blah", "y" : 1 } }
{ "_id" : 2, "status" : { "err" : "blahblah", "y" : 5 } }
> db.foo.update( { }, { $unset : { "status.err" : 1 } }, false, true )
> db.foo.find()
{ "_id" : 1, "status" : { "y" : 1 } }
{ "_id" : 2, "status" : { "y" : 5 } }

#2: Using up RAM

if I do the $unset command on a column in a collection that is very large, mongodb uses up all of the ram on the server (as if it's trying to store the entire collection in memory)

That's exactly what MongoDB is trying to do. MongoDB uses memory-mapped files. MongoDB will pull all of the data into RAM and let the operating system manage the virtual memory concerns.

So when you do a query with no indexes, you're basically asking MongoDB to walk through every item in the collection. It's basically a giant for loop operating on all of your data, so this is going to require loading everything from disk.

Up to now, this is all normal.

... then the server crashes

This is not normal. I have run this type of update command on hundreds of millions of documents without crashing the server. Are you able to provide any more detail on this problem? Do you have log files?

If so, I would suggest taking your bugs to the Google Groups, so they can help identify the source of the crash. http://groups.google.com/group/mongodb-user

Gates VP
  • 44,957
  • 11
  • 105
  • 108
  • 2
    He said that CrawlStatuses is a list of nested objects, and your status is a single subobject. It doesn't seem to work with a list of nested objects... – Kuba Suder Apr 22 '11 at 15:24