149

Per the Mongoose documentation for MongooseJS and MongoDB/Node.js :

When your application starts up, Mongoose automatically calls ensureIndex for each defined index in your schema. While nice for development, it is recommended this behavior be disabled in production since index creation can cause a significant performance impact. Disable the behavior by setting the autoIndex option of your schema to false.

This appears to instruct removal of auto-indexing from mongoose prior to deploying to optimize Mongoose from instructing Mongo to go and churn through all indexes on application startup, which seems to make sense.

What is the proper way to handle indexing in production code? Maybe an external script should generate indexes? Or maybe ensureIndex is unnecessary if a single application is the sole reader/writer to a collection because it will continue an index every time a DB write occurs?

Edit: To supplement, MongoDB provides good documentation for the how to do indexing, but not why or when explicit indexing directives should be done. It seems to me that indexes should be kept up to date by writer applications automatically on collections with existing indexes and that ensureIndex is really more of a one-time thing (done when a new index is being applied), in which case Mongoose's autoIndex should be a no-op under a normal server restart.

Nick S.
  • 2,203
  • 3
  • 19
  • 21
  • It is always good to do indexing with a separate script that does not get executed on every deployment. It is more from a maintenance point of view. Otherwise, some developers will someday add some index for a collection that already has millions of records and then it might incur a major business loss. – Krrish Raj Jul 16 '21 at 10:32

3 Answers3

161

I've never understood why the Mongoose documentation so broadly recommends disabling autoIndex in production. Once the index has been added, subsequent ensureIndex calls will simply see that the index already exists and then return. So it only has an effect on performance when you're first creating the index, and at that time the collections are often empty so creating an index would be quick anyway.

My suggestion is to leave autoIndex enabled unless you have a specific situation where it's giving you trouble; like if you want to add a new index to an existing collection that has millions of docs and you want more control over when it's created.

JohnnyHK
  • 305,182
  • 66
  • 621
  • 471
  • 13
    I have a question to add...What if I set it false? Than will the indexes be created when i insert the data or do I need to explicitly create it. I am sorry if this is a novice question but it would be really helpful if you answered. – Saransh Mohapatra Jun 21 '13 at 14:18
  • 6
    @SaranshMohapatra When `autoIndex` is false, you need to call [ensureIndexes](http://mongoosejs.com/docs/api.html#model_Model.ensureIndexes) on your model to create its indexes. – JohnnyHK Jun 23 '13 at 15:10
  • Than will I have to call it every time or just once defining the model? – Saransh Mohapatra Jun 24 '13 at 15:10
  • @SaranshMohapatra when you define (compile) your model. I do that when I first start the app. Now the hard think is to decide to drop all indexes and recreate them, in case you schema changes. – Moss Jul 31 '13 at 12:15
  • 3
    @JohnnyHK do you still agree with your answer now that it's almost 2016? – Alexander Mills Dec 17 '15 at 09:04
  • 1
    @JohnnyHK When I have `autoIndex: true`, and later if I remove some index from my source code, the index still exists in the MongoDB server. Are there ways to automatically remove the index on application startup when they are deleted from the source code? – Azamat Abdullaev Mar 18 '21 at 03:49
  • 1
    @AzamatAbdullaev No, you need to remove indexes manually (in the shell or via code). – JohnnyHK Mar 18 '21 at 14:14
  • 1
    @JohnnyHK do you still agree with your answer now that it's already 2022? – Janson Chah Feb 09 '22 at 02:44
49

Although I agree with the accepted answer, its worth noting that according to the MongoDB manual, this isn't the recommended way of adding indexes on a production server:

If your application includes ensureIndex() operations, and an index doesn’t exist for other operational concerns, building the index can have a severe impact on the performance of the database.

To avoid performance issues, make sure that your application checks for the indexes at start up using the getIndexes() method or the equivalent method for your driver and terminates if the proper indexes do not exist. Always build indexes in production instances using separate application code, during designated maintenance windows.

Of course, it really depends on how your application is structured and deployed. If you are deploying to Heroku, for example, and you aren't using Heroku's preboot feature, then it is likely your application is not serving requests at all during startup, and so it's probably safe to create an index at that time.

In addition to this, from the accepted answer:

So it only has an effect on performance when you're first creating the index, and at that time the collections are often empty so creating an index would be quick anyway.

If you've managed to get your data model and queries nailed on first time around, this is fine, and often the case. However, if you are adding new functionality to your app, with a new DB query on a property without an index, you'll often find yourself adding an index to a collection containing many existing documents.

This is the time when you need to be careful about adding indexes, and carefully consider the performance implications of doing so. For example, you could create the index in the background:

db.ensureIndex({ name: 1 }, { background: true });
Community
  • 1
  • 1
Tom Spencer
  • 7,816
  • 4
  • 54
  • 50
  • 3
    Ok, so all you have to do is NOT start your server until all the ensureIndex callbacks have fired for each collection. – Alexander Mills Dec 17 '15 at 09:06
  • @AlexMills how do you ensure that? – lonelymo May 26 '16 at 06:09
  • async.each(Object.keys(models), function (key, cb) { models[key].ensureIndexes(cb) }, cb) – Alexander Mills Feb 24 '17 at 01:39
  • just call ensureIndexes on each mongoose model, wait for all to finish, then start your server; I also recommend waiting for db connections to happen before starting your server as well – Alexander Mills Feb 24 '17 at 01:40
  • what does it mean {name: 1}, is it the same as {name:true}? – Alexander Mills Mar 09 '17 at 22:56
  • 3
    There is no `ensureIndex` anymore. There is `createIndex` instead. Am I right? – jack blank Apr 15 '18 at 19:25
  • sadly, `background: true` is the default, no need to specify that – tar May 21 '19 at 12:55
  • 1
    background is now deprecated. Mongodb will ignore it from 4.2. Mongodb will now do the index building using a specialized build which is asynchronously except during the start and end of the build process as specified in the docs https://docs.mongodb.com/manual/reference/method/db.collection.createIndexes/ – Nagabhushan Baddi Nov 06 '19 at 10:59
1

use this block code to handle production mode:

const autoIndex = process.env.NODE_ENV !== 'production';
mongoose.connect('mongodb://localhost/collection', { autoIndex });
Masih Jahangiri
  • 9,489
  • 3
  • 45
  • 51