Database call in Sails.js beforeCreate() model hook causes validation error

Question

I have a Sails.js model called "products" which is a MongoDB collection. In the model, I have a beforeCreate() hook that generates a unique productId like '170921-00001' in the format 'YYMMDD-count` where count is the record number created for the day. This is what my model looks like:

module.exports = {

    attributes: {
        name: { type: 'string', required: true },
        productId: { type: 'string', unique: true }
    },


    beforeCreate: function(values, next) {

        var moment = require('moment');

        // Generate the productId
        Products.count({
            createdAt: {
                '>=': moment().format('YYYY-MM-DD'),
                '<': moment(moment().format('YYYY-MM-DD')).add(1, 'days').format('YYYY-MM-DD')
            }
        }).exec(function(error, productCount) {

            if (error) {
                throw error;
            }

            var count = '',
                totalPlaces = 5 - productCount.toString().length;
            while (totalPlaces > 0) {
                count = count + '0';
                totalPlaces--;
            }

            values.productId = moment().format('YYMMDD') + '-' + (count + productCount);

            next();
        });
    }
};

The issue is, this break when I am trying to insert multiple products into the collection in a single database call. I get the following error:

debug: Error (E_VALIDATION) :: 1 attribute is invalid
MongoError: E11000 duplicate key error index: my-app.products.$productId_1 dup key: { : "170921-00002" }
    at Function.MongoError.create (/Users/Nag/Code/my-app/web-service/node_modules/sails-mongo/node_modules/mongodb-core/lib/error.js:31:11)
    at toError (/Users/Nag/Code/my-app/web-service/node_modules/sails-mongo/node_modules/mongodb/lib/utils.js:114:22)
    at /Users/Nag/Code/my-app/web-service/node_modules/sails-mongo/node_modules/mongodb/lib/collection.js:658:23
    at handleCallback (/Users/Nag/Code/my-app/web-service/node_modules/sails-mongo/node_modules/mongodb/lib/utils.js:95:56)
    at resultHandler (/Users/Nag/Code/my-app/web-service/node_modules/sails-mongo/node_modules/mongodb/lib/bulk/ordered.js:421:14)
    at /Users/Nag/Code/my-app/web-service/node_modules/sails-mongo/node_modules/mongodb-core/lib/connection/pool.js:455:18
    at /Users/Nag/Code/my-app/web-service/node_modules/async-listener/glue.js:188:31
    at _combinedTickCallback (internal/process/next_tick.js:73:7)
    at process._tickDomainCallback (internal/process/next_tick.js:128:9)
    at process.fallback (/Users/Nag/Code/my-app/web-service/node_modules/async-listener/index.js:529:15)

Invalid attributes sent to undefined:
 • productId
   • A record with that `productId` already exists (`170921-00002`).

When I insert a single record, it works fine but when I insert multiple records, the first record gets inserted but every subsequent record generates that error. Is it because the document is getting inserted before the hook computes the productId before even it has completed inserting the previous record? How do I solve this?

What version of Sails are you using? And can you post the code that you're using to create the records? — sgress454, Sep 22 '17 at 16:57

score 0 · Answer 1 · edited Oct 22 '21 at 13:48

This is a pretty tough one to solve with Waterline. If this convention for the productId is not too embedded in your code yet, the wisest thing might be to change it before too much is built around it.

I have a similar issue in my code, but with a Model that I never create multiples of in a single call - to guard against coincidental "collisions" I just use a try catch in the before create and rerun the create in the catch block.

But here's another solution... maybe it would be possible to store a live day-count in your Product.js file:

var day = new Date();
var daycount = 0;

module.exports = {

    attributes: {
        name: { type: 'string', required: true },
        productId: { type: 'string', unique: true }
    },
    beforeCreate: function(values, next) {
        var moment = require('moment');
        // Generate the productId
        Products.count({
            createdAt: {
                '>=': moment().format('YYYY-MM-DD'),
                '<': moment(moment().format('YYYY-MM-DD')).add(1, 'days').format('YYYY-MM-DD')
            }
        }).exec(function(error, productCount) {
            if (error) {
                next(error);
            }

            // stop here to check your day and count variables...
            var today = new Date();
            if (productCount === 0) { // none created yet today
                day = today;
                daycount = 0;
            } else if (daycount === 0) { // must have started the app with records in db
                daycount = productCount;
            }
            if (day < today && day.getDate() < today.getDate()) { // new day
                day = today;
                daycount = 0;
            }
            productCount = Math.max(productCount, daycount++); // notice the increment
            // now proceed as before...

            var count = '',
                totalPlaces = 5 - productCount.toString().length;
            while (totalPlaces > 0) {
                count = count + '0';
                totalPlaces--;
            }
            values.productId = moment().format('YYMMDD') + '-' + (count + productCount);
            next();
        });
    }
};

Even if many product create calls all have their beforeCreate run before being added to the database (and thus get the same productCount), the daycount and day variables should be held live in memory and updated one by one.

Even if this works, it feels precarious - I would give serious thought to restructuring your data rather than take this approach!

score 0 · Answer 2 · answered Oct 28 '17 at 14:05

0

When you send data to your api, are you establishing the name of your product?

name: { type: 'string', required: true } <= Here, the name is required from client to api.

answered Oct 28 '17 at 14:05

Damian Pumar

11
1

Your question should be in the comments. Not an answer to the question. – Anastasios Selmani Oct 28 '17 at 14:24

score 0 · Answer 3 · edited Nov 16 '17 at 12:35

0

You should make unique values before update method:

values.productId = moment().format('YYMMDD') + '-' + (count + productCount);

This line of your code might be:

values.productId = moment().format('YYMMDD') + '-' + (count + productCount)+ Math.random();

it generate always unique id.

edited Nov 16 '17 at 12:35

Rob

2,243
4
29
40

answered Nov 16 '17 at 10:55

bhavesh

453
3
11

Database call in Sails.js beforeCreate() model hook causes validation error

3 Answers3