2

So I am using NeDB as a data store for a simple little project, and I want to create a little API that inserts something new and returns the list of all items in the datastore, including the new one.

async function willAddNewItem() {
    // db is NeDB Datastore
    const existing = db.getAllData();
    const sortedIds = existing
        .map(item => item.id)
        .sort((one, another) => one - another);
    const id = sortedIds.length === 0
        ? 0
        : sortedIds.slice(-1)[0] + 1;
    const newItem = { id, foo: 'bar' };

    await new Promise(resolve => db.insert(newItem, () => resolve()));

    return db.getAllData()
        .sort((one, another) =>
            new Date(one.updatedAt).getTime() - new Date(another.updatedAt).getTime()
        );
}

However, every time I call this function, I get the list of items I had before inserting the new one. On the next call, the item I added last time would be there, but not the new one. Refreshing my page (which results in calling db.getAllData() again to populate the initial page) shows everything it should. It’s only missing immediately after I insert it. So it would appear that the callback on db.insert is not actually waiting until the insertion is complete (nor is there any documentation that I can find about this).

Adding an index to id fixes this, and I do want an index on id so that is good, but I still want to know why this is happening/where I have to be careful about it happening in the future. For that matter, I’m a little worried that the only reason adding the index to id worked is because it happened to be a little faster, so it was “done” before I called db.getAllData() when it wasn’t before—and that this may not be consistent.

I could use something like

const afterAdding = await new Promise(resolve =>
    db.insert(newItem, (_, newDoc) => resolve([...existing, newDoc]))
);

and return the sorted afterAdding, but this seems a little dubious to me (I’d really like to get exactly the db’s state rather than try to “recreate” it myself, assuming that insert has done exactly and only what I expected), and in any event I would like to know more about how NeDB works here.

Also, yes, I know I’m ignoring possible errors on the insert (I have checked to see if this is the cause; it is not), and I probably don’t want .getAllData() at all but rather some .find query. This is a basic test case as I get familiar with NeDB.

KRyan
  • 7,308
  • 2
  • 40
  • 68
  • 1
    Have you tried checking for the presence of an error on the callback on `insert`? Could it be that it's simply not inserting the new data? – OliverRadini Dec 06 '18 at 14:55
  • 1
    @OliverRadini No, no error, and on the next call I get the list including the first one, and then on the third call the second is there, and so on. Edited to clarify that. – KRyan Dec 06 '18 at 14:55
  • I'm having a hard time reproducing this with the code you have, when I try, the function returns all the data, including the newly added item. Are you able to provide any more detail about the implementation you have? – OliverRadini Dec 06 '18 at 15:15
  • @OliverRadini Aside from some basic boilerplate responding to the API call, and a slightly larger object being inserted (`id`, two short strings, an empty array, and a number), this really matches what I have. As I noted in an edit, adding an index to `id` fixes the problem, and actually when I remove that index I can’t reproduce the problem anymore, either. Which makes me wonder if it wasn’t just a build snafu (I’m actually using Typescript) that caused me to still be using my original version (which did not `await` the insert) when I was trying to test the `await` version. – KRyan Dec 06 '18 at 15:29
  • @OliverRadini Thanks for taking the time to try to reproduce, I really appreciate that. Unfortunately, unless I can reproduce it myself (assuming it wasn’t a build snafu—I’m using `tsc` in watch mode and `nodemon` and it’s been finicky), I doubt this question is likely to go anywhere. Even more unfortunately, I cannot think of any way to determine for sure that it was a build problem. I’m sure going back to the version without `await` will fail to work, but that doesn’t rule out some weird timing issue with the `await` version... – KRyan Dec 06 '18 at 15:33
  • that's a shame; hopefully someone who has used nedb can investigate this and come up with a useful answer! – OliverRadini Dec 06 '18 at 15:39

0 Answers0