3

Im trying to make simpe desktop app with electron or nw.js (not decided yet). But I see a big problem right now with databases.

I need to handle something about 1-2 millions of records, so i started to search and test. Sqlite looks ok, it can handle it, but its hard to compile for both - nw and electron. Always some errors and I gived up for now. So I tested NeDB. Fast, little, nice. After inserting 200k records on nw.js, database was never fully loaded, and count operation was not possible. So i never have any chance to test it with million records. Key / value databases to much work, need SQL or collections like MongoDb.

Do you have any ideas to make fast, embedded database for Nw.js/Electron desktop apps?

Daimos
  • 1,473
  • 10
  • 28

2 Answers2

0

After some research I think that only way for this is sqlite3. It was impossible to compile for me, but sqlite3 for node have nice feedback. I opened issue and after few hours there was new version.

Few problems you may have when you compile sqlite3 for nw.js:

x64 or x86

Check architecture or every component you have. My first problem was that I was using x64 and I installed node-gyp with I option. So i used:

npm install --global --production windows-build-tools

Better dont do it. I had python before, but windows-build-tools installed other anyway, and my x64 python was switched to x86. Install everything you need your self, using same architecture or you will have problems.

Sqlite3 compiled, I added new module and cannot update

Its known issue. So if you want to add any module to your nw app, remove all node_modules and compile sqlite3 again.

Daimos
  • 1,473
  • 10
  • 28
  • Which db are you using for your application? I did this https://stackoverflow.com/questions/51094968/embed-mongodb-with-electron but not completely successful – Rohit Jul 03 '18 at 12:49
  • At the end I used sqlite. Compilation sometimes was hard, but possible. Works fine, but for now 200k records only – Daimos Jul 04 '18 at 18:46
  • @Daimos, after 200K are there performance issues? Have you tried working with IndexedDB which comes as a 'browser DB' within Chromium? – Vass Feb 16 '22 at 04:56
0

Have you tried breaking up the storage in NeDB in a way that distributes that data across multiple file stores? I handle millions of records in NeDB by creating a "root" database that tracks where records are being stored, and then breaking my records up into smaller files, it's not pretty but it works.

You could try Loki.js - http://lokijs.org - I have not yet myself so I cannot say for sure that it's performance claims are honest.

tremor
  • 3,068
  • 19
  • 37