I am trying to create a big number of documents in couchDB with nano bulk. I am talking about a million documents in a single DB. I have a loop and in the loop I have a counter that counts 1000 documents and adds them with nano bulk. That works until i change the loop cycles to 1 million. The running script freezes and like the couchdb server too. I think it is something to do with the javascript?
Asked
Active
Viewed 92 times
0
-
Want to post your script? – clay Nov 30 '16 at 16:56
-
@clay http://pastebin.com/CV5ZjG6b , couldn't paste it here, sorry. – Stoyan Nov 30 '16 at 17:07
-
I won't comment on the code in an answer, because it cannot be seen in the question. Check to make sure your `fillWithBulk` function is being called when you expect, and as often as you expect. Based in your code's logic, `count >= 1000` will be correct only once in your 1000 test case, and 1000 times in your 1M test case. Seems like this was a snippet from your full code, so I cannot see other context. – clay Nov 30 '16 at 17:29
-
You code is probably freezing because your sending millions of documents in batchs without any delay between them. Also, if CouhDB freeze aswell, you should have some errors logs to share with us since normally, CouchDB, would handle the traffic. – Alexis Côté Nov 30 '16 at 18:24
-
Use https://www.npmjs.com/package/big-numbers for converting, formatting, arithmetics and trigonometric operations – alexey28 May 13 '19 at 06:43