How to create over 60,00,000 edges in seconds in OrientDB

Asked Oct 17 '16 at 10:38

Active Oct 17 '16 at 10:38

Viewed 100 times

I want to create over 60,00,000 edges in few minutes(within 30 minutes) in OrientDB. Is it Possible ??

values for each edge is taking from a text file and using Node JS filestream's readline (used version Node 4.x.x LTS version)

I have tried from Node.js using OrientJS driver and using batch;..batch of Queries ended by ;..commit.

It takes almost 3 hr and 30 min to complete it.

asked Oct 17 '16 at 10:38

Anijit Sau

Hi, have you tried with Java? could be useful for you? – Michela Bonizzi Oct 17 '16 at 10:39
You should try something to read text files faster, thats for sure your bottleneck, read the entire file in one time, read line by line is too slow, after read the entire file, you could use some childs so you make requests parallelly. – Jairo Oct 17 '16 at 10:42
No. I have to do it with Node JS – Anijit Sau Oct 17 '16 at 10:46
2

For the beginning, read the file in one step with fs.readFile, it will speed your process a lot. – Jairo Oct 17 '16 at 10:47
@anijit Can you share the code ? – wolf4ood Oct 17 '16 at 12:20
2

orientdb says it build for such type(one of yours) but I too never had a great performance. It took around 3 hrs for me to create 9k vertex, edges and then it gave heap error at -xmx512M heap size – Prakash P Oct 20 '16 at 07:42
@PCP Yes, I have got that HEAP error thing sometimes. – Anijit Sau Oct 24 '16 at 06:42

0 Answers0