Some way to create 1 million relationships with a neo4j query

Question

With this query I am importing 75000 nodes from my csv file. (Category)

USING PERIODIC COMMIT 
LOAD CSV WITH HEADERS 
FROM "file:///prodcategory.csv" AS row 
CREATE (:Category {id: row.idProdCategory, name: row.name, idRestaurant: row.idRestaurant});

And with this query I am also importing 1 million nodes from my csv file (Product)

USING PERIODIC COMMIT 
LOAD CSV WITH HEADERS 
FROM "file:///products.csv" AS row 
CREATE (:Product {id: row.idProduct, idProductCategory: row.idProductCategory,name: row.name,idRestaurant:row.idRestaurant ,description: row.description, price: row.price, shipping_price: row.shippingPrice});

I am using this query to create the relationship between id -> category and idProductCategory -> products.

MATCH (category:Category {id: category.id})
MATCH (Product:Product {idProductCategory: Product.idProductCategory})
WHERE Product.idProductCategory=category.id
MERGE (category)-[:OF_CATEGORY]->(Product);

This query only creates 2999 relationships and I do not believe the 1 million relationships I should create, please if there is a method or configuration to be able to create more than 1 million relationships please help me I would be very grateful.

Dave Bennett · Accepted Answer · 2017-07-10T23:46:05.883

4

Ensure you have indexes on Product.idProductCategory.

I assume that the category id is unique across categories.

CREATE CONSTRAINT ON (category:Category) ASSERT category.id IS UNIQUE;

I assume that there are multiple products with the same category ID.

CREATE INDEX ON :Product(idProductCategory);

Then you can simply match each category and then for each category find the appropriate products and create the relationships.

// match all of your categories
MATCH (category:Category)

// then with each category find all the products
WITH category 
MATCH (Product:Product {idProductCategory: category.id })

// and then create the 
MERGE (category)-[:OF_CATEGORY]->(Product);

If you are running into memory constraints you could use the APOC periodic commit to wrap your query...

call apoc.periodic.commit("
  MATCH (category:Category)
  WITH category 
  MATCH (Product:Product {idProductCategory: category.id })
  MERGE (category)-[:OF_CATEGORY]->(Product)
",{limit:10000})

edited Jul 10 '17 at 23:46

answered Jul 10 '17 at 21:56

Dave Bennett

10,996
3
30
41

Thanks for the help apparently I have problems with my memory that I have 4G ram can solve this in some configuration I am linux 1. This is the message that shows me: – Claure Jul 10 '17 at 23:31
There is not enough memory to perform the current task. Please try increasing 'dbms.memory.heap.max_size' in the neo4j configuration (normally in 'conf/neo4j.conf' or, if you you are using Neo4j Desktop, found through the user interface) or if you are running an embedded installation increase the heap by using '-Xmx' command line flag, and then restart the database. – Claure Jul 10 '17 at 23:31
you could wrap it with apoc.periodic.commit – Dave Bennett Jul 10 '17 at 23:47
I get the following error when applying the query with apoc: {java.lang.OutOfMemoryError: Java heap space -> 1} I need to thank you for your help – Claure Jul 11 '17 at 15:44
have you extended your heap in the `conf/neo4j.conf` file? – Dave Bennett Jul 11 '17 at 16:36
yes dbms.memory.heap.initial_size=1512m dbms.memory.heap.max_size=1512m dbms.memory.pagecache.size=10g – Claure Jul 11 '17 at 19:00
Thanks for the help I was able to create the relationships thanks to your help. – Claure Jul 11 '17 at 23:52

score 1 · Answer 2 · answered Jul 10 '17 at 21:54

try to change your query to this... you are using too many filters in your query

check docs for MATCH

MATCH (category:Category),(Product:Product)
WHERE Product.idProductCategory=category.id
MERGE (category)-[:OF_CATEGORY]->(Product)

you can also just change your second import query, so you do not need a separate query for linking.

USING PERIODIC COMMIT 
LOAD CSV WITH HEADERS 
FROM "file:///products.csv" AS row 
CREATE (p:Product {id: row.idProduct, name: row.name,idRestaurant:row.idRestaurant ,description: row.description, price: row.price, shipping_price: row.shippingPrice})
MATCH (c:Category{id:row.idProductCategory}
MERGE (p)-[:OF_CATEGORY]->(c)

Some way to create 1 million relationships with a neo4j query

2 Answers2