Export Neo4j.rb Nodes with Associations as JSON

Question

I have an app that is using the Neo4j.rb gem in a Rails app on a smallish database with only a few thousand nodes/relationships and I'm now writing a batch upload and download script. Mostly for backup/migration purposes. I have 2 nodes for now- Terms and Permission Groups. It's pretty easy to export each node with just the properties, but I'm missing the associations within each node. Is there a way to export each node with their properties AND associations?

Example on a DB with only 2 nodes who have an association with each other:

# Create blank holding array
2.0.0-p353 :001 > bulk = []
 => []

# Iterate through each node and drop into array
2.0.0-p353 :002 > Term.all.each do |t|
2.0.0-p353 :003 >     bulk << t
2.0.0-p353 :004?>   end
 => [#<Term access_designation: nil, created_at: Mon, 20 Jul 2015 18:12:39 +0000, created_by: nil, data_availability: nil, definition: "New York", name: "Awesome Term", notes: nil, possible_values: nil, sensitivity_access_notes: nil, sensitivity_classification: nil, source_system: nil, updated_at: Mon, 20 Jul 2015 18:12:39 +0000, updated_by: nil>]
2.0.0-p353 :005 > PermissionGroup.all.each do |pg|
2.0.0-p353 :006 >     bulk << pg
2.0.0-p353 :007?>   end
 => [#<PermissionGroup created_at: Mon, 20 Jul 2015 18:14:29 +0000, created_by: nil, name: "my group", updated_at: Mon, 20 Jul 2015 18:14:29 +0000, updated_by: nil>]

# output result to json
2.0.0-p353 :008 > bulk.to_json
 => "[{\"term\":{\"name\":\"Awesome Term\",\"definition\":\"New York\",\"source_system\":null,\"possible_values\":null,\"notes\":null,\"data_availability\":null,\"sensitivity_classification\":null,\"access_designation\":null,\"sensitivity_access_notes\":null,\"created_at\":\"2015-07-20T18:12:39+00:00\",\"updated_at\":\"2015-07-20T18:12:39+00:00\",\"created_by\":null,\"updated_by\":null,\"id\":\"ed820017-24d9-4895-aea2-2a899c83987a\"}},{\"permission_group\":{\"name\":\"my group\",\"created_at\":\"2015-07-20T18:14:29+00:00\",\"updated_at\":\"2015-07-20T18:14:29+00:00\",\"created_by\":null,\"updated_by\":null,\"id\":\"7f6d87cf-a2d3-488e-be03-ca4087e48986\"}}]"

The resulting JSON prettified looks like:

[
    {
        "term": {
            "name": "Awesome Term",
            "definition": "New York",
            "source_system": null,
            "possible_values": null,
            "notes": null,
            "data_availability": null,
            "sensitivity_classification": null,
            "access_designation": null,
            "sensitivity_access_notes": null,
            "created_at": "2015-07-20T18:12:39+00:00",
            "updated_at": "2015-07-20T18:12:39+00:00",
            "created_by": null,
            "updated_by": null,
            "id": "ed820017-24d9-4895-aea2-2a899c83987a"
        }
    },
    {
        "permission_group": {
            "name": "my group",
            "created_at": "2015-07-20T18:14:29+00:00",
            "updated_at": "2015-07-20T18:14:29+00:00",
            "created_by": null,
            "updated_by": null,
            "id": "7f6d87cf-a2d3-488e-be03-ca4087e48986"
        }
    }
]

But it's missing associations. I know I have them when I query

2.0.0-p353 :010 > t = Term.first
 => #<Term access_designation: nil, created_at: Mon, 20 Jul 2015 18:12:39 +0000, created_by: nil, data_availability: nil, definition: "New York", name: "Awesome Term", notes: nil, possible_values: nil, sensitivity_access_notes: nil, sensitivity_classification: nil, source_system: nil, updated_at: Mon, 20 Jul 2015 18:12:39 +0000, updated_by: nil>
2.0.0-p353 :011 > t.permission_group
 => #<PermissionGroup created_at: Mon, 20 Jul 2015 18:14:29 +0000, created_by: nil, name: "my group", updated_at: Mon, 20 Jul 2015 18:14:29 +0000, updated_by: nil>
2.0.0-p353 :012 > t.permission_group.name
 => "my group"

So my question is how do I output all nodes with their properties and associations? Is there a fancy neo4j.rb magic method or do I have to write out the cypher query by hand?

Answer 1

This is a good question. You just reminded me that I've wanted to create methods to get the IDs for associations. I just put together some working code which will go into master (still needs tests) which provides the following:

t.permission_group_id # UUID
t.permission_group_neo_id # Neo4j ID

pg.term_ids
pg.term_neo_ids

Obviously that won't be out for a bit. Also the to_json for ActiveNode would need to change to output these IDs in the JSON (maybe based on an option). Also there are ActiveRel models to be considered.

All that being said I'm not sure that this is the best way to go about doing backup. The problem is that if you go to restore you'll have models on both sides of the relationship trying to create that relationship which means that you'll end up with two relationships. You can do a MERGE, but there are also times when you want two relationships ;)

So I would suggest using the built in Neo4j backup tools. There is the neo4j-backup tool:

http://neo4j.com/docs/stable/re04.html

Though I think that might only be in enterprise (I'm not seeing it in my community install).

There is also the dump command from Cypher:

http://neo4j.com/docs/stable/shell-commands.html#_dumping_the_database_or_cypher_statement_results

Which I note in the docs is an experimental feature.

Does that address your issue? If not I can look into other solutions ;)

Answer 2

I still think that backup up via Neo4j directly is best. Since you said that your database wasn't very large you should be able to do some simple Cypher queries like this:

MATCH n RETURN n

MATCH (a)-[r]->(b) RETURN a, r, b

Depending on what information you get back for the r you may be able to do just:

MATCH ()-[r]->() RETURN r

In Neo4j.rb you can run a Cypher query like this:

Neo4j::Session.current.query('MATCH n RETURN n')

or like this:

Neo4j::Session.current.query.match(:n).pluck(:n)