7

Lets say i have a class Graph defined.

graph iowa = {.....nodes:{edges.......}}

and similar graph clusters.

once the script is running its all in the object.

But how do you store the Graph info (including whatever its attributes were defined) ? Do you save it in a db table, csv ?

Possible unsatisfactory queries: Storing multiple graphs in Neo4J (It says newo4j one instance supports one Graph at a time and to subgraph under one graph if you want to run multiple graphs)

How to store graph data in a database?

*****UPDATE*****: On Alexanders answer below on available Graph DBs, I would like to know if anyone knows of a possible fork of VertexDB where they've imported this high performance lib into any other language preferably NodeJS or Python?

http://www.dekorte.com/projects/opensource/vertexdb/docs/manual.html

And does anyone have experience reading this in using AllegroGraph vs Neo4j?

Community
  • 1
  • 1
user2290820
  • 2,709
  • 5
  • 34
  • 62
  • 1
    adjacent matrix and edge lists are the most simple formats (with some king of key-value database for payloads). If you do not want to implement it, use pickle : http://pymotw.com/2/pickle/ . (just be careful of recursion limit) – lucasg Oct 24 '13 at 12:59
  • @georgesl o i can use numpy to create matrices and store values and keys. But then again, where do I store these matrice values /which format? – user2290820 Oct 24 '13 at 13:13

3 Answers3

9

The NetworkX python library provides a couple of ways to store graph data:

  1. Serialize a graph to JSON
  2. Pickle a graph
  3. Store a graph in the gml format (link)
  4. Write graph to .dot file (link).

Storing graphs in a relational database is certainly a bad option, because of complexity of the data, complexity of graph queries, which is hard to write in SQL and the performance of these queries. Graph Databases, NOSQL and Neo4j article describes this problem in great details.

However, I think, you should really consider to use Neo4j. I hope after some investigations, you'll find a way to adapt it to your needs. After all, there are a lot of alternatives to the Neo4j.

DB-engines is a great resource to visit. There are a lot of useful information, in particular, you can compare any database with another and find the one, that best suites you.

Also, you can refer to this beautiful answer.

Update:

Since you are using structures, similar to JSON, you can consider using JSON-oriented databases, such as CouchDB or MongoDB. They have decent perfomance and scale better, than Neo4j.

Community
  • 1
  • 1
Alexander Zhukov
  • 4,357
  • 1
  • 20
  • 31
  • 1
    +1 for all practical solutions you have given. Thanks. Im starting to use NetworkX but created my own graph module from scratch since I felt for small scale use and just starting out it would be better for me to understand and ofcourse debugging takes time but then NetworkX Tutorials and documentations are lengthy, for graph types, and implementations of algos for various graphs,etc. – user2290820 Oct 24 '13 at 13:16
2

If you have your graph data stored in a dictionary it is pretty simple to write this to a json file with json.dump(), and load it back into memory with json.load(). Or if it is a list of dictionaries you would use json.dumps() and json.loads(). Python docs here.

ChrisProsser
  • 12,598
  • 6
  • 35
  • 44
  • gosh yeah, you are right. yes its in dict form. Hey checkout ujson on that note. So you mean to say store all graphs as json format , one file per graph-json ? – user2290820 Oct 24 '13 at 13:15
  • That is certainly an option. Depending on how large your files are it might take a while to load them in while starting your program up, but you should have fast access to the data once it is loaded into dictionaries. – ChrisProsser Oct 24 '13 at 13:29
2

The VertexDB implements a RESTful API to provide a query tools, so this is not gonna be a problem to use it with python, as the only thing you'll need is to write appropriate HTTP requests to the server.

Alexander Zhukov
  • 4,357
  • 1
  • 20
  • 31
  • I also like the link you gave http://stackoverflow.com/questions/9302295/nosql-solution-for-persisting-graphs-at-scale/9645518#9645518 ; I think Ill go with that. I should also probably look into this later on when Im comfortable with that since this is pure C implementation, should be faster I guess – user2290820 Oct 24 '13 at 13:37