0

I am experimenting around with Giraph. To run the algorithms in Giraph I need the graph data to be in Vertex Input Format. Almost all the available Big Data online is in Edge List Format. I wrote a code in Java to convert this Edge List format into VertexInputFormat. This works for smaller graphs with almost 800k edges. However for the graph that I need, every time I run the program its giving me Heap space exceeded error. I tried increasing the Heap size to maximum. Still the error persisted.

The file on which I am running is about 15GB in size.

I don't know much about how the algorithms(PageRank, SingleSourceShortestPath etc..,) are written in Giraph but I do know that they all take a graph in VertexInputFormat as input.

The help I am looking for is:

  1. An optimized code to convert EdgeInputFormat to VertexInputFormat (or)
  2. Any Online tool that can help in this conversion (or)
  3. PageRank algorithm that takes EdgeInputFormat as input.
Community
  • 1
  • 1
natcomp256
  • 706
  • 7
  • 13

1 Answers1

0

Sorry, I didn't get the point on why you want to use the VertexInputFormat only, Giraph also provides EdgeInputFormat API, why can't you use that?

Syden
  • 8,425
  • 5
  • 26
  • 45
saleem
  • 1
  • 2
  • I downloaded Giraph from the official page and it is already compiled and there is a Snapshot available. I understood that it is the snapshot of all the algorithms implemented by default. They too are available in `$GIRAPH_HOME/giraph-examples/../org/apache/giraph/examples` folder. All the algorithms in this folder takes input in VertexInputFormat. You mentioned EdgeInputFormat API. Could you provide more insight into that? Thanks for the concern. – natcomp256 Jan 07 '17 at 12:16
  • You can check the giraph-core/.../org/apache/giraph/io/formats folder for all the available input formats, When are you running the program you simply say -eif (eif stands for edge input formats) and -eip (edge input path) instead of -vif and -vip. And you can also implement your own input formats. – saleem Jan 09 '17 at 06:34