Stack Overflow
Stack Overflow

Questions tagged [giraph]

Apache Giraph is an iterative graph processing system built for high scalability.

Apache Giraph is an iterative graph processing system built for high scalability.

For example, it is currently used at Facebook to analyze the social graph formed by users and their connections.

Giraph originated as the open-source counterpart to Pregel, the graph processing architecture developed at Google and described in this paper.

Both systems are inspired by the Bulk Synchronous Parallel model of distributed computation introduced by Leslie Valiant.

Bulk Synchronous Parallel (BSP) abstract computer is a bridging model for designing parallel algorithms. It differs from Parallel random access machine (PRAM) by not talking communication and synchronization for granted. An important part of analyzing a BSP algorithm rests in qualifying the synchronization and the communication needed.

Giraph adds several features beyond the basic Pregel model, including master computation, sharded aggregators, edge-oriented input, out-of-core computation, and more.

With a steady development cycle and a growing community of users worldwide, Giraph is a natural choice for unleashing the potential of structured datasets at a massive scale.

References

156 questions
0
votes
0 answers

Giraph Example ShortestPathComputation reports FAILED state

I have installed and deployed giraph-1.1.0 using the hadoop yarn profile with the following command mvn -Phadoop_yarn -Dhadoop.version=2.6.0 -DskipTests clean package From the folder giraph-examples/target I tried running the…
Sai Ganesh Muthuraman
  • 111
  • 7
0
votes
1 answer

Container gets killed on Worker, when doing flush, after completing superstep, and the entire application hangs - Giraph

I'm running a Giraph application on EMR. I'm using a cluster of 1 master and 10 slaves, all m3.2xlarge machines. The application consist, basically, on a BFS through the spanish version of Wikipedia (i adapted the Wikipedia information for fitting…
chomp
  • 1,352
  • 13
  • 31
0
votes
0 answers

Hadoop 0.20.203.0-RC1 reduce stuck at 0%

I am following this guide below to install Apache Giraph. http://giraph.apache.org/quick_start.html I have followed all the steps correctly but when I try to run the wordcount example given in the link, the program gets stuck at 16/08/13 14:49:29…
Kakaji
  • 1,421
  • 2
  • 15
  • 23
0
votes
0 answers

NoClassDefFoundError hadoop/yarn/client/YarnClientImpl

I installed giraph-1.0.0 on my 2-node cluster, built with hadoop version 2.0.4-alpha where as i have hadoop 2.7.1 installed on my cluster. When I tried to run the giraph example for Simple Shortest Path Vertex, I got the following…
Sai Ganesh Muthuraman
  • 111
  • 7
0
votes
1 answer

What are the factors to consider while choosing a Graph DB for about 30 TB data

I'm in the process of developing a software system ( Graph Database ) to study the interconnection between multiple components. It could end up with about 30 TB of data. I would like to know what all factors to consider in choosing the right…
pjesudhas
  • 399
  • 4
  • 13
0
votes
1 answer

Apache Giraph/Hadoop: Iterating through custom ArrayWritable

I thought this would be simple to implement, but it's starting to be a pain. I've got a ArrayWritable subclass like so: public class VertexDistanceArrayWritable extends ArrayWritable { public VertexDistanceArrayWritable() { …
user3760657
  • 397
  • 4
  • 16
0
votes
1 answer

Giraph ShortestPath demo never exits, Patch 756 already applied (I think)

I am a novice to Hadoop and Giraph. I am trying to run the Giraph ShortestPaths example using Giraph 1.1 on our server, which is running YARN. After much hair-pulling, I finally got it to run. Now the problem is to get it to stop. The giraph process…
Edward Swing
  • 121
  • 1
  • 9
0
votes
1 answer

Could not find or load main class 256 - Yarn cluster

i'm currently running a single node yarn cluster, and for some reason, i can't execute even a example that comes with map reduce (grep, wordcount, etc). With this line i execute grep: $HADOOP_HOME/bin/yarn jar…
chomp
  • 1,352
  • 13
  • 31
0
votes
1 answer

Halt giraph when all vertex have converged

I'm new at Giraph, so maybe my question has an obvious response. I'm implementing an algorithm on Giraph that needs to stop when all vertex have converged and do some computations afterwards. My approach was halting every vertex once it has…
joseprupi
  • 317
  • 4
  • 13
0
votes
1 answer

How to pipe the complete graph to Giraph through TinkerPop 3 stack?

I've a graph with different types of nodes & relationships. Each type of node has 3-4 properties. For testing purpose on HDFS, I'm using GraphSON file to store this graph. Now I want to analyse this graph using Giraph. I've explore Giraph's IO…
Parth
  • 729
  • 8
  • 23
0
votes
1 answer

Is it possible to manipulate the graph loaded in memory by Giraph?

I've read that Giraph is a graph analysis framework, but is it anyway possible to add, remove or update the nodes & edges of the graph loaded in memory by Giraph?
Parth
  • 729
  • 8
  • 23
0
votes
1 answer

Can Giraph reuse the graph in memory from previous job for subsequent jobs?

I've around million nodes & billion edges, so does Giraph keep the graph in memory such that it can be used again for subsequent jobs or does it loads & stores the complete graph before & after every job respectively?
Parth
  • 729
  • 8
  • 23
0
votes
0 answers

Giraph ShortestPathExample java.net.ConnectException: Connection refused

I have already run the SimpleShortestPathsComputation on single node pseudo-distribution and I am currently trying to run it on a 2 node cluster on Ubuntu 14.04. But every time I got the following Error: (Complete Logs below) ERROR…
Rofti
  • 16
  • 3
0
votes
2 answers

is hadoop necessary for using Giraph

I want to use Giraph as a graph processing tool for my work. I am familiar with Mahout and I know I can use some parts of Mahout such as Recommender System without using Hadoop. However, I don't know if this is the same for Giraph and whether I can…
HimanAB
  • 2,443
  • 8
  • 29
  • 43
0
votes
3 answers

How to set SplitMasterWorker value as false in giraph

I try to execute the giraph custom code from eclipse IDE, and when i try to run the code its showing Exception in thread “main” java.lang.IllegalArgumentException: checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have only one…
ldmi
  • 7
  • 5
1 2 3
…
10 11