Stack Overflow
Stack Overflow

Questions tagged [giraph]

Apache Giraph is an iterative graph processing system built for high scalability.

Apache Giraph is an iterative graph processing system built for high scalability.

For example, it is currently used at Facebook to analyze the social graph formed by users and their connections.

Giraph originated as the open-source counterpart to Pregel, the graph processing architecture developed at Google and described in this paper.

Both systems are inspired by the Bulk Synchronous Parallel model of distributed computation introduced by Leslie Valiant.

Bulk Synchronous Parallel (BSP) abstract computer is a bridging model for designing parallel algorithms. It differs from Parallel random access machine (PRAM) by not talking communication and synchronization for granted. An important part of analyzing a BSP algorithm rests in qualifying the synchronization and the communication needed.

Giraph adds several features beyond the basic Pregel model, including master computation, sharded aggregators, edge-oriented input, out-of-core computation, and more.

With a steady development cycle and a growing community of users worldwide, Giraph is a natural choice for unleashing the potential of structured datasets at a massive scale.

References

156 questions
0
votes
1 answer

How to write and run apache Giraph Custom code?

I have been working on giraph from last 10 days.I got the ideas how to install and execute the given examples in Giraph. But I want to design my own custom code,so i need some help of you.If anyone is done with this please let me know and give some…
ldmi
  • 7
  • 5
0
votes
2 answers

Confuse between hadoop, giraph and twister

I am working on Big Data and my project consist of graphs having text data.I have to calculate similarity between vertices, hop probabilities, number of connected components, page rank vector, and random walk in one project itself. I implemented…
Nikhil Bhaware
  • 13
  • 5
0
votes
2 answers

Error using Apache Giraph

I am receiving the following error when I run the SimpleOutDegreeCountComputation proogramme in Apache Giraph. I am using JsonLongDoubleFloatDouble as the input format. $HADOOP_HOME/bin/hadoop jar …
Deepa Kumari
  • 11
0
votes
0 answers

PageRank job on Apache giraph is not producing output file

Command: hduser@bigtapp:/usr/local/giraph$ hadoop jar /usr/local/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.7.0-jar-with-dependencies.jar org.apache.giraph.GiraphRunner…
ldmi
  • 7
  • 5
0
votes
1 answer

Hadoop 2.4.0 and Giraph 1.1.0 in Amazon EMR?

I started with hadoop recently, and I’m trying to use it with giraph (because i need it for manipulating huge graphs). So, for building giraph (1.1.0, latest stable version), I chose the 2.4.0 release of hadoop (according to this link, it's the…
chomp
  • 1,352
  • 13
  • 31
0
votes
1 answer

Apache Giraph : Number of vertices processed by each partition

I am a newbie trying to understand the working of Giraph 1.2.0. with hadoop 1.2.1. Is there any way to figure out the number of vertices processed by each mapper?
Ravikant
  • 71
  • 2
  • 10
0
votes
1 answer

PageRankBenchmark - Reduce at 0% - Giraph example

I’m testing my Single Node Cluster Giraph installation using PageRankBenchmark example, in the following way: $HADOOP_HOME/bin/hadoop jar /usr/local/hadoop/share/hadoop/mapreduce/giraph-examples-1.1.0-for-hadoop-2.4.0-jar-with-dependencies.jar…
chomp
  • 1,352
  • 13
  • 31
0
votes
1 answer

How to configure GIRAPH 1.1.0 pom.xml for it to work with HADOOP 2.6.0 ? Installation ERROR

Im trying to add GIRAPH 1.1.0 to HADOOP 2.6.0 I have to edit the pom.xml somehow in order to package GIRAPH correctly. I run the command mvn -Phadoop_yarn -Dhadoop.version=2.6.0 package I edited the default pom.xml file in the line (1292) :…
anu
  • 39
  • 5
0
votes
1 answer

Giraph's estimated cluster heap 4096MB ask is greater than the current available cluster heap of 0MB. Aborting Job

I'm running Giraph using Hadoop 2.5.2 on a 5 node cluster. But when I try to run the SimpleShortestPathsComputation example, I get this error: Exception in thread "main" java.lang.IllegalStateException: Giraph's estimated cluster heap 2000MB ask…
mindcrime
  • 657
  • 8
  • 23
0
votes
1 answer

Trouble running Apache Giraph on YARN cluster (Hadoop 2.5.2)

I'm trying to run the basic ShortestPaths example using Giraph 1.1 on Hadoop 2.5.2. I'm running in actual cluster model (eg, not psuedo-distributed) and I can run standard mapreduce jobs OK. But when I try to run the Giraph example, it seems to…
mindcrime
  • 657
  • 8
  • 23
0
votes
1 answer

Maintain an array structure per Vertex

Throughout a Giraph graph, I need to maintain an array on a Vertex basis to store the results of several "health" checks done at the Vertex level. If it as simple as writing a new Input format that will get carried over? My worry goes to the fact…
gdoubleu
  • 72
  • 1
  • 7
0
votes
1 answer

Giraph Shortest Path Examples

I have problem running the given example SimpleShortestPathsComputation with the tiny_graph.txt. I always get stuck at the process of Reduce Part after the Map part. It always shows me 100% maps and 0% reduced and hangs there all the time. Having a…
Alfred
  • 59
  • 1
  • 1
  • 2
0
votes
1 answer

What mechanism that Giraph's workers do when receiving messages in vertices?

I am curious, in Giraph's worker API documentation, I see an explanation about this method: public void storeCheckpoint() // Both the vertices and the messages need to be checkpointed in order for them to be used. // This is done after all messages…
Algorithman
  • 1,309
  • 1
  • 16
  • 39
0
votes
1 answer

Giraph and Cassandra

Did anybody try to use Giraph and DSE Cassandra? I try to run but process hangs: 14/10/21 16:38:24 INFO mapred.JobClient: Running job: job_201410211229_0028> 14/10/21 16:38:25 INFO mapred.JobClient: map 80% reduce 0% Command line is: dse hadoop…
pavel
  • 29
  • 6
0
votes
1 answer

How to read LZO compressed HDFS files in Giraph

I am looking for an input format for Giraph which can read LZO compressed files. It appears the current input format GiraphRunner uses by default is BspInputFormat which has no mention of LZO compression. Is this simply an oversight and I'll have…
user2744380
  • 31
  • 2
1 2 3
…
10 11