Stack Overflow
Stack Overflow

Questions tagged [giraph]

Apache Giraph is an iterative graph processing system built for high scalability.

Apache Giraph is an iterative graph processing system built for high scalability.

For example, it is currently used at Facebook to analyze the social graph formed by users and their connections.

Giraph originated as the open-source counterpart to Pregel, the graph processing architecture developed at Google and described in this paper.

Both systems are inspired by the Bulk Synchronous Parallel model of distributed computation introduced by Leslie Valiant.

Bulk Synchronous Parallel (BSP) abstract computer is a bridging model for designing parallel algorithms. It differs from Parallel random access machine (PRAM) by not talking communication and synchronization for granted. An important part of analyzing a BSP algorithm rests in qualifying the synchronization and the communication needed.

Giraph adds several features beyond the basic Pregel model, including master computation, sharded aggregators, edge-oriented input, out-of-core computation, and more.

With a steady development cycle and a growing community of users worldwide, Giraph is a natural choice for unleashing the potential of structured datasets at a massive scale.

References

156 questions
0
votes
3 answers

Which is better: sending many small messages or fewer large ones?

I have an app whose messaging granularity could be written two ways - sending many small messages vs. (possibly far) fewer larger ones. Conceptually what moves around is a set of 'alive' vertex IDs that might get filtered at each superstep based on…
Matthew Cornell
  • 4,114
  • 3
  • 27
  • 40
0
votes
1 answer

Which is better to use to manage Vertex state: POJO instance variables or Giraph values?

I'm confused about when it's OK to use Vertex instance variables to maintain state rather than proper Giraph values ala getValue(). An interesting example I found in the source demonstrates both: SimpleTriangleClosingVertex, which has both an…
Matthew Cornell
  • 4,114
  • 3
  • 27
  • 40
0
votes
1 answer

Error running giraph on cloudera CDH5.1.0

So, I have Hadoop 2.3.0-cdh5.1.0 installed on my cluster... and I compiled giraph thru the following command.. mvn clean package -DskipTests -Dhadoop=non_secure -Phadoop_2.0.0 So, the compilation worked just fine and then the pagerank benchmark also…
frazman
  • 32,081
  • 75
  • 184
  • 269
0
votes
2 answers

NoClassDefFoundError with gradle, giraph, and hadoop

So, I've been looking around a lot and I haven't found a good answer to my question, and this is driving me crazy, so I figured I'd ask here and hopefully I can get help. I'm trying to do automated testing in a Giraph project using gradle. I'm a…
Matthew Saltz
  • 385
  • 4
  • 8
0
votes
2 answers

Apache Giraph Graph Partitioning.... Can a partition p1 resides partially in worker w1 and partially in worker w2?

I am newbie in Apache Giraph. My question is related to Giraph graph partitioning. As far as I know, Giraph partition the large graph randomly.... possibly #partitions>#workers in order to load balance. But, my question is, is #partitions/worker…
user3606212
  • 83
  • 2
  • 7
0
votes
1 answer

Running a Giraph job showing the following error

INFO zookeeper.ClientCnxn: Opening socket connection to server debashis-Pseudo-Hadoop/127.0.1.1:22181. Will not attempt to authenticate using SASL (unknown error) 14/06/17 13:07:28 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected…
Debashisenator
  • 1,621
  • 4
  • 17
  • 16
0
votes
1 answer

How to process a large file in Hadoop?

This is a noobie question I have a hadoop setup and thinking of uisng Giraph or Hama for graph based computation. I have a large file in the form 3 4 3 7 3 8 5 6 where each column denotes vertices and each row denote edges. For normal programs I…
user567879
  • 5,139
  • 20
  • 71
  • 105
0
votes
1 answer

cannot find jars for the imported Giraph packages

I was trying to run the SSSP giraph program from Eclipse. But I couldnt get the following packages imported import org.apache.giraph.graph.BasicVertex; import org.apache.giraph.graph.BspUtils; import org.apache.giraph.graph.GiraphJob; import…
user567879
  • 5,139
  • 20
  • 71
  • 105
0
votes
1 answer

Compiling a project that uses old-versioned Giraph with Maven

I have a project that was developed from another group and uses old-versioned Giraph. The project is made based on the Giraph project itself; The project structure and files are exactly same to the Giraph, except that some .java sources for more…
aqjune
  • 478
  • 1
  • 3
  • 17
0
votes
1 answer

Giraph ZooKeeper port problems

I am trying to run the SimpleShortestPathsVertex (aka SimpleShortestPathComputation) example described in the Giraph Quick Start. I am running this on a Hortonworks Sandbox instance (HDP 2.1) using VirtualBox, and I packaged giraph.jar using…
Mahana
  • 128
  • 1
  • 8
0
votes
1 answer

Which version of CDH using Cloudera Manager automatically Installs JDK1.7?

I am using Cloudera Manager with CDH4.2.2 for my 3+1 cluster. On starting the installation with cloudera manager, it automatically downloads and installs JDK1.6. I want to use JDK1.7 with CDH for my convinience. Is it possible or is there any…
Debashisenator
  • 1,621
  • 4
  • 17
  • 16
0
votes
1 answer

compilation error with apache giraph (building from the trunk)

===================Below is the error============================ [INFO] Building jar: /home/rob/dev/apache/giraph-trunk/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar [INFO] Total time:…
user3220461
  • 1
0
votes
2 answers

zookeeper configs for Giraph 1.0 on Hadoop 2.2.0

New to stack exchange and Giraph so please overlook mistakes and ask any clarifying questions. OS: ubuntu 13.10 Hadoop/Yarn: hadoop-2.2.0/ (2-node cluster) Giraph: 1.0.0 (EDIT: trunk) I'm getting a NullPointerException (NPE) when I attempt to run…
Vishal
  • 1,253
  • 1
  • 11
  • 17
0
votes
1 answer

Trouble running a giraph job (classnotfoundexception)

I'm attempting to build the SimpleShortestPathsComputation example included with Giraph and run it from within my home directory. Basically, I'm just trying to tweak the SimpleShortestPaths example and run it without any hassle (not quite sure what…
the_man_slim
  • 1,155
  • 2
  • 11
  • 18
0
votes
1 answer

giraph-formats - build jar with dependencies

I am trying to build giraph-formats jar...with dependencies.. to use it for my project, but even after a successfull built, I am not able to get the jar file. I have tried to add assembly part to pom, to build jar with dependencies, but it misses…
Pradeep
  • 243
  • 1
  • 3
  • 10
1 2 3
…
10
11