Giraph: Using Text as VertexId

Question

I try to test Giraph.

VertexId type Text

Input Edge-Base

If I use Text as VertexId, I get error. If LongWritable, everything is OK.

Questions: 1. Is it OK to use Text as VertexId? 2. If yes, what an I doing wring?

Error:

14/10/15 14:59:28 INFO worker.InputSplitsCallable: call: Loaded 1 input splits in 0.08243016 secs, (v=0, e=12) 0.0 vertices/sec, 145.57777 edges/sec
14/10/15 14:59:28 ERROR utils.LogStacktraceCallable: Execution of callable failed
java.lang.ArrayIndexOutOfBoundsException
    at java.lang.System.arraycopy(Native Method)
    at org.apache.giraph.utils.UnsafeArrayReads.readFully(UnsafeArrayReads.java:103)
    at org.apache.hadoop.io.Text.readFields(Text.java:265)
    at org.apache.giraph.utils.ByteStructVertexIdDataIterator.next(ByteStructVertexIdDataIterator.java:65)
    at org.apache.giraph.edge.AbstractEdgeStore.addPartitionEdges(AbstractEdgeStore.java:161)

Custom format:

public class TextDoubleTextEdgeInputFormat extends
    TextEdgeInputFormat<Text, DoubleWritable> {
  /** Splitter for endpoints */
  private static final Pattern SEPARATOR = Pattern.compile("[\t ]");

...

Main class:

public class HelloWorld extends
        BasicComputation<Text, Text, NullWritable, NullWritable> {

    @Override
    public void compute(Vertex<Text, Text, NullWritable> vertex,
            Iterable<NullWritable> messages) {
        System.out.print("Hello world from the: " + vertex.getId().toString()
                + " who is following:");
        for (Edge<Text, NullWritable> e : vertex.getEdges()) {
            System.out.print(" " + e.getTargetVertexId());
        }
        System.out.println("");
        vertex.voteToHalt();

    }

    public static void main(String[] args) throws Exception {
        System.exit(ToolRunner.run(new GiraphRunner(), args));
    }
}

Test starter:

@Test
public void textDoubleTextEdgeInputFormatTest() throws Exception {

    String[] graph = { "1 2 1.0", "2 1 1.0", "1 3 1.0", "3 1 1.0",
            "2 3 2.0", "3 2 2.0", "3 4 2.0", "4 3 2.0", "3 5 1.0",
            "5 3 1.0", "4 5 1.0", "5 4 1.0" };

    GiraphConfiguration conf = new GiraphConfiguration();
    conf.setComputationClass(HelloWorld.class);
    conf.setEdgeInputFormatClass(TextDoubleTextEdgeInputFormat.class);
    // conf.setEdgeInputFormatClass(TextLongL6TextEdgeInputFormat.class);

    // conf.setVertexOutputFormatClass(IdWithValueTextOutputFormat.class);

    InternalVertexRunner.run(conf, null, graph);
}

If you could resolve this, it will be very useful that you share your answer with the rest of this community. — chomp, Jul 30 '15 at 21:18

score 0 · Answer 1 · answered Feb 01 '15 at 22:35

I'm not experienced in Giraph but in Apache SparkX the VertexId has type Long.

I wouldn't be surprised if design patterns of Apache Giraph are reused in Apache Spark GraphX. I therefore guess that type Long is the way to go because your implementation of type LongWritable is successful.

Giraph: Using Text as VertexId

1 Answers1