Serialization in Hadoop - Writable

Question

This is the class that implements Writable ..

public class Test implements Writable {
    List<AtomicWritable> atoms = new ArrayList<AtomicWritable>();

    public void write(DataOutput out) throws IOException {
        IntWritable size = new IntWritable(atoms.size());
        size.write(out);
        for (AtomicWritable atom : atoms)
            atom.write(out);
    }

    public void readFields(DataInput in) throws IOException {
        atoms.clear();
        IntWritable size = new IntWritable();
        size.readFields(in);
        int n = size.get();
        while(n-- > 0) {
            AtomicWritable atom = new AtomicWritable();
            atom.readFields(in);
            atoms.add(atom);
        }
    }
}

I will really appreciate if one can help me understand how to invoke write and readFields method. Basically I m failing to understand how to construct Test object in this case. Once the object is written to DataOutput obj, how do we restore it in DataInput object. This may sound silly, but am a newbie to Hadoop and have been assigned a project that uses Hadoop. Please help.

Thanks!!!

Small pro tip: if you're going to be asking a lot of [questions here](http://stackoverflow.com/users/2503555/rekha-gupta?tab=questions), especially on a less crowded subject like Hadoop (less crowded than C# and Java), make sure that you upvote and accept answers for questions that you've [previously asked](http://stackoverflow.com/questions/17220884/how-to-serialiaze-list-collection-object-in-hadoop). If you don't do this, you'll quickly run out of sympathy with the community. — jason, Jun 21 '13 at 05:49

jason · Accepted Answer · 2013-06-21T05:58:30.260

0

Basically I m failing to understand how to construct Test object in this case.

Yup, you're missing the point. If you need to construct an instance of Test and populate atoms, then you need to add a constructor to Test:

public Test(ArrayList<AtomicWritable> atoms) {
     this.atoms = atoms;
}

or you need to use the default constructor and add a method or a setter that lets you add items to atoms or set the value of atoms. The latter is actually pretty common in the Hadoop framework, to have a default constructor and a set method. cf., e.g., Text.set.

You don't call readFields and write; the Hadoop framework does that for you when it needs to serialize and deserialize inputs and outputs to and from map and reduce.

edited Jun 21 '13 at 05:58

answered Jun 21 '13 at 05:48

jason

236,483
35
423
525

never forget the default constructor ;-) – Thomas Jungblut Jun 21 '13 at 05:56
@Thomas Jungblut: He can't set `atoms` then (so he either needs a constructor that lets him do that, or a setter). Point taken though, I'll make a small edit. Thanks. – jason Jun 21 '13 at 05:57
Thanks a lot Jason and Thomas!! This helps!! Also, appreciate your tip above about posting questions :) Am very new to this site. – rekha gupta Jun 21 '13 at 06:57

Serialization in Hadoop - Writable

1 Answers1