Mapreduce mapper to pass both key and value as text fields

Question

I have a small project where I have a file with 2 tab-separated text columns. I want my mapper to read them from the file and set the second column as key and the first column as value to the reducer, but I can't seem to get it done. Then, I want to pass the (key, value) pairs to reducer where it will create for each key a list of all its values.

public static class TokenizerMapper extends Mapper<Object, Text, Text, IntWritable> {

    private final static IntWritable one = new IntWritable(1);
    private Text site = new Text();
    private Text tag = new Text();

    public void map(Object key, Text value, Context context) 
                        throws IOException, InterruptedException {

        StringTokenizer itr = new StringTokenizer(value.toString(), "\t");
        while (itr.hasMoreTokens()) {
            site.set(itr.nextToken());
            tag.set(itr.nextToken());
            context.write(tag, site);
        }
    }
}

I am getting an error on the conext.write line. How can I solve this?

score 3 · Accepted Answer · answered Dec 07 '16 at 08:25

The value (site) you're writing in context.write(tag, site); is a Text object but you have Mapper<Object, Text, Text, IntWritable>. You should change this to Mapper<Object, Text, Text, Text>.

At the moment you're telling it you're going to output an IntWritable as the value.

You can also remove private final static IntWritable one = new IntWritable(1); since it isn't being used.

Mapreduce mapper to pass both key and value as text fields

1 Answers1