How to pass string as value in mapper?

Question

I am trying to pass a string as value in the mapper, but getting error that it is not Writable. How to resolve?

public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {

    String TempString = value.toString();
    String[] SingleRecord = TempString.split("\t");

    //using Integer.parseInt to calculate profit
    int Amount = Integer.parseInt(SingleRecord[7]);
    int Asset = Integer.parseInt(SingleRecord[8]);
    int SalesPrice = Integer.parseInt(SingleRecord[9]);
    int Profit = Amount*(SalesPrice-Asset);

    String ValueProfit = String.valueOf(Profit);
    String ValueOne = String.valueOf(one);

    custID.set(SingleRecord[2]);
    data.set(ValueOne + ValueProfit);
    context.write(custID, data);

}

What is `data` and what is `custId`? – Thomas Jungblut Nov 11 '14 at 17:43 — Thomas Jungblut, Nov 11 '14 at 17:43

score 0 · Answer 1 · answered Nov 12 '14 at 06:15

0

Yahoo's tutorial says :
Objects which can be marshaled to or from files and across the network must obey a particular interface, called Writable, which allows Hadoop to read and write the data in a serialized form for transmission.

From Cloudera site :
The key and value classes must be serializable by the framework and hence must implement the Writable interface. Additionally, the key classes must implement the WritableComparable interface to facilitate sorting.

So you need an implementation of Writable to write it as a value in the context. Hadoop ships with a few stock classes such as IntWritable. The String counterpart you are looking for is the Text class. It can be used as :

context.write(custID, new Text(data));

OR

Text outValue = new Text();
val.set(data);
context.write(custID, outValue)

I case, you need specialized functionality in the value class, you may implement Writable (not a big deal after all). However seems like Text is just enough for you.

answered Nov 12 '14 at 06:15

blackSmith

3,054
1
20
37

we tried your suggestion, but still getting an error: Multiple markers at this line - The constructor Text(IntWritable) is undefined - The method write(Text, TextWritable) in the type TaskInputOutputContext is not applicable for the arguments (Text, Text) – Raunak Tibrewal Nov 12 '14 at 17:36
Perhaps `data` is an `IntWritable`. Please share complete code of your `Mapper` to get a quick solution. – blackSmith Nov 13 '14 at 12:18
`public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { String TempString = value.toString(); String[] SingleRecord = TempString.split("\t"); int Amount = Integer.parseInt(SingleRecord[7]); int Asset = Integer.parseInt(SingleRecord[8]); int SalesPrice = Integer.parseInt(SingleRecord[9]); int Profit = Amount*(SalesPrice-Asset); String ValueProfit = String.valueOf(Profit); String ValueOne = String.valueOf(one); custID.set(SingleRecord[2]); context.write(custID, new Text(data)); }` – Raunak Tibrewal Nov 13 '14 at 18:16
I indeed meant `Mapper` - the whole class, not only the `map` method, which you've already shared. The main issue with `data` - it's definitely not a string. Or you've messed up with imports. – blackSmith Nov 14 '14 at 05:01

score 0 · Answer 2 · answered Dec 18 '15 at 10:09

0

you havent set data in map function according to import text in above,and TextWritable is wrong just use Text as well.

answered Dec 18 '15 at 10:09

NikNik

11
1
4

Hello and welcome to StackOverflow! It will be a lot helpful to provide a code snippet or an example (see [how to answer](http://stackoverflow.com/help/how-to-answer)). – Jia Jian Goi Dec 18 '15 at 10:32

How to pass string as value in mapper?

2 Answers2