0

I am modifying an MapReduce program in the Record Reader and wanted to write a test case for mapper to call customised InputFormat or Record Reader. I have modified test case for record reader but the record reader test cases are not an mrunit one.

How to call the customised Record Reader from Mapper, as withInputFormat function is not listed under MapDriver.newMapDriver?

Please find the snapshot of my code:

@Before
public void setUp(){
MapDriver<Object, Text, Text, Text> mapDriver = MapDriver.newMapDriver(new myMapper());
}

@Test
public void testFunction1() throws IOException {
mapDriver.withInput(...).withOutput(...).runTest();
}

Thanks

Aavik
  • 967
  • 19
  • 48
  • well, `MRUnit` has in its API `withOutputFormat` method. I guess there will be `withInputFormat`. This is a good question!! – Kenry Sanchez Feb 13 '20 at 04:17

1 Answers1

0

I was checking the MRUnit API, so. If you want to add a custom RecordReader i would guess you must have a custom InputFormat due to must assign the custom RecordReader into the createRecordReader method of the custom InputFormat class.

So, MRUnit API allows you to assign a custom InputFormat with also the custom OutputFormat.

public MapDriver<K1,V1,K2,V2> withOutputFormat(Class<? extends org.apache.hadoop.mapreduce.OutputFormat> outputFormatClass,
                                      Class<? extends org.apache.hadoop.mapreduce.InputFormat> inputFormatClass)

Configure Mapper to output with a real OutputFormat. Set InputFormat to read 
output back in for use with run* methods

Parameters:
outputFormatClass -
inputFormatClass -

Returns:
this for fluent style

Based on this, you can call mapDriver.withOutFormat(customOutputFormat.class, customInputFormat.class). With this way, you can use your RecordReader for testing.

Kenry Sanchez
  • 1,703
  • 2
  • 18
  • 24
  • https://mrunit.apache.org/documentation/javadocs/1.1.0/org/apache/hadoop/mrunit/mapreduce/MapDriver.html#withOutputFormat(java.lang.Class,%20java.lang.Class) – Kenry Sanchez Feb 13 '20 at 04:36