Apache MRUnit is a Java library that helps developers unit test Apache Hadoop map reduce jobs.
Apache MRUnit is a Java library that helps developers unit test Apache Hadoop map reduce jobs.
Questions tagged [mrunit]
64 questions
1
vote
0 answers
Hadoop Mrunit throughing exceptions
I am getting the following exceptions when I am running unit tests for MapReduce. I have imported every possible jar. I am not sure where I went wrong. I have Included the screenshot of jars I imported below.
I was testing a simple code:
import…
k c
- 21
- 4
1
vote
2 answers
mrunit execute test method error HADOOP_HOME or hadoop.home.dir are not set
I want to test hadoop2 mapreduce code，use mrunit but have a error.local machine don't install hadoop in windows.
pom.xml below:
org.apache.hadoop
…
Yunhui Zhu
- 13
- 1
- 6
1
vote
1 answer
MRUnit Example for MultipleOutputs
I have written a Map only hadoop job in which i have used MultipleOutputs concept. The problem here is, i want to test this code with MRUnit. I don't see any working example for MultipleOutputs testing.
My mapper code will be like,
public void…
Jahathesh
- 11
- 5
1
vote
2 answers
How to test a Mapper which emits key as null i.e. context.write(null, );
I have a mapreduce program with only mapper and no reducer set. i want to test this. i am having below test code
@Test
public void testMapper() throws IOException {
mapDriver.withInput(new LongWritable(0l), new Text(
…
Sam
- 1,333
- 5
- 23
- 36
1
vote
1 answer
what to use in place of mrunit withInputValue deprecated
I am using MRUnit to test Map Reduce code. I can't use .withInputValue as it is deprecated.
I could not locate an equivalent that works. setInputValue does not work either. What is the work-around?
user3707934
- 13
- 2
1
vote
2 answers
Conflicting API when trying to run MRUnit example
I've been playing around with MRUnit and tried running it for a hadoop wordcount example following the tutorial for wordcount and unit testing
Though not a fan, I've been using Eclipse to run the code and I keep getting an error for setMapper…
Phreakradio
- 176
- 1
- 18
1
vote
1 answer
MRUnit on windows without hadoop cluster
I am looking to see if there is a way to run hadoop Mapreduce unit tests in Windows without having a hadoop setup. Does MRUnit run on windows (without cygwin) in Eclipse as a Java maven project?
Thanks
Srivatsan Nallazhagappan
Srivatsan Nallazhagappan
- 297
- 1
- 4
- 14
1
vote
0 answers
Test GenericWritable using MRUnit
Say for example, I have a class like below :
public class MyGenericWritable extends GenericWritable {
private static Class[] CLASSES = null;
static {
CLASSES = (Class[]) new Class[] {
…
Balaji Boggaram Ramanarayan
- 7,731
- 6
- 43
- 46
1
vote
1 answer
Unit testing Mongo-Hadoop jobs with MrUnit
How do you unit test mongo-hadoop jobs?
My attempt so far:
public class MapperTest {
MapDriver d;
@Before
public void setUp() throws IOException {
WordMapper mapper = new WordMapper();
…
marko
- 2,841
- 31
- 37
1
vote
2 answers
Found class org.apache.hadoop.mapreduce.TaskInputOutputContext, but interface was expected
I’m trying to use MRUnit 1.0.0 to test a Hadoop v2 Reducer, but I get an exception when trying:
java.lang.IncompatibleClassChangeError:
Found class org.apache.hadoop.mapreduce.TaskInputOutputContext, but interface was expected
…
Ken Williams
- 22,756
- 10
- 85
- 147
1
vote
0 answers
MRUnit and AvroMapper
AvroMapper requires 2 parameters:
public static class AvroRecordMapper
extends AvroMapper >
and mapDriver requires 4 parameters:
K1 V1 K2 V2
Can I use MRUnit for unit tests with AvroMapper?
Venus
- 136
- 2
- 9
1
vote
2 answers
setting up configuration in mrunit
I have been searching in mrunit documentation but hasnt been able to find it so far..
How do i pass configuration parameters in my mrunit.
So for example, if i take the wordcount example.
Lets say, in my driver code I am setting this…
frazman
- 32,081
- 75
- 184
- 269
0
votes
1 answer
mrunit - Using mrunit.mapreduce.MapDriver not callling customised Record Reader
I am modifying an MapReduce program in the Record Reader and wanted to write a test case for mapper to call customised InputFormat or Record Reader. I have modified test case for record reader but the record reader test cases are not an mrunit…
Aavik
- 967
- 19
- 48
0
votes
1 answer
Mrunit: how to set value comparator for MultiOutput test
is there a way to set key/value Comparator to test MultiOutput in mrunit?
Based on my analysis:
The mrunit doesn't use key/value Comparators to test the equality of expected and actual values of multioutput.The key/value comparators are only used…
Rahul Sharma
- 5,614
- 10
- 57
- 91
0
votes
3 answers
java.lang.VerifyError: Expecting a stackmap frame at branch, when using PowerMockRunner
I upgraded the MRUnit version to 1.1.0 in my project, to use the ReduceDriver for testing multiple outputs. After making changes to my test (to make it work with the upgrade), I get this error:
java.lang.VerifyError: Expecting a stackmap frame at…
Sarin
- 197
- 3
- 13