Stack Overflow
Stack Overflow

Questions tagged [mrunit]

Apache MRUnit is a Java library that helps developers unit test Apache Hadoop map reduce jobs.

Apache MRUnit is a Java library that helps developers unit test Apache Hadoop map reduce jobs.

64 questions
2
votes
0 answers

ORC Mapper unit testing using MRUnit

I have a map program which process ORC file. From the driver I set the orcformat as input format. job.setInputFormatClass(OrcNewInputFormat.class); In the OrcNewInputFormat the value is OrcStruct. In Map method Writable value passed as parameter…
skhurana
  • 91
  • 5
2
votes
0 answers

MRUnit - Test success depends of output order

I have created a small example to figure out how to use MRUnit for testing MapReduce code. The example that I am trying to run is wordcount, so I won't paste the code here since it's familiar to all. I have written tests using JUnit. What confuses…
Kobe-Wan Kenobi
  • 3,694
  • 2
  • 40
  • 67
2
votes
2 answers

Maven dependencies not downloading for MRunit

I am new to Maven. I have created a Java Project for MRunit testing and converted it into Maven Project. I have added below dependencies into pom.xml file. However I don't see the Maven Dependencies folder or any downloads from the internet into the…
user4971722
2
votes
2 answers

MRUnit Create HBase Result Properly

I have a mapreduce job where the mapper reads from several HBase tables. It works fine on my cluster. I am writing some unit tests retroactively with MRUnit. I am attempting to compose a Result object from a list of manually instantiated KeyValue…
Eric Minor
  • 29
  • 2
2
votes
0 answers

Error MRUnit and MultipleOutput, missing outputs

I'm trying to test some MapReduces with MRUnit 1.1.0, but I got this error: 14/10/13 14:19:49 ERROR mrunit.TestDriver: Missing expected outputs for namedOutput 'a' 14/10/13 14:19:49 ERROR mrunit.TestDriver: Missing expected output (b, a) for…
Guille
  • 2,248
  • 3
  • 24
  • 42
2
votes
0 answers

How to write MR unit test to read avro based schema records and emit text base key values

Even I have a similar requirement. I want to read avro records having some schema and emit text datatype key values, I need to write the MR Unit test cases for this. I have wrote the following code but it is giving me the following exception: …
user848066
  • 63
  • 7
2
votes
1 answer

MRUnit reducer test: Mismatch in value class

My reducer unit test throws a "Mismatch in value class" exception after the migration to MapReduce 2: Mismatch in value class: expected: class org.apache.hadoop.io.IntWritable actual: class com.company.MyWritable The error message itself is clear to…
Sascha Vetter
  • 2,466
  • 1
  • 19
  • 36
2
votes
1 answer

Hadoop unit test with mocked out Context

I am trying to write an unit test for a hadoop job. The catch is that the mapper uses the Context argument passed on to it in order to determine which file is being read by it at that moment. It makes the following call: String inputFile =…
dipteshc
  • 31
  • 3
2
votes
1 answer

MapReduce unit test fails to mock DistributedCache.getLocalCacheFiles

With Apache MRUnit I'm able to unit test my MapReduce program locally before running it on the cluster. My program needs to read from DistributedCache so I wrap DistributedCache.getLocalCacheFiles in a class which I mock in my unit test. I setup a…
manuzhang
  • 2,995
  • 4
  • 36
  • 67
2
votes
2 answers

java.lang.NullPointerException using MRUnit. Custom key serialization error

I'm trying to test a simple MapReduce project using MRUnit. I set the input for the mapDriver and then call mapDriver.runTest() (I've also tried with mapDriver.run() but produces the same error). I have written a custom key which overloads the…
Sam
  • 893
  • 8
  • 21
2
votes
1 answer

Mocking the context object in MR unit

I'm new to Hadoop and this is my first mapper program, which I'm unit testing through MR unit. I'm passing the parameter(year) which I set through the config object Configuration config =new Configuration() config.set("Year",…
user1876832
  • 31
  • 4
1
vote
2 answers

MapReduce Mrunit error

I'm a new in Hadoop. Yesterday I followed the book and used JUnit as the apper unit test for meteorological data. But there were some problems. This is my pom file: junit …
PC12138
  • 21
  • 1
  • 1
  • 4
1
vote
1 answer

java.lang.ClassCastException: org.apache.hadoop.io.serializer.WritableSerialization cannot be cast to org.apache.hadoop.io.serializer.Serialization

I am testing MultipleOutputs using MRUnit. The test cases are getting failed with the following message. java.lang.ClassCastException: org.apache.hadoop.io.serializer.WritableSerialization cannot be cast to…
MJBibin
  • 41
  • 7
1
vote
1 answer

Tweaking an output before testing using MRUnit

I am using creating an application using mapreduce2 and testing the same using MRUnit 1.1.0. In one of the tests, I am checking the output of a reducer which puts the 'current system time' in it's output. I.e. at the time the reducer executes, the…
Ankit Khettry
  • 997
  • 1
  • 13
  • 33
1
vote
1 answer

MR-Unit vs JUnit for unit testing

Can anyone please explain what is the gain of using MR-Unit for unit testing MR jobs compairing to usage of JUnit and Mockito? Concretely, what are the things that I can do that I cannot do them using JUnit, or it is much more difficult? My idea is…
Kobe-Wan Kenobi
  • 3,694
  • 2
  • 40
  • 67
1
2
3 4 5