How to unit test Java Hbase API

Question

I am using the Java HBase API to get a value from Hbase. This is my code.

public class GetViewFromHbaseBolt extends BaseBasicBolt {

private HTable table;
private String zkQuorum;
private String zkClientPort;
private String tableName;

public GetViewFromHbaseBolt(String table, String zkQuorum,
        String zkClientPort) {
    this.tableName = table;
    this.zkQuorum = zkQuorum;
    this.zkClientPort = zkClientPort;
}

@Override
public void prepare(Map config, TopologyContext context) {
    try {
        table = getHTable();
    } catch (IOException e) {

        e.printStackTrace();
    }
}

@Override
public void execute(Tuple tuple, BasicOutputCollector collector) {
    try {
        if (tuple.size() > 0) {

            Long dtmid = tuple.getLong(0);

            byte[] rowKey = HBaseRowKeyDistributor.getDistributedKey(dtmid);
            Get get = new Get(rowKey);
            get.addFamily("a".getBytes());

            Result result = table.get(get);
            System.out.println(result);
            byte[] bidUser = result.getValue("a".getBytes(),
                    "co_created_5076".getBytes());
            collector.emit(new Values(dtmid, bidUser));
        }

    } catch (IOException e) {
        e.printStackTrace();

    }

}

@Override
public void declareOutputFields(OutputFieldsDeclarer declarer) {

    declarer.declare(new Fields("dtmi", "bidUser"));

}

// setting up the conections for Hbase
protected HTable getHTable() throws IOException {
    HTable table = null;
    Configuration conf;
    conf = HBaseConfiguration.create();
    conf.set("hbase.zookeeper.property.clientPort", zkClientPort);
    conf.set("hbase.zookeeper.quorum", zkQuorum);

    table = new HTable(conf, tableName);

    return table;
}

This works fine. Now I am writing a Unit Test using the Mockito API. On my test class I am getting a java.langNullPointerException when this when(table.get(any(Get.class))).thenReturn(result); is called.

   public class GetViewFromHbaseBoltTest {

@Mock
private TopologyContext topologyContext;
@Mock
private  HTable table;
//HTable table = mock(HTable.class);


@Test
public void testExecute() throws IOException {

    long dtmId = 350000000770902930L;
    final byte[] COL_FAMILY = "a".getBytes();
    final byte[] COL_QUALIFIER = "co_created_5076".getBytes();

    // A mock tuple with a single dtmid
    Tuple tuple = mock(Tuple.class);
    when(tuple.size()).thenReturn(1);
    when(tuple.getLong(0)).thenReturn(dtmId);

    List<KeyValue> kvs = new ArrayList<KeyValue>();
    kvs.add(new KeyValue(COL_FAMILY, COL_QUALIFIER, Bytes
            .toBytes("ExpedtedBytes")));
    Result result = new Result(kvs);


    when(table.get(any(Get.class))).thenReturn(result);

    BasicOutputCollector collector = mock(BasicOutputCollector.class);

    GetViewFromHbaseBolt bolt = mock(GetViewFromHbaseBolt.class);
    // Execute the bolt.
    bolt.execute(tuple, collector);

    ArgumentCaptor<Values> valuesArg = ArgumentCaptor
            .forClass(Values.class);
    verify(collector).emit(valuesArg.capture());
    ArrayList<Object> d = valuesArg.getValue();

// verify

}

you are calling 'execute' method on a mock object. Are you supposed to call it on real object? — K139, May 01 '15 at 19:37
@K139 . Well, Will it make a difference if call it from a real object? — danilo, May 01 '15 at 20:18
The difference with calling a mock object vs the real object is that the mock object is just a stub with no functionality. You want to test the real GetViewFromHbaseBolt object, not a stub that does nothing. This is why the test cannot run, because it does not test anything. Fix that and redo the question.. — kg_sYy, May 08 '15 at 15:43

K139 · Accepted Answer · 2015-05-01T20:33:44.660

Edit

'Unit Test' is to verify, and validate a particular 'unit of code'. If that 'unit of code' depends on external calls, then we mock those calls to return certain values.

If you don't invoke the 'actual unit of code' from your test case, then what is the point in unit testing the code. got it?

You should mock only external calls from actual method, not the actual method itself.

So I would mock like:

@Mock
private HBaseRowKeyDistributor hDist;

@Mock
private HTable table;

@Test
public void sampleTest(){

    //Mock data and external calls

    final byte [] COL_FAMILY = "a".getBytes();
    final byte [] COL_QUALIFIER = "co_created_5076".getBytes();

    List<KeyValue> kvs =new ArrayList<KeyValue>();
    kvs.add(new KeyValue(COL_FAMILY, COL_QUALIFIER,
                           Bytes.toBytes("ExpedtedBytes")));
     Result result = new Result(kvs);

     byte[] rowKey = "Hello".getBytes();
     when(hdist.getDistributedKey(anyString()).thenReturn(rowKey);

     when(table.get(any(Get.class))).thenReturn(result);

     //Call the actual method
     calloriginalUnitTestingMethod();


     //Now Verify the result using Mockito verify like
     verify(...)

     //You could also intercept the attributes

}

fabwu · Answer 2 · 2018-09-13T11:22:07.327

Maybe you noticed that your code is bad testable. The reason for the bad testability is that your class do two things (create a table and execute a tuple). I would recommand you to split your class into two classes and inject the class which create the table into the class which execute a tuple.

This class create a Table:

public class HTableFactory {

    private String tablename;    

    public HTableFactory(String tablename) {
        this.tablename = tablename;
    }

    public HTable getTable() {}
}

Then your GetViewFromHbaseBolt class look something like the following:

public class GetViewFromHbaseBolt extends BaseBasicBolt {

    private HTableFactory factory;
    private HTable table;

    public GetViewFromHbaseBolt(HTableFactory factory) {}

    @Override
    public void prepare(Map config, TopologyContext context) {
        table = factory.getTable();
    }

    @Override
    public void execute(Tuple tuple, BasicOutputCollector collector) {}

    @Override
    public void declareOutputFields(OutputFieldsDeclarer declarer) {}
}

Now your GetViewFromHbaseBolt has the following dependencies:

HTableFactory
Tuple
BasicOutputCollector
OutputFieldsDeclarer

In your testclass you can mock out this dependencies and inject them into GetViewFromHbaseBold.

score 0 · Answer 3 · edited May 23 '17 at 12:32

0

Have you checked that you initialize your mocks right? I am not an expert on Mockito but it seems to me you need to either set the test runner to Mockito runner or explicitly initialize the annotated mocks (@Mock). For example, see Using @Mock and @InjectMocks

My guess is your "table" variable is null when you run the line that produces the NullPointer.

edited May 23 '17 at 12:32

Community

1
1

answered May 08 '15 at 15:51

kg_sYy

1,127
9
26

How to unit test Java Hbase API

3 Answers3

Edit