How to send some data to Mapper class (running on data in a HBase database)

Question

I need to send some information for mapper jobs running on nodes in HBase. I already have defined the data as static member in the class but it seems when the mapper is being run on the other nodes, the data is not being transferred to the nodes. Is there any way to do that?

public class GetResultFromVerticesIDMapper extends TableMapper< ResultVal, IntWritable > {

    public static HashMap<Long, Boolean> vertexIDsHashMap;
    public static int                    nResultComponents;

...
}

score 1 · Accepted Answer · answered Sep 16 '16 at 08:39

I did this through sending parameter to the configuration:

Configuration conf;
conf.addResource(new Path(hbaseConfigurationFile));
conf.set("simplificationParameters", simpParameters.toString());

and getting them into mapper class as follows:

_simplificationParameters.fromString(context.getConfiguration().get("simplificationParameters"));

score 1 · Answer 2 · answered Sep 19 '16 at 19:05

Above way you mentioned is one way.. Other way, if you send System properties then it will be set to context automatically.

while running the program you can pass parameters as system properties (with -D option) or accept input and set them using System.setProperty("simplificationParameters","user_entered_input")

you can take it back in mapper or reducer like the same way you did it in your answer... i.e. context.getConfiguration().get("simplificationParameters")

How to send some data to Mapper class (running on data in a HBase database)

2 Answers2