0

I have a map reduce job which will do a bulk load in MapR table one at time. If i have to load another Mapr DB table then i will have to write another job for doing bulk load.Is there any way to do bulk load in single map reduce job?

Thanks in advance.

Shashi
  • 2,686
  • 7
  • 35
  • 67
  • You need to bulk load into 2 tables using one map reduce? And what is the source of these map reduce jobs – Ramzy Jun 25 '15 at 21:10
  • We have a job which will read data from Hive and it will push that data into hbase. Data should get divided based certain logic in different hbase table.i.e certain list of hfile should be loaded into htable1 and another set should be loaded in htable2. – Shashi Jun 26 '15 at 05:56
  • The bulk loading starts with HFiles right. So you mean to say that you are creating separate HFiles based on your condition? – Ramzy Jun 26 '15 at 15:40
  • That i what i wanted to do. Lets say i am reading a simple text file in mapper and dividing data on certain condition. Based on those condition,first part should go into table1 and another part should go into table2. I dont want to run same piece of code for different table.In single job i should be able to load both table. – Shashi Jun 29 '15 at 06:04
  • The nearest I can see is, use multiple output feature in reducer and write HFiles to those outputs. Once output is ready, you can use `LoadIncrementalHFiles loadFfiles = new LoadIncrementalHFiles(configuration); HTable hTable = new HTable(configuration, "yourtable"); loadFfiles.doBulkLoad(new Path("outputPath1"), hTable);` – Ramzy Jun 29 '15 at 15:27
  • Thanks Ramzy. Just one confusion i have here, will this approach work for MapR DB table? – Shashi Jul 01 '15 at 09:05

0 Answers0