Usually record reader passes line by line to the mapper or n lines. Can full block be accessed in the mapper? This means the record reader will give full block to the mapper instead of line by line? Does this approach makes sense?
Thanks
Asked
Active
Viewed 30 times
0
-
1Do you mean the entire block at once? because a mapper will read one block in it's entirety anyway. – philantrovert Jan 22 '18 at 10:18
-
Yes the entire block. Usually the entire block is read line by line or by some other record criteria. – shujaat Jan 22 '18 at 10:32
-
1I think you need to implement custom `InputFormat` to your block size will be equal to split size. – luminousmen Jan 22 '18 at 13:02
1 Answers
1
Need to note that split size is not equal to block size by default. To do what you want you need to implement custom InputFormat and to be more exact method getSplits(JobContext context). And make InputSplit size to be equal to block size.
luminousmen
- 1,971
- 1
- 18
- 24