Is there any methods in R-Hadoop mapreduce, similar to setup() and cleanup() in Java mapreduce? I have to run a part of code such as a db call only once, before start of all reducers. Is there any provisions to do that when writing mapreduce code using R?
Asked
Active
Viewed 67 times
1 Answers
0
If you are talking about rmr2, there is not. If you detailed your use case, I could try to think of an alternative or learn something even if I failed. I assume that this DB access needs to happen between map and reduce but it's important to understand if the DB operation depends on the output of the map phase and if the reduce phase depends on the result of the DB operation. It's a pretty unique requirement from my experience and it seems to me you have to solve it with two jobs, one map only and one reduce only.

piccolbo
- 1,305
- 7
- 17