Questions tagged [combiners]
105 questions
0
votes
1 answer
Execuation time of MapReduce with Combiner
I have a MapReduce job that reads a file then collect all words that are 5 characters or less and start with an upper case letter using the first letter as a key. I ran the job twice once without using a combiner and the second time with a combiner.…
nj2012
- 105
- 2
- 14
0
votes
1 answer
Calculation weigths in combine classifier outputs
I have 3-class classification problem and I have ensemble with 4 classifiers. Each classifier returns supports for each class. Now I want to combine this outputs using weighted average with c*L weights (c=3, L=4), but I don't know how can I…
veeveeoor
- 205
- 1
- 2
- 11
0
votes
1 answer
Hadoop combiners with aggregator functions
When using one of the aggregator function in place of a reducer, will Hadoop be smart enough to use same function for combiner phase?
spacemonkey
- 19,664
- 14
- 42
- 62
0
votes
1 answer
Why does part of my data go into the reducer directly without going into the combiner
I am using version 0.20.0 of hadoop.
I have set the combiner class successfully. And my program runs successfully.
However, I found about 5% of my data didn't go into the combiner after coming out from the mapper, this data go into the reducer…
JoJo
- 1,377
- 3
- 14
- 28
0
votes
2 answers
Which class should the Combiner be inherited from (or extend) in Hadoop?
In Hadoop, the mapper and reducer classes should extend the Mapper or Reducer interface. However, I could not find the interface which the combiner class should extend. What is the signature for the combiner class in Hadoop?
HHH
- 6,085
- 20
- 92
- 164
0
votes
1 answer
Mapper output doubled in combiner
The mapper outputs the value only once. But when i checked the combiner is having the value twice. This is really strange to me.
Pls help.
Module of map code from where output is sent:
for(int i = 0; i
Mahalakshmi Lakshminarayanan
- 714
- 2
- 11
- 34
0
votes
2 answers
In-mapper combining and when does cleanup routine of mapper executes?
I'm trying simple bi-gram (word pair) count, I tried with simple "pair" approach, now I just modified to try "Stripes" approach, but in Cleanup routine of mapper, somehow my all keys are same word pair (as in last word pair!) and counts.
e.g. text…
code muncher
- 1,592
- 2
- 27
- 46
0
votes
1 answer
Hadoop configuration - are mapper/combiner affected by io.sort.factor and io.sort.mb?
If I modify the io.sort.factor and io.sort.mb, does the local sort taking place on the map side use these variables, or are they only used by the sort done on the reducer side?
Dolan Antenucci
- 15,432
- 17
- 74
- 100
0
votes
2 answers
Hadoop Combiner Class for Text
I'm still trying to get an intuition as to when to use the Hadoop combiner class (I saw a few articles but they did not specifically help in my situation).
My question is, is it appropriate to use a combiner class when the value of the pair is of…
honeywind
- 227
- 3
- 13
-1
votes
2 answers
python combine two file which data is inconsistency
now I have two files: A and B. Both are csv files.
csv A has some data:
a b c
1 2 3
4 5 6
7 8 9
.....
csv B has some data:
b d
7 0
8 3
2 1
.....
now I want to combine the two csv files A and B for a new file C like that:
a b c d
1 2 3 1
7 8…
kevintao
- 36
- 1
- 3
-1
votes
3 answers
Compare values and combine
How can I combine the text values of the same pr value?
Array (
[0] => Array ( [ID] => 1 [text] => text1 [pr] => project1)
[1] => Array ( [ID] => 2 [text] => text2 [pr] => project1)
[2] => Array ( [ID] => 2 [text] => text3 [pr] => project2)
[3] =>…
M1NT
- 386
- 1
- 4
- 13
-2
votes
4 answers
Combine cells from one column into one string
df = pd.read_csv(filename.csv)
corpus = df.corpus
How can I combine series of text strings (from one column) into a list?
from column 'corpus':
row 1: Hail Mary.
row 2: Hi Bob.
row 3: Hey Sue.
into
[Hail Mary. Hi Bob. Hey Sue.]
Looking for a…
spacedustpi
- 351
- 5
- 18
-2
votes
3 answers
Javascript Array combine elements to get a unique array
Suppose I have a JS array like this:
[
{
"lat": 49.26125,
"lon": -123.24807,
"weight": 120
},
{
"lat": 49.26125,
"lon": -123.24807,
"weight": 80
},
{
"lat": 49.26125,
"lon": -123.24807,
"weight": 160
…
DownToEarthCoder
- 3
- 2
-2
votes
1 answer
How to Combine Cells with Numbers Into A Custom Number Format?
So I've been searching for a couple of hours now and am currently stuck. I've been given 3 columns with number values in them. In this particular scenario, they are measurements (feet, inches, quarter inches). I have been asked to combine the…
Karl G
- 1
- 1
-4
votes
1 answer
R: abc<-c(1:15) vs abcd<-1:15
What is the difference between the following commands:
abc <- 1:15
abcd <- c(1:15)
abc
abcd
The output is:
> abc <- 1:15
> abcd <- c(1:15)
> abc
[1] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
> abcd
[1] 1 2 3 4 5 6 7 8 9 10 11 12 13 14…
Utkarsh Mittal
- 141
- 8