Hadoop partitioning deals with questions about how hadoop decides which key/value pairs are to be sent to which reducer (partition).
Questions tagged [hadoop-partitioning]
339 questions
-1
votes
1 answer
can we use log4j in mapreduce?
Can we use log4j to log in mapreduce? If so, provide the steps to use log4j in map-reduce to log the information. I have written the below log4.properties but, nothing was logged.
Mr Shetty
- 19
- 1
- 4
-1
votes
1 answer
Hadoop mapreduce using 2 mapper and 1 reducer using c++
Following the instructions on this link, I implemented a wordcount program in c++ using single mapper and single reducer. Now I need to use two mappers and one reducer for the same problem.
Can someone help me please in this regard?
user3532122
- 15
- 4
-1
votes
1 answer
I have to implement hadoop, so it can process the data of call detail records?
I have configured HDFS, Datanode and namenode and also hbase.
I have stored a CDR csv file in HDFS. So how can I map it with Hbase and make ready to process it?
-1
votes
2 answers
Hadoop Map Task : Read the content of a specified input file
I'm pretty new to Hadoop environment. Recently, I run a basic mapreduce program. It was easy to run.
Now, I've a input file with following contents inside input path directory
fileName1
fileName2
fileName3
...
I need to read the lines of this file…
-1
votes
4 answers
New user SSH hadoop
Installation of hadoop on single node cluster , any idea why do we need to create the following
Why do we need SSH access for a new user ..?
Why should it be able to connect to its own user account?
Why should i specify a password less for a new…
Surya
- 3,408
- 5
- 27
- 35
-2
votes
1 answer
Spark dataset withColumn add partition id
I am trying to write a helper function that takes a dataset of any typeDataset[_], and returns with one new column "partitionId" which is the id of the partition that single data unit belongs to.
For example, if I have a dataset below and by default…
HayreddinLuo
- 91
- 1
- 6
-2
votes
1 answer
For some of the hive queries I wasn't able to see the o/p?
My query is
SELECT txnno, product FROM txnrecsbycat TABLESAMPLE(BUCKET 2 OUT OF 10) ORDER BY txnno;
I am getting success but unable to view my O/p
My o/p is:
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile…
-2
votes
1 answer
I resarch about HDFS failures. For this I need to HDFS logs . Where can I download the logs?
I resarch about HDFS failures. For this I need to HDFS logs . Where can I download the logs ?
Mehdi Medadian
- 7
- 1
- 3
-3
votes
1 answer
How exactly to process data on Hadoop,Hive,Pig
I have learnt the basics of Apache Hadoop Hive. And know majority of commands. Now, how to exactly work on the data. I have huge amt of data available with me(got it from a person). But dont know what exactly to do.
The data(.xlsx) is weekly sales,…
Sanjeev
- 17
- 1
- 6