Stack Overflow
Stack Overflow

Questions tagged [namenode]

The Hadoop NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept.

The NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept. It does not store the data of these files itself.

Client applications talk to the NameNode whenever they wish to locate a file, or when they want to add/copy/move/delete a file. The NameNode responds to the successful requests by returning a list of relevant DataNodes where the data lives.

200 questions
0
votes
1 answer

How to balance data between Datanodes in Hadoop?

I have a multinode cluster in Hadoop consisting of two machines(one name node and two data nodes in each machine). I am using: hadoop fs -put dir1 hdfspath In the above command: will the data be distributed in both machines or only in one…
Mosab Shaheen
  • 1,114
  • 10
  • 25
0
votes
1 answer

Spark not same input/ouput directory size (for same data)

In order to reduce the number of blocks allocated by the NameNode. I'm trying to concatenate some small files to 128MB files. These small files are in gz format and the 128MB files must be in gz format too. To accomplish this, I'm getting the sum…
Philippe Paulos
  • 352
  • 1
  • 5
  • 14
0
votes
1 answer

How to distribute data between Datanodes/Slavenodes in Hadoop?

I have a multinode cluster in Hadoop consisting of two machines. first machine (configured master and slave) has name node and data node running and the second machine (configured slave) has data node running. I want to upload and distribute the…
Mosab Shaheen
  • 1,114
  • 10
  • 25
0
votes
2 answers

HDFS blocks corrupted after reboot

I have Hadoop on ubuntu 16.10 Everything works fine: I'm able to upload input file in HDFS & perform map-reduce operations. But when I reboot my PC, all HDFS blocks are corrupted and NameNode starts in SafeMode So I have to 1) Leave the SafeMode 2)…
Pizze99
  • 23
  • 6
0
votes
1 answer

starting data node on the worker node throws this error

I am using hadoop-2.5.1 and hbase-1.0.1 when i start the datanode on a worker node by hadoop-daemon.sh start datanode it throws SLF4J:class path contains multiple  SLF4J bindings and data node is not getting started
ltps
  • 37
  • 5
0
votes
0 answers

Unable to start namenode in Hadoop

I am installing Hadoop 2.7.3 on my Ubuntu 16.0.4 system. I am getting following error while trying to execute start-dfs.sh. I have checked all configuration files. node@hellbot:~$ start-dfs.sh 17/01/28 20:46:26 WARN util.NativeCodeLoader: Unable to…
Kumar-58
  • 47
  • 6
0
votes
2 answers

Namenode daemon not starting properly

I have just started learning hadoop from the book Hadoop: The definitive guide. I followed the tutorial for Hadoop installation in Pseudodistribution mode. I enabled the passwordless login to ssh. Formatted the hdfs filesystem before using it for…
Faye
  • 127
  • 1
  • 1
  • 14
0
votes
1 answer

hadoop-Not able to run namenode without a format

I know that this question has been asked before,saw the following solution: dfs.name.dir /home/hduser/hadoop/data I modified my…
ssharma
  • 339
  • 1
  • 5
  • 22
0
votes
1 answer

NameNode in hadoop not running and no access to (http://localhost:50030)

I'm using Hadoop 1.2.1 standalone mode .. when i use jps command the output is : 2528 TaskTracker 3044 Jps 2380 JobTracker 2140 DataNode 2286 SecondaryNameNode There is no NameNode running i don't know why ?.. And when i try to open the…
Dalya
  • 374
  • 1
  • 3
  • 15
0
votes
1 answer

Hadoop NameNode

I am a beginner in Hadoop. Two big issues that I am trying to fix are :- 1.)While starting hadoop through cmd (start-all.cmd) my namenode is not starting. It is showing the below error . SHUTDOWN_MSG :Shutting down NameNode at Admin/192.168.1.6…
Subhojeet Chatterjee
  • 11
  • 4
0
votes
1 answer

hadoop connection refused error on changing data/name nodes

I have a standalone hadoop setup using VMware workstation. I can start hadoop using the following commands ./start-dfs.sh ./start-yarn.sh Here is the hdfs-site.xml dfs.namenode.name.dir …
user1384205
  • 1,231
  • 3
  • 20
  • 39
0
votes
2 answers

Process to restart Namenode in IBM BigInsights (enabled GPFS - a transparency layer of HDFS)

I am working on IBM Hadoop distribution (BigInsights) which has been installed using Apache Ambari and currently, has GPFS (general parallel file system) enabled as a transparency layer of HDFS. On Ambari, we have enabled maintenance mode on HDFS…
Abhishek Sakhuja
  • 197
  • 2
  • 16
0
votes
1 answer

Namenode not starting

I'm trying to setup a hadoop cluster. But the namenode isn't starting. When i check the logs, it shows : 2016-10-19 05:23:10,140 FATAL org.apache.hadoop.hdfs.server.namenode. NameNode: Failed to start namenode. …
Aditya
  • 43
  • 2
  • 8
0
votes
1 answer

Error reading edit log in hadoop namenode

I am using 3 node hdfs cluster, it was running fine from past few months but from few days i am getting below exception in the logs frequently in one of the namenode, it was Active node but due to this error hdfs failover itself to secondary…
neerajtiwari
  • 24
  • 3
0
votes
3 answers

Could not format the Namenode in hadoop

While formatting namenode by executing the command " bin/hadoop namenode -format " i get this error: hduser@ubuntu:~/hadoop/bin$ ./hadoop namenode -format DEPRECATED: Use of this script to execute hdfs command is deprecated. Instead use the hdfs…
Y.Asmae
  • 61
  • 6
1 2 3
…
13 14