Stack Overflow
Stack Overflow

Questions tagged [namenode]

The Hadoop NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept.

The NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept. It does not store the data of these files itself.

Client applications talk to the NameNode whenever they wish to locate a file, or when they want to add/copy/move/delete a file. The NameNode responds to the successful requests by returning a list of relevant DataNodes where the data lives.

200 questions
0
votes
1 answer

What is the difference between NameNode and NameSpace

As a beginner to Hadoop, I am confused between NameNode and Namespace. From my understanding both of them have the same functionality, which they both contains metadata. Is there any difference between them?
Fallon19
  • 55
  • 1
  • 8
0
votes
1 answer

How to enable GC logging for Apache Hadoop NameNodes, while preventing log file overwrites and capping disk space usage

We recently decided to enable GC logging for Hadoop NameNodes on a number of clusters (exact version varies) as a aid to looking into NameNode-related memory and garbage collection problems. We want this for all NameNodes in the cluster (active and…
Jim Hoagland
  • 481
  • 1
  • 4
  • 20
0
votes
1 answer

Hadoop Name Node HA Hive failures

I have two namenodes in HA environment. And Hive configured to point to HA namespace. But intermediatly my Hive fails pointing to passive namenode giving below error even tough my active namenode is still in service. Kindly help me dig where the…
zill silveira
  • 1
  • 2
0
votes
1 answer

How can i use one node as a datanode in multiple hadoop clusters

I want to use one computer as a data node in two different hadoop clusters. I tried changing the ports, but that did not work. Please tell me if there are any port changes needed.
shivakrishna
  • 41
  • 3
0
votes
0 answers

Hadoop name node fails to load in VirtualBox after sometime

I've configured and am running Hadoop v2.7.2 along with OpenJDK v7 u191. Once the configuration is done, all the process namely name node, data node, secondary name node, etc. runs perfectly (Checked with jps). I'm running Ubuntu 14.0.4 LTS in…
Mr. Hobo
  • 530
  • 1
  • 7
  • 22
0
votes
0 answers

Increase in fsimage size by 1g over a period of 1 day

fsimage size in hadoop cluster increased by 1G when there was no adhoc jobs run in one day . Need help how to check what caused the increase in size
pshenoy
  • 1
0
votes
2 answers

Namenode Block Management

Suppose I have a file of 50mb and my default HDFS block size is 64mb. So after storing this 50mb file, we are left with 14mb block size which can be used to store some other file. Now namenode keeps track of the block informations of the files…
Rahul Mukherjee
  • 3
  • 3
0
votes
0 answers

Enabling Kerberos in Hortonworks made DN dead even after regenerating Keytabs

I have set up hadoop cluster with HA and all the services where good and running, now I tried to enable kerberos where everything went well but restarting services didn't finish and I aborted. I see kerberos is in enable state. Now when I start…
sudheer
  • 338
  • 1
  • 6
  • 17
0
votes
1 answer

No namenode to start while setting single node hadoop

I am trying to set up a single node cluster in hadoop, but after giving the command start-all.sh , jps doesn't show namenode and when I stop all processes, I get the result as no namenode to stop. Also, I saw at few places that clearing the tmp…
Neel Shah
  • 319
  • 7
  • 16
0
votes
1 answer

Hadoop's NameNode and DataNode Service did not run in single_mode

I installed Hadoop 2.7.2 on Ubuntu 16.04 in single mode. But neither NameNode nor DataNode Services run after starting the Hadoop. hduser@saber-Studio-1435:/usr/local/hadoop$ start-all.sh This script is Deprecated. Instead use start-dfs.sh and…
saber
  • 41
  • 3
0
votes
2 answers

Use of secondary namenode in Hadoop in 2.x

As far as i know, Hadoop 1.x had secondary namenode but was used to create an image of the primary namenode and it updates the primary namenode when it fails and again starts up. But what is the use of secondary namenode in Hadoop 2.x given that we…
shubham kumar
  • 25
  • 5
0
votes
3 answers

NameNode and secondary Namenode

What is the role of a Secondary namenode, if at all Namenode fails ?
Mehdi Medadian
  • 7
  • 1
  • 3
-1
votes
1 answer

Namenode can't running

I built a single hadoop,when I tried ./start-dfs.sh enter image description here namenode didn't work logs↓ enter image description here I have tried to delete tmp and initialize nanenode format hdfs namenode -format, but it also didn't work 50070…
eacin
  • 1
-1
votes
1 answer

Re-format filesystem error starting Hadoop services on Mac

iMac 2020 Intel, MacOS Monterey 12.6, Java 1.8, Hadoop 3.3.4 as at 9-Feb-23 I am getting this error when starting Hadoop with this command: $HADOOP_HOME/sbin/start-all.sh Irrespective of response as Y or N, the error keeps running on the terminal…
Merlion Curry
  • 1
  • 1
-1
votes
1 answer

hadoop installation, to start secondary namenode, nodemanagers, and resource managers

I have installed hadoop 3.1.0 clusters on 4 linux machines, hadoop1(master),hadoop2,hadoop3,and hadoop4. I ran start-dfs.sh and start-yarn.sh, and saw only namenodes and datanodes running with jps. secondary namenodes, nodemanagers and…
davidzxc574
  • 471
  • 1
  • 8
  • 21
1 2 3
…
13
14