Stack Overflow
Stack Overflow

Questions tagged [namenode]

The Hadoop NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept.

The NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept. It does not store the data of these files itself.

Client applications talk to the NameNode whenever they wish to locate a file, or when they want to add/copy/move/delete a file. The NameNode responds to the successful requests by returning a list of relevant DataNodes where the data lives.

200 questions
0
votes
0 answers

How namenode will store large fsimage file in memory on start up

Suppose we have metadata generated for years and fsimage have data more than RAM memory. Now when namenode starts it loads fsimage in-memory but in such case where fsimage size itself is more than RAM then how hadoop handles it?
nayak0765
  • 175
  • 2
  • 15
0
votes
1 answer

frequently getting stale alerts for Datanodes

I am getting stale alerts every hr, if the last point of contact between Data nodes and Namenode is more than 30 s we get these alerts. I am not able to find the root cause of this slowness, I have 32 cores system, but when this alert is generated…
Kaustubh Ghode
  • 11
  • 1
0
votes
1 answer

Hadoop localhost:9870 browser interface is not working

I need to do data analysis using Hadoop. Therefore I have installed Hadoop and configured as below. But localhost:9870 is not working. Even I have format namenode every time I worked with that. Some articles and answers of this forum mentioned that…
Virgil
  • 3
  • 1
  • 4
0
votes
0 answers

NameNode and Datanode not starting in Hadoop on start-dfs.cmd

So I am trying to setup Hadoop by using this as reference: Towardsdatascience.com click here now the error: E:\RIYA\hadoop-env\hadoop-3.2.1\sbin>start-dfs.cmd 2 cmd pop up, one for datanode and one for namenode DataNode…
Riya Pal
  • 55
  • 1
  • 5
0
votes
1 answer

Accessing HDFS configured as High availability from Client program

I am trying to understand the context of the working and not working program which connects HDFS via nameservice(which connects active name node - High availability Namenode) outside HDFS cluster. Not working program: When i read both config files…
Adhish
  • 65
  • 9
0
votes
1 answer

Hadoop hdfs namenode failing to format with error java.lang.UnsupportedOperationException

Getting below error while running command 'hdfs namenode -format' for installing hadoop on windows 11. Does anyone has solution to this? Have also tried all ways to write folderpath. Re-format filesystem in Storage Directory root=…
Sudhanshu Prakash
  • 11
  • 3
0
votes
1 answer

Hadoop HDFS active NameNode is not becoming active after upgrade

I'm upgrading hadoop from version 3.0.0 to version 3.2.2. Following are the steps I followed: Get active namenode: $ hdfs haadmin -getServiceState nn1 standby $ hdfs haadmin -getServiceState nn2 active Turn safemode on and save…
nobody
  • 1
  • 2
0
votes
1 answer

Whenever I restart my ubuntu system (Vbox) and start my hadoop , my name node is not working

Whenever I restart my ubuntu system (Vbox) and start my Hadoop, my name node is not working. To resolve this I have to always the folders of namenode and datanode and format Hadoop every time I restart my system. Since 2 days am trying to resolve…
Gaurav
  • 1
0
votes
2 answers

How to remove the very large files under /hadoop/hdfs/journal/hdfsha/current/

in our HDP cluster - version 2.6.5 , with ambari platform we noticed that /hadoop/hdfs/journal/hdfsha/current/ folder include huge files and more then 1000 files as -rw-r--r--. 1 hdfs hadoop 494690334 Dec 28 11:37…
jessica
  • 2,426
  • 24
  • 66
0
votes
1 answer

Hadoop namenode Format:ExitCodeException exitCode=-1073741515:

I was trying to install Hadoop and while formatting name node i am getting this error Re-format filesystem in Storage Directory root= C:\hadoop\data\namenode; location= null ? (Y or N) Y 2021-01-03 11:30:36,360 INFO namenode.FSImage: Allocated new…
Mueez Khan
  • 107
  • 3
  • 11
0
votes
0 answers

secondary namenode failed to start ssh: connect to host 0.0.0.7 port 22: Connection timed out

Starting secondary namenodes [7] 7: ssh: connect to host 0.0.0.7 port 22: Connection timed out secondary namenode not started due to connection timeout how to fix this on Ubuntu 20.04.1 LTS? vivek@7:~$ start-all.sh WARNING: Attempting to start all…
vivek kumar
  • 33
  • 1
  • 5
0
votes
0 answers

HDFS Dead Datanode

I'm working in a HDP-3.1.0.0 environment, the HDFS version I'm using is the 3.1.1.3.1, the cluster is composed by 2 Namenodes and 4 Datanodes. After a reboot of the HDP services (stop all and start all), the cluster seems working well, but I see the…
GiuVi
  • 23
  • 4
0
votes
1 answer

how to disbale Namenode web UI?

I want to disable HDFS web UI http://localhost:50070 . I tried to disable it by below config,however it is still accessible. dfs.webhdfs.enabled false Enable or disable webhdfs. Defaults to…
Eman Hassan
  • 1
  • 1
0
votes
1 answer

How can I solve this Hadoop core-site.xml problem?

I'm using Hadoop-2.7.6 and want to build fully-distributed. I gave 3 VM servers and they can only use 32222 port. In /etc/hosts, 1xx.xxx.xxx.xx1 namenode 1xx.xxx.xxx.xx2 slave1 1xx.xxx.xxx.xx3 slave2 Then in…
withkikoz
  • 1
0
votes
1 answer

Spark in kerberized Hadoop environment and High Availability enabled: Spark SQL can only read data after write task

We were using a kerberized Hadoop environment (HDP 3.1.4 with Spark 2.3.2 and Ambari 2.7.4) for a long time, everything went well so far. Now we enabled NameNode high availability and have the following issue: When we want to read data using Spark…
D. Müller
  • 3,336
  • 4
  • 36
  • 84
1 2 3
…
13 14