Stack Overflow
Stack Overflow

Questions tagged [namenode]

The Hadoop NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept.

The NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept. It does not store the data of these files itself.

Client applications talk to the NameNode whenever they wish to locate a file, or when they want to add/copy/move/delete a file. The NameNode responds to the successful requests by returning a list of relevant DataNodes where the data lives.

200 questions
1
vote
0 answers

Unable to import new fsimage using "hdfs namenode importCheckpoint"

I have a namenode whose fsimage is corrupted and i only have one fsimage available. I have a secondarynamenode which has the checkpoints available. Now, i wanted to importCheckpoint to a new directory. Old directory: cd…
Rakesh Bharadwaj
  • 23
  • 6
1
vote
0 answers

Hadoop HA Namenode goes down with the Error: flush failed for required journal

Namenode goes down with below error: using hadoop version : 2.0.0 ERROR INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at XXXXXX FATAL org.apache.hadoop.hdfs.server.namenode.FSEditLog: Error: flush failed for required…
AERROW
  • 41
  • 3
1
vote
1 answer

How to format datanodes after formatting the namenode on hdfs?

I've recently been settings up hadoop in pseudo distributed mode and I have created data and loaded that into HDFS. Later I have formatted namenode because of a problem. Now when I do that, I find that the directories and the files which were…
Sai Darahaas Ayyangalam
  • 13
  • 1
  • 4
1
vote
0 answers

How to install hadoop?

I'm trying to get Hadoop running 'Local Mode' on my Windows 10 machine. When I run the command: bin/hadoop namenode -format I get the following error message, and the folder hadoop which contains hdfs Isn't being created. Why do I get this…
Bata Omou
  • 7
  • 3
1
vote
1 answer

Hadoop : Starting Datanode doesn't seem to respond

My testing environment I'm trying to deploy into my testing environment an Hadoop Cluster based on 3 nodes : 1 Namenode (master : 172.30.10.64) 2 Datanodes (slave1 : 172.30.10.72 and slave2 : 172.30.10.62) I configured files with master properties…
Essex
  • 6,042
  • 11
  • 67
  • 139
1
vote
2 answers

Hadoop block checksum: stored in namenode too?

The checksum of a HDFS block is stored in a local file, along with the raw content of the block, both on each of the dedicated datanodes (replica). I am wondering: is the checksum of a block stored also within the namenode, as part of the metadata…
dmdevito
  • 51
  • 3
1
vote
1 answer

What is the best way to test hadoop?

I have completed the hadoop cluster setup with 3 journal nodes for QJM, 4 datanodes, 2 namenode, 3 zookeeper but I need to confirm whether the connectivity had made successfully between them so, I am searching for a too which can perform the…
user9184250
1
vote
1 answer

Error in formating the namenode in Hadoop single cluster node

I am trying to install and configure hadoop in the system Ubuntu 16.04, as per the guidelines of https://data-flair.training/blogs/installation-of-hadoop-3-x-on-ubuntu/ all the steps were run successfully, but while trying to run the command hdfs…
Shantha Anand
  • 11
  • 1
1
vote
0 answers

Hadoop: Secondary NameNode Permission Denied

I'm attempting to run Hadoop in pseudo-distributed mode to learn how the system work. To install it, I've downloaded Hadoop-3.0.0 from the site, untarred it. I've done my configurations as follows (leaving out the configuration tags for…
Zach
  • 341
  • 1
  • 3
  • 14
1
vote
1 answer

From where the namenode gets information of the datanode

While saving a file on the HDFS, it will split the file and store accordingly and stores the information on the edit log and it's all fine. My question is: when I request the read operation to the namenode, from where it will look the datanode…
user9184250
1
vote
0 answers

Access multiple hadoop namenodes from Spark on YARN?

I am working with a system that Spark jobs run on YARN with one default Hadoop namenode. Recently, I have added another Hadoop namenodes to my system. I now want the Spark jobs read the input data from the default namenode and write the output to…
Quy Doan
  • 13
  • 3
1
vote
2 answers

NameNode: Failed to start namenode in windows 7

I am trying to install Hadoop in windows machine, in middle I got the below error. Logs 17/11/28 16:31:48 ERROR namenode.NameNode: Failed to start namenode. java.lang.UnsatisfiedLinkError:…
MoNa
  • 365
  • 4
  • 13
  • 31
1
vote
1 answer

Unable to delete HDFS Corrupt files

I am unable to delete corrupt files present in my HDFS. Namenode has run into Safe mode. Total number of blocks are 980, out of which 978 have reported. When I run the following command, sudo -u hdfs hdfs dfsadmin -report The report generated…
user8846809
1
vote
0 answers

Hbase configuration with Hdfs HA

I am trying to setup hbase ha with hadoop ha. I have done setting hadoop ha setup and tested it. But in hbase setup, while starting, I am getting this following error java.lang.RuntimeException: Failed construction of Master: class…
Prabhu R.D
  • 55
  • 7
1
vote
0 answers

Secondary namenode stuck while loading INodes

While starting secondary namenode, it gets stuck while loading Inodes. There are lot of GCs happening on secondary namenode, but its not moving ahead of this that. tstamp_01 INFO org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Transfer took…
tic_tac_toe
  • 13
  • 1
  • 4
1 2 3
…
13 14