Stack Overflow
Stack Overflow

Questions tagged [namenode]

The Hadoop NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept.

The NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept. It does not store the data of these files itself.

Client applications talk to the NameNode whenever they wish to locate a file, or when they want to add/copy/move/delete a file. The NameNode responds to the successful requests by returning a list of relevant DataNodes where the data lives.

200 questions
0
votes
0 answers

HDFS not accessible from a java application

I have a clouder quickstart container and I try to get the list of file under HDFS using a java application; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileStatus; import org.apache.hadoop.fs.FileSystem; import…
C.yasser
  • 11
  • 7
0
votes
0 answers

Namenode service unstable in hadoop 1.2.1

I have setup a hadoop cluster with 1 namenode and 1 datanode (using hadoop version 1.2.1) but when I start both nodes, the namenode service dies (does not appear in list of running java processes) within seconds (datanode service remains up). Can…
sm1988
  • 9
  • 2
0
votes
1 answer

Wrong IP mapping on some data nodes in hadoop

I have a hadoop setup on 7 nodes configured using local domains using /etc/hosts. It looks like this 1.2.3.4 hadoop-master 1.2.3.5 hadoop-slave-1 1.2.3.6 hadoop-slave-2 1.2.3.7 hadoop-slave-3 1.2.3.8 hadoop-slave-4 1.2.3.9 hadoop-slave-5 1.2.3.10…
Avik Aggarwal
  • 599
  • 7
  • 28
0
votes
1 answer

Difference between NameNode heap usage and ResourceManager heap usage (trying to find NameNode heap usage cause)?

What is the difference between NameNode heap usage and ResourceManager heap usage? I am trying to find heavy NameNode heap usage cause. In the ambari dashboard, I see... when running some sqoop jobs. Not sure what is causing the NN usage to be so…
lampShadesDrifter
  • 3,925
  • 8
  • 40
  • 102
0
votes
0 answers

Cloudera Namervice proxy setup with existing nameservice with HA

Repurpose the existing nameservice for the NameNode Proxies, and create a new nameservice to represent the NameNodes of the previous nameservice.
nesinor
  • 1,514
  • 1
  • 10
  • 20
0
votes
1 answer

Unable to start hadoop problem with namenode

Once I install Hadoop and type hdfs namenode –format or hadoop namenode -format in cmd for the 1st time, Am getting below error, can anyone help me in solving this. 1st it is asking me this: Re-format filesystem in Storage Directory root=…
ShaX
  • 1
  • 2
0
votes
1 answer

Secondary Name Node in Hadoop

Suppose for checkpoint default time is 1hr. If Name Node goes down after 55min from last checkpoint. We loss the last 55 min data(edit log file data is not added in fsImage)?
Dharmendra Barfa
  • 1
0
votes
1 answer

In Hadoop 3.1.0 namenode is working but datanode is not working

In Hadoop 3.1.0 namenode is working but datanode is not working showing below message: STARTUP_MSG: build = https://github.com/apache/hadoop -r 16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d; compiled by 'centos' on 2018-03-30T00:00Z STARTUP_MSG: java…
Abinas Sahu
  • 1
0
votes
1 answer

Create directory on remote hdfs from localhost using java

Create a directory on the remote HDFS from localhost Written code but don't understand what and where to write the core-site.xml and the hdfs-site.xml. public class HadoopCall { public void demomkdir(String dir) throws IOException { …
Abhishek Dutta
  • 75
  • 4
  • 12
0
votes
0 answers

Cannot create directory in HDFS. Name node is in safe mode

I'm have deployed Hadoop in docker which is running on aws ec2 ubuntu AMI instance. when I try to create a directory in hdfs it says Cannot create directory. Name node is in safe mode below are the properties in hdfs-site.xml name:…
richie
  • 13
  • 5
0
votes
1 answer

Data-node Alive from ambari isnt stable

we have ambari cluster version - 2.6.1 and hadoop version 2.6.4 number of datanode are - 10 from ambari dashboard we can see the window that show the follwing: DataNodes Live 9/10 but after few min all datanode are alive as: DataNodes…
Judy
  • 1,595
  • 6
  • 19
  • 41
0
votes
0 answers

Call From sandbox-hdp.hortonworks.com/172.18.0.2 to localhost:8020 failed on connection exception

I use Hortonworks Sandbox HDP 2.6.5 and putty to use Linux shell. My OS is window10. I put some JSON file on HDFS and I want to open this file with pyspark. I use below python file on linux, typing "spark-submit example.py" on shell from pyspark.sql…
James Shin
  • 1
  • 1
0
votes
1 answer

Hortonworks HDFS Name Node tryLock issue on startup

We're using HDP HDFS module version 2.7.3.2.6.5.0-292. The server was stuck and had to be hard-reseted - now the Name Node service throws an error upon start up. After successfully acquiring a lock file, it instantaneously fails by trying to acquire…
Eran Betzalel
  • 4,105
  • 3
  • 38
  • 66
0
votes
2 answers

How can we exclude some data nodes only from one namespace in hdfs federation?

I was checking the hadoop federation. Here the data nodes are common, only namespace will be different. How can we exclude some datanodes only from one namespace.
Rahul
  • 459
  • 2
  • 13
0
votes
1 answer

In which scenarios we do not allocate blocks to files?

The HDFS Quota Guide says the following for Space Quota. In which scenarios we do not allocate blocks to files? A quota of zero still permits files to be created, but no blocks can be added to the files.
Aravind Yarram
  • 78,777
  • 46
  • 231
  • 327
1 2 3
…
13 14