A DataNode stores data in the HFS (HadoopFileSystem). A functional filesystem has more than one DataNode, with data replicated across them.
Questions tagged [datanode]
86 questions
0
votes
0 answers
Error when starting HDFS in Cloudera Manager - Address already in use when trying to bind to '/var/hdfs-sockets/dn'
I am getting error after installation and I am not able to start HDFS data node.
I am always getting Error:
Exception in secureMain
java.net.BindException: bind(2) error: Address already in use when trying to bind to '/var/hdfs-sockets/dn'
at…
Veljko
- 1,708
- 12
- 40
- 80
0
votes
1 answer
Hadoop : swap DataNode & NameNode without losing any HDFS data
I have a cluster of 5 machines:
1 big NameNode
4 standard DataNodes
I want to change my current NameNode with a DataNode without losing the data stored in HDFS, so my cluster could become:
1 standard NameNode
3 standard DataNodes
1 big…
Noa Be
- 277
- 3
- 10
0
votes
1 answer
How to restore data from BP folder in HDFS
Configuration : We have 1 HDFS Namenode & 3 data nodes.
Now, namenode got restarted and for some reason cluster ID got changed (Datanodes were still running)
We tried updating Cluster ID from datanode to namenode and vice-versa but no luck.…
Sunil Agarwal
- 4,097
- 5
- 44
- 80
0
votes
1 answer
Need help to fix org.apache.hadoop.ipc.RemoteException - AWS EMR Spark Scala Application
I am running a Spark/Scala App on AWS EMR - 12 node cluster. I have multiple transformations happening where i write to HDFS and read back from hdfs to complete transformations and finally write to S3.
During one of these transformations i recently…
qubiter
- 235
- 1
- 5
- 19
0
votes
0 answers
Increase efficiency of name and data node in HDFS
i have 1 name node and 2 data node with 32 gb ram in HDFS. But it is not taking more than 30 people at a time.Is there a way to improve the efficiency of name and data node
Didhin
- 1
- 1
0
votes
0 answers
Hadoop DataNode not running in slave machine
Getting the below error in the datanode logs as shown in the screen shot.
Tried formating the namenode and stop-all.sh -- start-all.sh. No luck.
Cleared the /app/hadoop/tmp folder more than twice.
2018-07-10 19:02:12,715 ERROR…
Rakesh TS
- 47
- 4
0
votes
1 answer
When I add new datanode to the cluster, what happens to new blocks
When i add new data node to the running cluster, there are some running jobs, what will happen to the new blocks which are created by running jobs?
Tushar Bhoyar
- 3
- 2
0
votes
1 answer
Error : Could not find or load main class org.apache.hadoop.hdfs.server.datanode.DataNode
I have Hadoop 2.7.1 e it worked successfully. Next I downloaded apache-hive-2.1.1-bin and I edited the ".bashrc" file to update the environment variables for user. Now when I start Hadoop with the command '*/sbin/start-dfs.sh', there is the error:…
Danny
- 165
- 1
- 12
0
votes
0 answers
datanode not starting on windows node
I have created 2 node hadoop cluster, in which one one is windows machine (datanode) and another node is linux machine (namenode and datanode).
When I start cluster using start-dfs.sh from linux, it should start datanode on windows machine.
but it…
Vish
- 186
- 3
- 17
-1
votes
2 answers
Datanode directory is empty
I have started hdfs, but datanodes are not shown in the website.
Although when i run the "jps" command at the datanode machines:
41348 Jps
41028 DataNode
Also, Datanode directory is empty.
this is the content of the hdfs-site.xml file:
…
Radwa Ahmed
- 131
- 1
- 1
- 9