4

I've started hadoop cluster composed of on master and 4 slave nodes.

Configuration seems ok:

hduser@ubuntu-amd64:/usr/local/hadoop$ ./bin/hdfs dfsadmin -report

When I enter NameNode UI (http://10.20.0.140:50070/) Overview card seems ok - for example total Capacity of all Nodes sumes up.

enter image description here

The problem is that in the card Datanodes I see only one datanode.

Datanode card

Jakub Kuszneruk
  • 1,188
  • 1
  • 12
  • 37
  • Which distribution and version of Hadoop are you using? Please check if there error stackTrache reported in log files (NameNode and DataNodes). – shanmuga Jul 29 '15 at 12:31
  • Could you please copy the result of this "hadoop dfsadmin -report" from any hadoop node. – rbyndoor Jul 29 '15 at 12:32
  • @ruby `hadoop dfsadmin` id deprecated, you have link in my post to `hdfs dfsadmin -report` result. – Jakub Kuszneruk Jul 29 '15 at 12:33
  • That looks absolutely alright except glitch in UI. Thanks – rbyndoor Jul 29 '15 at 12:42
  • have the same issue on hadoop 2.7.3 – mitnk Oct 05 '16 at 13:24
  • Hi, I am too having the same issue in k8s. hdfs dfsadmin -admin shows one datanode only randomly e.g. sometime datanode-0 and sometime datanode-1. Datanodes hostname is different datanode-0,datanode-1 but their name is same (127.0.0.1:9866(localhost)). Can this be the issue? If yes, how to solve it – NumeroUno Jul 06 '20 at 14:44

8 Answers8

5

I came across the same problem, fortunately, I solved it. I guess it causes by the 'localhost'.

  1. Config different name for these IP in /etc/host
  2. Remember to restart all the machines, things will go well.
shivshankar
  • 691
  • 2
  • 14
  • 31
wayde
  • 66
  • 1
  • 2
2

It's because of the same hostname in both datanodes. In your case both datanodes are registering to the namenode with same hostname ie 'localhost' Try with different hostnames it will fix your problem.

in UI it will show only one entry for a hostname.

in "hdfs dfsadmin -report" output you can see both.

Rahul
  • 459
  • 2
  • 13
  • Hi, I am too having the same issue in k8s. hdfs dfsadmin -admin shows one datanode only randomly e.g. sometime datanode-0 and sometime datanode-1. Datanodes hostname is different datanode-0,datanode-1 but their name is same (127.0.0.1:9866(localhost)). Can this be the issue? If yes, how to solve it – NumeroUno Jul 06 '20 at 14:46
1

The following tips may help you

  1. Check the core-site.xml and ensure that the namenode hostname is correct
  2. Check the firewall rules in namenode and datanodes and ensure that the required ports are open
  3. Check the logs of datanodes
  4. Ensure that all the datanodes are up and running
Amal G Jose
  • 2,486
  • 1
  • 20
  • 35
1

As @Rahul said the problem is because of the same hostname

change your hostname in /etc/hostname file and give different hostname for each host and resolve hostname with ip address /etc/hosts file

then restart your cluster you will see all datanodes in Datanode information tab on browser

swapnil kumbhar
  • 21
  • 2
1

I have the same trouble because I use ip instead of hostname, [hdfs dfsadmin -report] is correct though it is only one[localhost] in UI. Finally, I solved it like this:

<property>
       <name>dfs.datanode.hostname</name>                   
       <value>the name you want to show</value>
</property>

you almost can't find it in any doucument...

Anand
  • 1,820
  • 2
  • 18
  • 25
lpr
  • 11
  • 1
  • Hi, I am too having the same issue in k8s. hdfs dfsadmin -admin shows one datanode only randomly e.g. sometime datanode-0 and sometime datanode-1. Datanodes hostname is different datanode-0,datanode-1 but their name is same (127.0.0.1:9866(localhost)). Can this be the issue? If yes, how to solve it – NumeroUno Jul 06 '20 at 14:46
1

Sorry, feels like it's been a time. But still I'd like to share my answer: the root cause is from hadoop/etc/hadoop/hdfs-site.xml: the xml file has a property named dfs.datanode.data.dir. If you set all the datanodes with the same name, then hadoop is assuming the cluster has only one datanode. So the proper way of doing it is naming every datanode with a unique name: enter image description here

Regards, YUN HANXUAN

Michael Yun
  • 119
  • 1
  • 2
0

Your admin report looks absolutely fine. Please run the below to check the HDFS disk space details.

"hdfs dfs -df /"

If you still see the size being good, its just a UI glitch.

Akarsh
  • 389
  • 5
  • 15
0

My Problems: I have 1 master node and 3 slave nodes. when I start all nodes by start-all. sh and accessing the dashboard of master nodes. I was able to see only one data node on the web UI.

My Solution:

  1. Try to stop the Firewall temporary by sudo systemctl stop firewalld. if you do not want to stop your firewalld service then r allow the ports of the data node by sudo firewall-cmd --permanent --add-port{PORT_Number/tcp,PORT_number2/tcp} ; sudo firewall-cmd --reload
  2. If you are using sapretae user for Hadoop in my case I am using hadoop user to manage hadoop daemons then change the owner on your dataNode and nameNode file by. sudo chown hadoop:hadoop /opt/data -R

My hdfs-site.xml config as given in image My hdfs-site.xml config as given in image

  1. Check your daemons on data node by jps command. it should show as given in the below image.

jps Output
jps Output

Snigdhajyoti
  • 1,327
  • 10
  • 26
Arvind kumar
  • 1
  • 1