Hadoop2.2.0 can't visit the web http://:8088

Question

I setup a hadoop cluster with two nodes hadoop01(master- 10.0.0.151) and hadoop02(slaves- 10.0.0.152) when a type start-dfs.sh and then visit website my_ip(just 10.0.0.151 above):50070. It's successful.
But when i type start-yarn.sh then visit website my_ip:8088. It's failed.

my yarn-site.xml:

<configuration>
<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
    <name>yarn.resourcemanager.address</name>
    <value>hadoop01:8032</value>
</property>        
<property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>hadoop01:8030</value>
</property>

<property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>hadoop01:8031</value>
</property>

<property>
    <name>yarn.resourcemanager.admin.address</name>
    <value>hadoop01:8033</value>
</property>
<property>
    <name>yarn.resourcemanager.webapp.address</name>
    <value>hadoop01:8088</value>
</property>

core-site.xml:

<configuration>
<property>
     <name>fs.defaultFS</name>
     <value>hdfs://hadoop01:8020</value>
 </property>
 </configuration>

the same set as hadoop02(slave)

my hadoop01 hadoop-2.2.0/etc/hadoop/slaves set

hadoop01
hadoop02

After typing start-dfs.sh & start-yarn.sh then i type jps

hadoop01:
21594 NameNode
22345 NodeManager
22007 SecondaryNameNode
22171 ResourceManager
23147 Jps
21762 DataNode
hadoop02:
29861 NodeManager
30358 Jps
29665 DataNode

my /etc/hosts in hadoop01:

localhost       hadoop01
10.0.0.151      hadoop01
10.0.0.152      hadoop02

my /etc/hosts in hadoop02:

localhost       hadoop02
10.0.0.151      hadoop01
10.0.0.152      hadoop02

This bellow link is my yarn-nodemanager.log I upload to google dirve https://drive.google.com/file/d/0B7nCJ_XJWSrQN1BZVTVyOEgxd1E/edit?usp=sharing but yarn-nodemanager.log doesn't appear ERROR if i didn't miss some information..

Please help me improve the problem why I can't visit the website http://10.0.0.151:8088

**if need other information (such as hdfs-site.xml...etc) just tell me. I'll update..

netstat -tunalp | grep LISTEN
tcp        0      0 0.0.0.0:50010           0.0.0.0:*               LISTEN      17442/java
tcp        0      0 0.0.0.0:50075           0.0.0.0:*               LISTEN      17442/java
tcp        0      0 0.0.0.0:50020           0.0.0.0:*               LISTEN      17442/java
tcp        0      0 0.0.0.0:50090           0.0.0.0:*               LISTEN      17693/java
tcp        0      0 10.0.0.151:8020         0.0.0.0:*               LISTEN      17267/java
tcp        0      0 0.0.0.0:50070           0.0.0.0:*               LISTEN      17267/java
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
tcp6       0      0 :::13562                :::*                    LISTEN      21061/java
tcp6       0      0 10.0.0.151:8030         :::*                    LISTEN      20881/java
tcp6       0      0 10.0.0.151:8031         :::*                    LISTEN      20881/java
tcp6       0      0 10.0.0.151:8032         :::*                    LISTEN      20881/java
tcp6       0      0 10.0.0.151:8033         :::*                    LISTEN      20881/java
tcp6       0      0 :::33762                :::*                    LISTEN      21061/java
tcp6       0      0 :::8040                 :::*                    LISTEN      21061/java
tcp6       0      0 :::8042                 :::*                    LISTEN      21061/java
tcp6       0      0 :::22                   :::*                    LISTEN      -
tcp6       0      0 10.0.0.151:8088         :::*                    LISTEN      20881/java

After disable ipv6 , then i type netstat -tunalp | grep LISTEN

tcp        0      0 0.0.0.0:13562           0.0.0.0:*               LISTEN      30608/java
tcp        0      0 0.0.0.0:50010           0.0.0.0:*               LISTEN      29967/java
tcp        0      0 0.0.0.0:50075           0.0.0.0:*               LISTEN      29967/java
tcp        0      0 10.0.0.151:8030         0.0.0.0:*               LISTEN      30424/java
tcp        0      0 10.0.0.151:8031         0.0.0.0:*               LISTEN      30424/java
tcp        0      0 0.0.0.0:52992           0.0.0.0:*               LISTEN      30608/java
tcp        0      0 10.0.0.151:8032         0.0.0.0:*               LISTEN      30424/java
tcp        0      0 10.0.0.151:8033         0.0.0.0:*               LISTEN      30424/java
tcp        0      0 0.0.0.0:50020           0.0.0.0:*               LISTEN      29967/java
tcp        0      0 0.0.0.0:8040            0.0.0.0:*               LISTEN      30608/java
tcp        0      0 0.0.0.0:8042            0.0.0.0:*               LISTEN      30608/java
tcp        0      0 0.0.0.0:50090           0.0.0.0:*               LISTEN      30222/java
tcp        0      0 10.0.0.151:8020         0.0.0.0:*               LISTEN      29790/java
tcp        0      0 0.0.0.0:50070           0.0.0.0:*               LISTEN      29790/java
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -  
tcp        0      0 10.0.0.151:8088         0.0.0.0:*               LISTEN      30424/java
tcp6       0      0 :::22                   :::*                    LISTEN      -

Execute the command "netstat -tunalp | grep LISTEN" on your master node 10.0.0.151 and post the output — SachinJose, Apr 17 '14 at 15:18

score 10 · Answer 1 · edited May 05 '17 at 22:26

10

the reason is you should change 10.0.0.151:8088 to 0.0.0.0:8088. the method is open the yarn-site.xml, add:

<property> 
  <name>yarn.resourcemanager.webapp.address</name>
  <value>0.0.0.0:8088</value>
</property>

edited May 05 '17 at 22:26

OneCricketeer

179,855
19
132
245

answered Sep 10 '15 at 05:29

user5319773

101
1
2

Thank you! This is the one that solved this issue for me – Joon Mar 05 '19 at 21:49

score 7 · Accepted Answer · answered Apr 17 '14 at 16:57

7

Here the problem is resouce manager is running, but the port(8030,8031,8032,8033,8088) occupied by resource manager uses tcp6 instead of tcp(see the left portion). You have two options either you can disable ipv6 in the linux system then restart yarn services.

or

Try modify your yarn-site.xml only on master node as follows. Don't modify the yarn-site.xml in slave nodes:

<configuration>
<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<configuration>

Specifying hostname causes the ports started as tcp6, even with out specifying those ports it would take the default value. Have a look at the following default ports

http://hadoop.apache.org/docs/r2.2.0/hadoop-yarn/hadoop-yarn-common/yarn-default.xml

After modiying yarn-site.xml restart your yarn-service

answered Apr 17 '14 at 16:57

SachinJose

8,462
4
42
63

I disable ipv6 by add following three lines in hadoop01 & 02 at /etc/sysctl.conf But I still can't visit the website 8088, i'll update the information after i disable ipv6 and type netstat -tunalp | grep LISTEN! I also try the method just modify hadoop01(master) yarn-site.xml, but it also can't work.. – steven Apr 17 '14 at 17:33
when i type hadoop command such as start-dfs.sh. It will appear this warn : Unable to load native-hadoop library for your platform... using builtin-java classes where applicable. I know this problem is hadoop version 32bit improper with 64 bits ubuntu.. But it just warn, i think it is nothing to do with my problem 'cannot visit web 8088' – steven Apr 18 '14 at 05:29
And I try to establish vmware with ubuntu 12.04 32bits and hadoop-2.2.0 ,then i set up one node and it's successful to visit 50030 and 8088. But when i try again on my putty(my putty can connect to some vm) and just setup one node. it succeed in 50030 and fail in 8088... – steven Apr 18 '14 at 05:33
(my original problem is setup on my putty's vm) – steven Apr 18 '14 at 05:34

score 1 · Answer 3 · edited May 23 '17 at 11:47

1

For me this from Hadoop: binding multiple IP addresses to a cluster NameNode:

In hdfs-site.xml, set the value of dfs.namenode.rpc-bind-host to 0.0.0.0 and Hadoop will listen on both the private and public network interfaces allowing remote access and datanode access.

and opening 8088 in the firewall settings worked.

edited May 23 '17 at 11:47

Community

1
1

answered Oct 16 '14 at 15:46

user2095935

11
1

score 0 · Answer 4 · answered Jan 05 '15 at 12:24

This is related to https://issues.apache.org/jira/browse/HADOOP-605 ; Hadoop configuration scripts have added the -Djava.net.preferIPv4Stack=true flag to force IPv4 binding, but this is missing from the Yarn configuration script. You can fix this by adding at the end of bin/yarn (before the exec):

YARN_OPTS="$YARN_OPTS -Djava.net.preferIPv4Stack=true"

Hadoop2.2.0 can't visit the web http://:8088

4 Answers4

Linked