2

I am currently trying to set up a Hadoop distributed compute cluster on my network. Currently. my cluster nodes are encountering a problem communicating with my master server.

Right now I am working on two computers, CLIENT and SERVER.

On SERVER:

$ nmap SERVER -p 9000

Starting Nmap 5.21 ( http://nmap.org ) at 2012-05-29 13:16 PDT
Nmap scan report for ncoiasi1 (127.0.0.1)
Host is up (0.000032s latency).
Hostname ncoiasi1 resolves to 2 IPs. Only scanned 127.0.0.1
rDNS record for 127.0.0.1: localhost
PORT     STATE SERVICE
9000/tcp open  cslistener

On CLIENT:

$ nmap SERVER -p 9000

Starting Nmap 5.21 ( http://nmap.org ) at 2012-05-29 13:16 PDT
Nmap scan report for ncoiasi1 (10.23.95.197)
Host is up (0.00020s latency).
rDNS record for 10.23.95.197: NCOIASI1
PORT     STATE  SERVICE
9000/tcp closed cslistener

I have done the following things:

  1. Made sure both machines have an entry in /etc/hosts, and put ALL:ALL in the /etc/hosts.allow on both machines
  2. Disabled the firewall on both machines (safe to do since I am behind a stringent corporate firewall)
  3. used lsof to verify that the correct process is listening on 9000.

Any help would be appreciated; I know it's just some configuration I've forgotten somewhere, but I can't find where.

Skyhawk
  • 14,200
  • 4
  • 53
  • 95
ILikeFood
  • 399
  • 1
  • 5
  • 12

1 Answers1

5

Your server is set up to only listen on its loopback, not on the NIC exposed to the client. Try 'netstat -an | grep :9000' and you'll likely only see 127.0.0.1:9000. Edit the Hadoop server's configuration file so that it includes its NIC's address. This post covers what needs to be changed: https://stackoverflow.com/questions/4855808/hadoop-job-tracker-only-accessible-from-localhost

Community
  • 1
rnxrx
  • 8,143
  • 3
  • 22
  • 31
  • 1
    And that's just the ticket! It hadn't occurred to me that fs.default.name might limit the network the program was listening on. Thanks for your help! – ILikeFood May 29 '12 at 21:10