WGet from one site on a server to another site on the same server

Question

Hey all, I've recently been asked to administer a couple ubuntu boxes running web servers. I'm a dev by trade so if this question is fairly noob please forgive.

We have about a dozen sites running on this box. 2 of our sites need to talk back and forth over a restful api. Unfortunately we are having issues with the sited connection to each other via wget. When we try and run wget manually from the command line from the server pointing to a site also on that server it hangs and eventually times out. If we do the same thing from outside the server to the same site on the server it works.

Is there something that could be preventing sites on the same server from communicating with each other? The same thing happens pinging the site from the server.

Answer 1

I'd check /etc/hosts for incorrect entries. I'd also check the hosts resolution order and DNS configuration.

For example, if your run wget http://www.example.com/whatever and /etc/hosts contains an entry for www.example.com then this command would fail when run locally.

---

Edit

/etc/hosts doesn't contain "routes" in the normal sense (see netstat -nr) it contains associations between hostnames and ip-addresses.

You mention "internal ip names" and "external ip" - Are you referring to an "external ip address" and an "internal ip address" where the external ip-address is an address for the external interface of your router and hence the one used by Internet users to access your production web-server?

If so, it is likely that your web-server thinks that is an external off-LAN ip-address, forwards the wget traffic to your router, whose NAT & port-mapping doesn't allow routing to that ip-address from it's LAN side.

If so, edit /etc/hosts and either replace the external ip-address with the internal ip-address or add an extra entry test.host.local with the internal 10.n.n.n ip-address and use that new name in wget http://test.host.local/whatever.

Answer 2

I assume you have a setup like so:

Internet--NAT device/router--internal network-->web server

Suppose the web server has name www.example.com and resolves to the external IP address 1.2.3.4 and the NAT device DNATs 1.2.3.4 to 10.4.4.4 which is internal IP of the web server.

This works well for external clients but internal clients need additional configuration. Reason: Suppose the internal client (10.4.4.5) wants to reach www.example.com (1.2.3.4). It sends the traffic to the default router (let's say it is the NAT device though complex routing setups are more likely). The NAT device NATs the dest IP from 1.2.3.4 to 10.4.4.4 and send the traffic to 10.4.4.4. (At this point a dumb NAT may send a (useless) ICMP redirect because traffic is going out the same interface it came in on.) 10.4.4.4 gets the traffic, formulates a response, and sends it back to the client 10.4.4.5. No routing/NATing is done because the web server is on the same subnet. 10.4.4.5 receives traffic from 10.4.4.4 and goes "WTF? (i.e., sends a TCP reset packet) because it wasn't talking to 10.4.4.4. It was talking to 1.2.3.4 and doesn't know 1.2.3.4 is really 10.4.4.4. Only the NAT device knows that.

There are three solutions to this.

1. SNAT. When the NAT device receives a request for a NATed external IP from the inside it NATs the source address to (one of) its own IP address so it gets the response from the web server and can perform a DNAT on them.

2. Differing internal/external DNS. Configure the internal name servers to return 10.4.4.4 for www.example.com. (Requires another name server to server external queries with the external IP address.)

3. Since the client in this case is the web server itself you could add www.example.com as an alias for 127.0.0.1. (Easy but doesn't scale.)