0

I know there are tons of posts about not being able to connect to an AWS ec2 instance, but this is not quite the same. It is similar to What causes the 'Connection Refused' message? but that was not able to solve my problem.

I have quite a lot of experience with ec2 instances, but it is the first time I try it with REHL.

Here is my setup:

  • I have a machine A with an apache server listening on port 80
  • Machine A is also connected to an AWS EC2 server S with a reverse tunnel:

ssh -v -NTR 1101:localhost:80 someUser@myAwsIp -o ExitOnForwardFailure=yes

  • Server S has the following port forwarding:

iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 1101

In simple words, all traffic coming into S on port 80 should go to A on port 80. But it's not working. That is to say, when I run curl myAwsIp:80 from a machine B, curl returns with "connection refused".

Some basic facts:

  • I am sure the ip is correct since I ssh into the server to run these commands
  • I have no iptables rules to drop anything. Both 'filter' and 'nat' tables are entirely set to ACCEPT (except the rule mentioned above)
  • I have opened port 80 on my EC2 Management Console. It looks like this:

Ports Protocol Source

80 tcp 0.0.0.0/0 22 tcp 0.0.0.0/0

  • I am not very familiar with semanage, but from searching online I saw that it could be a problem. When I run semanage port -l | grep 80 I can see the line "http_port_t tcp 80, 81, 443, 488, 8008, 8009, 8443, 9000" which should be enough to let curl requests through, right?

My debugging has shown that:

  • the httpd server is running and listening on the right port because when I am logged in on S and I run curl localhost:1101 I get my webpage
  • the forwarding should be working because when I run nc -kl 1101 on S (instead of the ssh port forwarding) and when I run curl myAwsIp:80 from my machine B, I get an incoming connection request on netcat.

But still when I run curl myAwsIp:80 from my machine B, curl returns with "connection refused" (without nc running of course)

What am I missing??? Thanks for your help! This is frustrating! Note that this exact same setup was working on another AWS EC2 instance but with Amazon-linux installed... is there so much difference between the two?

Community
  • 1
cd127
  • 3
  • 2
  • Possible duplicate of [What causes the 'Connection Refused' message?](http://serverfault.com/questions/725262/what-causes-the-connection-refused-message) – user9517 Feb 14 '16 at 14:26
  • @Iain I reworded the question (added the actual complexity) to show that it is different from your suggested duplicate. – cd127 Feb 14 '16 at 16:27
  • There is no evidence to suggest that this is not a dupe. The IP address of localhost is not the same IP address as myAwsIp. – user9517 Feb 14 '16 at 16:34
  • could you please elaborate? I will do whichever tests you think may help. I already took so much effort to explain it all in detail... I also ran tcpdump before reformulating and I did not run into either of the two cases you describe in your post. I also ran netstat, which clearly shows sshd on port 1101 and nothing on port 80 (otherwise how do you explain the netcat getting the request when listening to port 1101?) – cd127 Feb 14 '16 at 16:46
  • @Iain could you please elaborate? I will do whichever tests you think may help. I already took so much effort to explain it all in detail... I also ran tcpdump before reformulating and I did not run into either of the two cases you describe in your post. I also ran netstat, which clearly shows sshd on port 1101 and nothing on port 80 (otherwise how do you explain the netcat getting the request when listening to port 1101?) – cd127 Feb 16 '16 at 06:51
  • @cd127 The link does answer your question. The only difference is that in your setup there are so many layers that you have lost track of which layer you have a problem with. Why are you even using ssh port forwarding here? Requests from users to a production system shouldn't be going through ssh port forwarding. – kasperd Feb 18 '16 at 12:18

1 Answers1

0

There is a few reasons why it is not advisable to use ssh port forwarding to get requests from your users to your production server.

Since you won't have an end-to-end TCP connection your server will never see the IP addresses of the clients. This means your logs will never show the correct client IP addresses which will likely make debugging problems harder for you in the future.

Additionally the added complexity introduces new possible failure scenarios which could ultimately lead to a less reliable service.

In your particular setup you also misconfigured the ssh port forwarding in a way which will prevent it from working. The port forwarding is listening on localhost(::1 and/or 127.0.0.1). But the connections it need to forward are arriving on an external IP address and thus will always get connection refused.

In order to get the ssh port forwarding working you have to add this line to your /etc/ssh/sshd_config

GatewayPorts clientspecified

Additionally your port forwarding need to specify a listening address which could for example look like this:

-R '[2001:db8::1]:1101:localhost:80' -R '192.0.2.3:1101:localhost:80'
kasperd
  • 30,455
  • 17
  • 76
  • 124