12

Disclaimer:

This topic is for HTTP Traffic (on Linux Platform). May be there is a good solution with ELB (with reasonable price for everyone) for the problem below. But so far i can not find any. That's why i need expert advices.

Problem:

I've been using AWS Elastic Load Balancing (ELB) for years. And suddenly realized there is a huge (and critical to me) drawback of ELB .. which is to block the inbound connections by IP.

Because once you are behind ELB(s), your Server Internal Firewalls (like: iptables) are useless already because all the forwarded traffics from ELB are stamped as ELB IP (not the real Client IP). ELB only forwards the Real Client IP as in X-Forwarded-For http header, which is useless for iptables. (Unless you can suggest there is a Linux Firewall like iptables which can also handle HTTP Traffic with XFF (X-Forwarded-For) header inside.)

I understand this is the normal behaviours of such Reverse Proxies, but i need to put a Firewall! I know on AWS, it is suggested to use VPC and the Network ACL rules to BLOCK the inbound connections by IP. But NACLs have the rule limits! (AWS only allows total of maximum 40 rules in NACLs)

Imagine you are running a high traffic Public website, and then need to block a lot bad IPs detected everyday. How would this 40 rules help?

Need Advice:

I'm start thinking of using Nginx as the Load-balancer (on a separate Instance). I've used Nginx before and it is a promising one. And of course, can replace ELB. And then:

  • use the iptables on that Nginx Instance! (So, that VM will become LB+Firewall)

But before i make a move,

  • Are there any better, expert advices?
  • What will be the big difference (impact) of not using ELB here?

Thanks all for advices.

夏期劇場
  • 17,821
  • 44
  • 135
  • 217
  • 2
    If you're using CloudFront as a CDN in front of your ELB then you can make use of Amazons [WAF](https://aws.amazon.com/waf/) (web application firewall) to do things like block inbound connections by IP, etc. If, as you are saying, you're running a "high traffic public website" then you likely do want to use CloudFront to improve performance, so using the AWS WAF should be fairly straightforward. – Bruce P Jan 20 '16 at 16:16

4 Answers4

4

Using ELBs remains valuable because redundancy is part of the service.

Using Nginx as load-balancer would be a single point of failure unless you also set up a standby server and something like heartbeat to automatically fail over to your spare Nginx server.

Consider a layered approach of using both ELB and Nginx. The ELB can forward traffic to two or more web hosts in different Availability Zones, each running Nginx. With Nginx and fail2ban, you can still block hosts by IP address. The general approach works like this:

  • Configure Nginx to log the real IP in the log files, not the ELB IP.
  • Configure fail2ban to watch the Nginx access logs and look for IPs to block.
  • When fail2ban detects an IP that it should block, it updates an Nginx include file and reloads Nginx for the rule to take affect. fail2ban expires bans the same way.

You could also exclude fail2ban and manually maintain a list of IPs to block as well.

A detailed explanation of the approach is available at "Nginx + Fail2ban Blocking IP behind AWS Load Balancer".

Mark Stosberg
  • 12,961
  • 6
  • 44
  • 49
  • Thank you. But i believe this is still using Virtual Hosts to ban/block the IPs. Am i right? This is what I'm already doing with Apache V Hosts. But what I want is to block by Firewall level. I don't want to still stress the Web Servers. – 夏期劇場 Jan 20 '16 at 05:52
  • You aren't stressing the "web servers," here, you're putting the burden on the proxy behind ELB. (+1 this, proxies between ELB and the app server is good advice.) Alternately, haproxy has a feature called "tarpit" which rejects requests from blocked addresses with custom static html or a default fake 500 Internal Server Error *after* a configurable delay, which is is managed by its event loop and has a very low resource cost. – Michael - sqlbot Jan 20 '16 at 10:42
  • @夏期劇場, yes you right-- the solution I proposed blocks requests at the Nginx level, not at the OS level. `haproxy` is good, too. – Mark Stosberg Jan 20 '16 at 13:54
  • @MarkStosberg, No. Then i can easily do such thing with Apache VirtualHost, or even with Htaccess file. Can easily block X-forwarded-for IPs from there. Of course this way we can do. But It's not what i want. I want the REAL FIREWALL to handle. Not at application level. – 夏期劇場 Jan 20 '16 at 15:37
3

Its been a while since this question was asked, but I thought it might be worth pointing out that both Classic and next generation Application Load Balancers now support Security Groups for limiting access to your load balancer - http://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-update-security-groups.html

Jeff
  • 1,538
  • 2
  • 18
  • 33
1

http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.managing.elb.html

You can change the listener protocol from HTTP to TCP if you want the load balancer to forward requests as-is. This prevents the load balancer from rewriting headers (including X-Forwarded-For) and does not work with sticky sessions.

nomad
  • 1,699
  • 5
  • 21
  • 35
  • 1
    I doubt this would work. With ELBs, no matter TCP or HTTP, the connections from clients terminate at the ELB, and the ELB has its own connections to the backend EC2 instances. Therefore, even in TCP's case, the source IP address would be of the ELB. – gmemon Feb 17 '17 at 21:01
  • 1
    You can change the protocol with the classic amazon load balancers. – James O'Brien Apr 05 '18 at 23:52
  • @gmemon My post is a direct quote from the Amazon docs that I linked so I'm pretty sure it works unless the Amazon docs are wrong. – nomad Apr 07 '18 at 02:13
0

Here's a tool I made for those looking to use Fail2Ban on aws with apache, ELB, and ACL: https://github.com/anthonymartin/aws-acl-fail2ban

It's useful for detecting and preventing DoS attacks and abuse of ec2 instances.

Anthony Martin
  • 403
  • 1
  • 5
  • 13