IIS Servers interrupted - thousands of http 460 responses - what could cause this and how to prevent

Question

On Wednesday we had a period of disruted service on the webserver side of things. We generally deal with 2,000-10,000 users connected to our site and on Wednesday at around 15:35 we received 30,000+ connections (within 5 minutes) which were all to our login page from 1 IP address (A customer that's a college) and all requests to this page returned a http code of 460 (which I've never come across before on any server).

I'm going to assume this was malicious as there is no other reason behind this many connections to the login page even with 10,000 users online nevermind from 1 customer at a time of only having 3-4,000 online.

What would someone have been trying to achieve to cause a 460 error, is there a known attack that would generate this? I'm going to pass it on to our security tester on Monday to look into, but thought I'd put it out there first too.

It's like the connections stayed live too as even after, connections were still not all going through, low CPU yet timeouts occuring. I had to restart IIS on each server to resolve...

Oh we're running 12 Windows 2012 R2 IIS web servers through Amazon ELB and our website is ASP.NET based and the SQL server CPU and batch requests didn't increase when these connection occured.

Here's an example row from the 30k+ logs:

RequestTime             RequestIPID     RequestProcessingTime       BackendProcessingTime       ClientResponseTime      ELBResponseCode     BackendResponseCode     ReceivedBytes       SentBytes       UserAgentID                                                                                                     httpquerystring             httpmethod      Path           Domain
28/09/2016 15:37:00     IP              0.002                       0                               -1                      460                     -                       60                      0               Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36   OrgID=nameoforg             GET             Login.aspx         https://hiddendomain

Regards Liam

Liam Wheldon · Answer 1 · 2017-02-14T15:02:10.043

We've actually now diagnosed this issue to be being caused by a slow HTTP attack (slowloris) caused by lots of requests with no connection header, causing a client disconnection from the load balancer but load balancer and IIS keeping the connection alive and therefore maxing out all the connections as they pick up a default of 120 seconds timeout. We're now working on making changes to the IIS limits. I'll update with further detail when we find our optimal settings to help mitigate such attacks.

We've set the following in web.config

<system.web>
     <httpRuntime requestValidationMode="2.0" targetFramework="4.5" maxRequestLength="50097151" maxUrlLength="2048" maxQueryStringLength="1024"/>
</system.web>
  <system.webServer>
     <security>
         <requestFiltering>
            <requestLimits>
               <headerLimits>
                  <clear />
                  <add header="Content-type" sizeLimit="100" />
               </headerLimits>
            </requestLimits>
         </requestFiltering>
      </security>
     <httpProtocol>
      <customHeaders>
        <clear/>
        <add name="X-Frame-Options" value="SAMEORIGIN" />
        <add name="X-XSS-Protection" value="1; mode=block"/>
      </customHeaders>
    </httpProtocol>
  </system.webServer>

Regards

Liam

score -1 · Answer 2 · answered Sep 30 '16 at 15:28

-1

The disrupted service to your web servers appears to have been caused by a denial-of-service (DoS) attack from a single IP address. For those hosting their own web servers, manual exclusion of problem IP addresses can be done via network hardware and/or software. Some network hardware vendors sell special defensive devices to automate this process with pre-defined rules to detect DoS attacks and then block the applicable IP addresses.

Since Amazon ELB is hosting your web servers, you may want to pursue enhanced security options from AWS. AWS Identity and Access Management (IAM) appears to be something that is applicable to your situation. Per AWS, "IAM is a feature of your AWS account offered at no additional charge."

answered Sep 30 '16 at 15:28

JohnH

1,920
4
25
32

Hi John, Thanks for your response. I understand that it's a DoS attack from single IP. However IAM doesn't offer protection like this IAM is for AWS rights management e.g. who can acces what within the API and console as well as Access keys and so on. We use Cloudflare too but not in it's full protection mode at the minute. I was looking more towards an answer of what would cause a 460 http response so I cna run tests to stopping this and also if the full CloudFlare protection would stop this. Cloudflare isn't fully implemented due to hundred of URLs needed and currently under testing. – Liam Wheldon Sep 30 '16 at 15:59
Can you review the web logs and examine the requests that produced the 460 HTTP status codes in their responses? You can then attempt to reproduce a similar request and response while testing. – JohnH Sep 30 '16 at 16:08
Hi John, the logs won't show anymore info than the example above as it won't show anything like the request data itself only the response. I'd like to know how someone makes the server produce a 460 and look into how I can prevent it by testing. – Liam Wheldon Oct 01 '16 at 16:26
Hi Liam, If you cannot get the full url request string from the AWS logs, then you may want to log the full url request strings to a SQL Server table or a .txt file. That would at least give you the information needed to help counter future attacks. Unfortunately, this will not help you to reproduce the past problem requests that generated responses with the 460 HTTP status codes. Sooner or later, I believe that you will probably need to have the ability to examine the full url request string transmitted to you web servers for diagnostic purposes. – JohnH Oct 03 '16 at 15:25
Oh sorry I didn't realise you mean the URL string. Yes of course I have that lol it's in the log example i added to my original post. My general question was about how comeone causes a 460 error not which page it was directed at.. – Liam Wheldon Oct 03 '16 at 15:27

IIS Servers interrupted - thousands of http 460 responses - what could cause this and how to prevent

2 Answers2