5

We have deployed a simple ServiceStack application delivering Podcast RSS-feeds in IIS 7. Scouring the IIS logs, we found that a tiny amount of the requests get a HTTP 500.19 error code.

Of the ~550 000 daily requests, around 600 of them fail with 500.19. Which means we successfully deliver RSS on 99.9% of the requests. I am still curious to understand why we fail for 0.1% of them.

500.19 generally happens due to an invalid configuration. How can the config fail for only 0.1% of the requests?

I have not been able to reproduce the error manually, so I don't have the HResult code. This is not present in the IIS log either. However, the sc-win32-status in the IIS log is 64

Some details:

  • The internal logging of the application never reports any error, suggesting the request never reaches our code.
  • The error usually happens to the biggest RSS (740KB gzipped), but also happens to smaller ones (140KB gzipped).
  • The failing requests usually comes from mobile clients (IOS, Android), which doesn't tell much as podcast is usually consumed on mobile.
  • The application is deployed on 6 nodes, load balanced with BigIP. Config file is not shared accross nodes. The 0.1% error rate is equal on all nodes.
  • We have output cache activated in IIS (but no kernel cache).
Øyvind Holmstad
  • 151
  • 1
  • One reason why some requests get a 500.19 while most are not could be that a different configuration applies to those requests and there is a problem with it. This could be because of a web.config in a sub-folder or a location node in the root web.config or even in applicationHost.config. Do the failing requests have anything in common, are they all in the same location? Just one thing to check. – Peter Hahndorf Jul 10 '15 at 11:07
  • 1
    @PeterHahndorf Requests to a specific url that I see have failed in the log, does not fail when I try them myself. The pattern we have seen is that the requests are from mobile phones, and often from IPs in cellular networks. That could be key to understanding what is happening. – Øyvind Holmstad Jul 10 '15 at 11:39
  • The win32 error 64: `The specified network name is no longer available.` is interesting, It indicates it has nothing to do with the server configuration. – Peter Hahndorf Jul 10 '15 at 12:02
  • Have you checked the physical location corresponding to the IP address of some of the requests which result in a 500.19 error code? If they are from somewhere you do not expect requests from, then put it down to malicious requests. You can always expect a number of them on a web server. – Andrew Morton Jul 10 '15 at 17:27
  • 1
    @AndrewMorton Yes. A lot of the failing requests come from known cellular networks in our area (no shocker, most Podcasts are consumed on the go). The error could somehow be related to big sized RSS-feeds and unstable cellular networks, but I guess that should leave other traces in the logs (timeouts). – Øyvind Holmstad Jul 11 '15 at 08:52

0 Answers0