0

Many HTTP requests to my server fail with 524 (Origin time-out). The resources that fail to be delivered are not always the same. Sometimes they work, and the next request they don't.

I whitelisted the Cloudflare IPs. Unfortunately this can't help much because the source IPs that actually hit my server are from a french Cloudflare server whose IPs (172.68.51.*) are not in the official Cloudflare IP list. So I temporarily added a firewall rule that allows everything (all IPs, all Ports, all protocols, etc.). Also I don't have any Block rules (which might override the allow everything rule).

What is even stranger is that the Windows firewall doesn't log any blocked requests for the timed-out resources. Also IIS doesn't log any requests for those resources so the problem has to be some time before IIS is involved.

If I completely disable the Windows firewall everything works as expected (all resources succeed to be delivered / no time-outs). If I do the requests via another domain (one that isn't mapped to Cloudflare) again everything works.

What might be the cause of the problem?

For testing I used Cloudflare's development mode (no caching but still proxying) because obviously there are no problems with resources that are already cached by Cloudflare.

Update: Even with bypassing all rules (Changing "Inbound connections" from "Block (default)" to "Allow") it doesn't work so it can't be a rule problem. The problem has to lie somewhere between disabling the firewall (works) and enabling the firewall but allowing all inbound and outbound connections (doesn't work).

Wireshark shows TCP Retransmission errors when the firewall is on. When disabling the firewall they don't occur:

enter image description here

user764754
  • 101
  • 4
  • "What might be the cause of the problem?". Since it works when the Windows firewall is turned off, it's that. It must be blocking something. – Tim Dec 01 '16 at 18:41
  • @Tim It must do this in a strange way because it doesn't log that anything is blocked. – user764754 Dec 02 '16 at 01:16
  • Agreed, however if only one things changes between working and non-working states that change is likely responsible. – Tim Dec 02 '16 at 01:17
  • @Tim Makes sense. I'll look at the log files again tomorrow and all the firewall settings. – user764754 Dec 02 '16 at 01:25
  • @Tim I updated my question. – user764754 Dec 02 '16 at 14:30
  • 1
    Just commenting to say that 172.68.51.* is in CF's official list, covered by 172.64.0.0/13, listed here: https://www.cloudflare.com/ips-v4 – parkamark Dec 02 '16 at 14:48
  • @parkamark Thanks, that's good to know. I'm not a server/network guy so I don't know IP wildcards. – user764754 Dec 02 '16 at 14:55

1 Answers1

0

The problem was indeed the Windows firewall although I whitelisted everything and no blocked packets were logged. There is this strange firewall stealth mode that doesn't log anything when it drops packets.

I disabled it for my active firewall profile and the time-outs where gone even when removing my allow-everything rule (I kept the Cloudflare IPs whitelisted though) .

Also a server restart was required in order for the Windows firewall to pick up the registry changes.

Update: I made this batch script to disable stealth mode:

REG ADD HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\WindowsFirewall\DomainProfile /v DisableStealthMode /t REG_DWORD /d 1
REG ADD HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\WindowsFirewall\PrivateProfile /v DisableStealthMode /t REG_DWORD /d 1
REG ADD HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\WindowsFirewall\PublicProfile /v DisableStealthMode /t REG_DWORD /d 1
REG ADD HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\WindowsFirewall\StandardProfile /v DisableStealthMode /t REG_DWORD /d 1
PAUSE
Community
  • 1
user764754
  • 101
  • 4
  • Good to hear you worked it out. If you can, please mark your own answer as "answered" so people don't come in and try to help. – Tim Dec 02 '16 at 17:49