0

Here's a step-by-step rundown of my end-of-week nightmare:

-Nov 8 @ 2am: One of our two internet providers began experiencing problems. Packets were being dropped and connectivity for our clients (accessing our web services) was being impacted. For insight I host our primary DNS here locally but outside of our network.

-Nov 9 @ 12:30pm: since we currently had only 2 DNS Resolvers set up (us as the primary and another external as a slave) I added 8.8.8.8 (Google public DNS) in as a name server and configured zone transfers to them. I was under the impression that there would be a delay before Google became a slave.

-Nov 9 @ 1:30pm: because of the packet drops I pulled the A records resolving to the IP address belonging to the bad provider. This left us with A records for only one IP (our good provider). The TTL at the time for all was set to 1 hour. After I pulled the records I set the TTL to 5 minutes so that I would be able to quickly add the removed A records back in when the provider resolved their issues.

-Shortly after this our website started disappearing in remote areas of the world.

-Nov 10 @ 7:30am: the bad provider fixes their equipment and I add back the A records that I had previously deleted.

My question is why is my website still inaccessible from certain parts of the world? Doing NSLOOKUP's I'm fine in Italy but not in Australia. Good in most of the USA but not all.

I also found (to my horror) that some DNS servers were caching my A records beyond their stated TTL. I did not even know this was possible. For instance; I created an A record with a TTL of 5 minutes, waited 10 and then checked with Google public DNS (8.8.8.8). It was there. Then I deleted the record, waited 20 minutes and checked again. It was still there. It stayed for some time but eventually disappeared.

Any help with this issue is greatly appreciated.

JohnyD
  • 1,614
  • 3
  • 20
  • 29
  • So you're saying that you added Google's public DNS (8.8.8.8) as a name server for your DNS namespace? I didn't think that was possible. – joeqwerty Nov 10 '11 at 13:42

1 Answers1

4

Google does not host authoritative DNS for folks. Their DNS service is for recursive resolution only.

Source: http://code.google.com/speed/public-dns/docs/intro.html

JakePaulus
  • 2,347
  • 16
  • 17
  • And the side effects of doing so? – JohnyD Nov 10 '11 at 14:34
  • The result is you are experience intermittent name resolution failures that are impacting your users. – JakePaulus Nov 10 '11 at 14:42
  • The side effect is that nothing happened. Your DNS zone isn't hosted by Google's public DNS servers and you were left with the original problem. Only when the failing provider resolved their issue did things get back to normal. – joeqwerty Nov 10 '11 at 15:53