0

We keep static files (images, javascript, and css) for our websites stored in a Google Storage bucket with different folders for different types of resources. Each file is accessed via its name coupled with a custom subdomain mapped via a CNAME record to the appropriate Google Storage bucket.

This approached has worked fine. Today, however, when attempting to access our main website in Chrome's incognito (private browsing) mode, all pages on the site wouldn't load. After some detective work, we've determined that the problem is with the files stored at Google Storage, which are not loading.

Unfortunately, this doesn't seem to be a problem specific to Google Chrome. It occurs in the private browsing modes in Firefox and Internet Explorer as well (at least on the Windows 8.1 Professional platform we're using for testing).

The problem appears to occur only if we use the CNAME-based approach for accessing a file. For example, if this method is used in a private browser window to access one of our image files on Google Storage,

Image of a crowd on Google Storage - direct access to Google Storage

the file can be viewed without a problem. If, on the other hand, the file is viewed in a private browsing window using the CNAME approach, like this

Image of a crowd on Google Storage - access via CNAME

the image will not load.

What's worse, for reasons we don't completely understand, once this problem occurs in a private browsing window, it will continue to interfere with the proper viewing of the website in regular (non-private browsing) browser windows in the case of some browsers.

Has anyone encountered this problem and, if so, found a solution for it?

Thanks in advance for any tips or suggestions.

UPDATE (2015-05-26)

This problem is still under investigation. It may be ISP-specific, although our ISP (Verizon) believes it is a problem on Google's end. An attempt to resolve the problem yesterday by tweaking some DNS settings seemed to solve the problem, but that was only temporary. We began to experience the problem again today. I will update this posting further as more information becomes available.

ADDITIONAL UPDATE (2016-08-25)

(Note: I originally wrote this update on 2015-05-26, but failed to post it, and discovered it today. I'm adding it to complete the description of the issue.)

This issue has been resolved. I cannot say for certain what the source of the problem was, but I can give further information on what exactly the nature of the problem was and what may have solved it.

As I mentioned in the comments below, this appears to have been an issue that was relatively isolated. Further investigation revealed that the problem was occurring only with access to the particular subdomain through Verizon Internet service (land-based or mobile) in the U.S. I do not know if the problem was a regional problem within the Verizon system, or throughout the entire Verizon system. But I do know it affected both landline and mobile access using Verizon.

The problem also evolved. What started as a problem accessing files at the subdomain in a browser's incognito mode became a problem regardless of what browsing mode was used. That said, it was only a problem if the attempt to load files from the subdomain was used with a browser. The files could be retrieved with no problem with, for example, wget. Also, pinging the subdomain also worked fine over the Verizon network.

As the problem became more acute, I decided to do a thorough check of the DNS settings related to the subdomain. Here is where I discovered what may have been causing the problem. There was a slight discrepancy between the DNS settings at the domain registrar and the (separate) DNS service that we use.

The discrepancy didn't lead to conflicting reports as to how the subdomain should be resolved (which is probably why this problem hadn't occurred in the past). But, if I recall correctly, it led to the DNS service providing the CNAME record for the subdomain, without the registrar's DNS information fully confirming that the DNS service had the right to provide that information.

This discrepancy was corrected. Within an hour or two, the problem resolved itself -- anyone viewing the file using the two links above should be successful with both links.

I cannot say for certain, however, whether the change to the DNS settings we made to resolve the discrepancy, or some updating at Verizon, was responsible for the problem being resolved. I will say, however, that I never reported the issue to Verizon. (I didn't get that far.)

Although the DNS discrepancy had existed for more than a year or two, and had not created any problems that we were aware of, I personally think it is what caused the problem.

simpleb
  • 11
  • 3
  • your issue is somewhere else... I can open both links on either private or normal browser – Patrice May 25 '15 at 17:07
  • @Patrice - Are you accessing the files from a Windows-based computer or a Mac or Linux system? We've tested this issue on several different Windows computers, using Windows 7 and Windows 8.1, and they all are experiencing the same issue I described above. (Granted, they also are from the same location / IP address, and perhaps the Internet connection is the issue ... something that we are checking.) – simpleb May 25 '15 at 17:16
  • I'm on a windows 7 machine, with the latest Chrome. Works on both. – Patrice May 25 '15 at 17:17
  • Thanks @Patrice - This may just be a local issue occurring at our specific location. It doesn't seem to be specific to Windows on our computers. I fired up an Ubuntu VirtualBox instance on my computer and encountered the same problem trying to load the file specified with the CNAME method. Oddly, I could wget and curl the file in the Ubuntu instance, but not view it in the browser in Ubuntu. Any suggestions on what the source of the problem might be would be very welcome. Thanks! – simpleb May 25 '15 at 18:27
  • Some experimentation we did yesterday to isolate the source of this problem led us to believe it may be related to our ISP, which is Verizon. We had a website visitor test access through Comcast Internet access, and there was not issue. However, there was an issue with the static9.light-kr.com CNAME domain when the site was viewed using Verizon's mobile network. We are investigating that possibility. – simpleb May 26 '15 at 18:43
  • I can attest I am not on Verizon, so this might be why I got it ^^ – Patrice May 26 '15 at 20:30
  • Thanks for the feedback, @Patrice . We thought you might be on a different network. We had checked with someone yesterday who accesses the site via Comcast, and he was not having any problem. The problem again has become extensive at our particular location, but we're not certain whether it's just a localized issue, or widespread. Verizon is trying to sort things out, and we've also reached out to Google (although we haven't heard back from Google yet). – simpleb May 26 '15 at 20:55
  • how did you reach out, if I may? – Patrice May 26 '15 at 21:21
  • @Patrice - We sent an email to gs-team@google.com , as suggested on the Google Cloud Stage ["Resources & Support" page](https://cloud.google.com/storage/docs/resources-support#support-packages). – simpleb May 26 '15 at 23:43
  • any answers from the gs team yet? (This is a)because I'm curious and b)... well I'm actually in the cloud support team, so I'll see what I can do from my end if they haven't ;) ) – Patrice May 27 '15 at 22:11
  • Thank you, @Patrice , for checking. No response yet from the GS team. And, yes, we're still having exactly the same problem that I outlined above. – simpleb May 28 '15 at 01:48

0 Answers0