12

I wish to serve images from a S3 bucket with Cloudfront as CDN frontend, for that I tried the following:

What I wish to acheive (Attempt 2) -- (Misses cloudfront cache randomly)

I have the following setup to serve images: ( Cloudfront --> Nginx --> S3 ) Cloudfront -> Nginx -> S3

<<<<<<<< Sample S3 headers >>>>>>>>>> Attempt-2 S3 headers

<<<<<<<< Sample Nginx -> S3 headers (Added Cache-Control) >>>>>>>>>> Attempt-2 Nginx->S3 headers

<<<<<<<< Sample Cloudfront -> Nginx -> S3 headers >>>>>>>>>> Attempt-2 Cloudfront->Nginx->S3 headers

What I am currently working with (Attempt 1) -- (Hits cloudfront as expected everytime)

Cloudfront -> S3

Cloudfront Settings:

What am I screwing up in Attemp-2 with my headers ? ( Cloudfront missing randomly )

Url(http://cdn.example.com/abc.jpg) & Url(http://cdn.example.com/abc.jpg?v=1) both will have same ETag, is that fine ?

Update

#AWS followed up on forums.aws.amazon.com, still waiting for a reply:

https://forums.aws.amazon.com/thread.jspa?threadID=144286&tstart=0#

Update2

  • Recent hit/miss behavioral change from cloudfront without changing anything.
  • Earlier the hits/misses were random with no fix pattern
  • Now, (with no change on my end) I am getting all hits 1 day and all misses the next day.
  • This suggests that its 24 hour cache but TTL and cache headers suggest 5 year cache expiry.
  • This is again weird and without any explanations.

Hey, AWS can you see this ???

Yugal Jindle
  • 44,057
  • 43
  • 129
  • 197
  • To be clear, you are saying that the Nginx version sometimes generates a Cloudfront hit, but does not always generate a cloudfront hit... correct? Does it hit more often than it misses, or does it miss more often than it hits? – Michael - sqlbot Jan 21 '14 at 07:45
  • Yes, (With Nginx in-between) it sometimes results in cloudfront hit and sometimes a miss. If I observe it over a small period of time (Hit it agian and again) - it hits and misses randomly. But when I observe it over a period of day, I mostly see misses. – Yugal Jindle Jan 21 '14 at 08:22

2 Answers2

1

After about 4 months of repetitive to-n-fro with amazon support failed to resolve the issue.

All problems still persisting:

  • The cache expires in about a day and misses after 24 hours. ( My expiry is 1 year )

  • All headers and aws settings verified by amazon support themselves

  • Unfortunately, the company is still paying for this awful experience due to lockin.

enter image description here

------ After 24 hours ------

enter image description here

------ After 24 hours ------

enter image description here

------ After 24 hours ------

enter image description here

-------- And so on.. -------

Concluding, The problem still stands unresolved and amazon support seems to have given up. This is quiet a strange experience since aws is something we generally take for granted.

:(

Yugal Jindle
  • 44,057
  • 43
  • 129
  • 197
0

Checking the 2 IPs you listed in AWS forum when you run dig resolver-identity.cloudfront.net:

74.125.190.17 is from Google in CA, USA   
67.215.80.11 is from OpenDNS from Singapore

Are you using OpenDNS or Google DNS? If you are, can you try to disable them and see if CloudFront will work properly from then on?

There is an article here talking about the disadvantage when you using OpenDNS/Google DNS with CDN, which may be helpful. http://apcmag.com/why-using-google-dns-opendns-is-a-bad-idea.htm

Yudong Li
  • 1,784
  • 2
  • 17
  • 32
  • Should we expect our users to do what you are suggesting ? Whatever impact any DNS server is creating should be taken care by `Cloudfront`. I suppose this is why `CDN` are costly because its hard to do. Will you ask your customers to do what you are suggesting ? – Yugal Jindle Feb 11 '14 at 06:16
  • Also, the other cloudfront backed directly with S3 works just fine on the same network and DNS resolver. How is that possible ? – Yugal Jindle Feb 11 '14 at 06:19
  • By looking at your location, it appears you are from India. Checking OpenDNS's closest location, its either in Singapore, or somewhere in Europe. To CloudFront, it can only see the request coming from Singapore, or EU, or anywhere the DNS redirect from, there is no way for CloudFront to help you/take care here. To put it shortly, when you try to use different products, there will always be limitations and conditions. I believe this is not unique to CloudFront but for most of the CDN products. – Yudong Li Feb 11 '14 at 06:20
  • For your second Q, to be honest I am not sure why, but I have a bit suspicion that the same result may happen as your try with the nginx proxy in the middle. If I were you, what I will do is set up a cron job running every half a minute for like an hour or so, to execute dig resolver-identity.cloudfront.net. Compare the result to see if it is indeed the cause of DNS. Again, I'm not certain this is the cause, that's why I suggest you to verify it first. – Yudong Li Feb 11 '14 at 06:26
  • How is my other Cloudfront backed with S3 directly working just fine, I am using both from this very same location India. Please suggest the difference from the point of view of Cloudfront ? – Yugal Jindle Feb 11 '14 at 06:27
  • I have a cron running to verify that `Cloudfront without Nginx` is hitting everytime (flawlessly) and `Cloudfront with Nginx` is missing randomly. Tested both on 2 different ISP providers (Same results). – Yugal Jindle Feb 11 '14 at 06:30
  • Can you compare the dig resolver-identity.cloudfront.net A record in your both two tests? What IPs are resolved? Who are they belonging to? Where are they? – Yudong Li Feb 11 '14 at 06:34
  • As shown in `aws forums` `dig resolver-identity.cloudfront.net` is resolving to different DNS addresses from the very same machine where I am conducting my tests. So, with the changing resolver `1 cloudfront` instance is working where as other is not. I doubt `DNS resolution` is causing this. – Yugal Jindle Feb 11 '14 at 06:50
  • Actually this is the weird thing, dig command has nothing to do with your setup, with or without nginx. It's just a normal unix request, as a result, I'd double check why the same dig returns different results. – Yudong Li Feb 11 '14 at 12:25
  • I know it has nothing to do with any setup. Whatever its yielding (2 different IP) is same for both `cloudfront` instances out of which 1 is functioning as expected and the other isn't. :( – Yugal Jindle Feb 11 '14 at 13:05
  • Please delete this answer, since it solves nothing. Its same as what's stated in aws forum. – Yugal Jindle Feb 11 '14 at 19:22