1

I have about 50k .jpg images stored in S3 and I have configured CloudFront to deliver those through the CDN. My cache hit rate over the past month is only about 16% on average. Under behavour, I have configured the CDN to have a Min TTL of 1 day (86400 seconds), a Max TTL of 1 year (31536000 seconds) and a default TTL of 1 day. From S3, each of my images responds with the following Cache-control header:

Cache-Control: public, max-age=31536000

Here are the other response headers from the Origin Server (S3). 

Accept-Ranges: bytes
Cache-Control: public, max-age=31536000
Content-Length: 22420
Content-Type: image/jpeg
Date: Fri, 24 Jan 2020 00:50:55 GMT
ETag: "75b4b9e2693058f143df962efe7305dd"
Last-Modified: Wed, 04 Dec 2019 15:20:08 GMT
Server: AmazonS3
x-amz-id-2: rpJdSKKlDkGqxIQrY2Lu887IYuwJPeoZQ40c8CQa11ZEjfuUdLtG19m/frrOeXtDK3gByc6pOlc=
x-amz-request-id: E68021E10D5DB297

If I wait a while (30 minutes) and request this image again, I get a cache miss. Immediately after, I can request the image again and get a cache hit. I can get a cache hit for subsequent calls as long as I don't leave the object untouched for any period of time. I don't understand why the Min TTL value that I set isn't keeping the images in cache longer. I do understand that Amazon will remove images from the Cache that aren't accessed as frequently as others, but this is causing my cache miss rate to become quite high. How can this issue be resolved?

enter image description here

Thanks in advance.

UPDATE

Below I have posted my behavior configs:

enter image description here

Here are my Cloudfront response headers:

accept-ranges: bytes
age: 53
cache-control: public, max-age=31536000
content-length: 6707
content-type: image/jpeg
date: Sat, 25 Jan 2020 00:06:47 GMT
etag: "b511907ae7c3c2299d1e3022f6253f2f"
last-modified: Fri, 11 Oct 2019 05:08:40 GMT
server: AmazonS3
status: 200
via: 1.1 15e6db7015a157ee446cfd6592fca1b3.cloudfront.net (CloudFront)
x-amz-cf-id: AFAkhVgvzj9NjiEFz-222m0uJE5G8B5R2bE06K5Dnsn3Rdz2NqEkdw==
x-amz-cf-pop: ATL52-C1
x-cache: Hit from cloudfront

Wait 5 minutes and request the same image again. Here are the headers for that response. 

accept-ranges: bytes
cache-control: public, max-age=31536000
content-length: 6707
content-type: image/jpeg
date: Sat, 25 Jan 2020 00:12:01 GMT
etag: "b511907ae7c3c2299d1e3022f6253f2f"
last-modified: Fri, 11 Oct 2019 05:08:40 GMT
server: AmazonS3
status: 200
via: 1.1 e11ff5647028a0e212255f92e5436d8b.cloudfront.net (CloudFront)
x-amz-cf-id: a7ha0wB_cyWO77fmoXNtxiMLOX-3ufFU0McgpWwkoYpzFlCMB6CmCw==
x-amz-cf-pop: DFW3-C1
x-cache: Miss from cloudfront

I am noticing that each time I get a miss the x-amz-cf-pop header changes.

Cannon Moyer
  • 3,014
  • 3
  • 31
  • 75
  • In the Behaviors section, there is a parameter called Cache Based on Selected Request Headers, is that set to "None"? – Nidhi Jan 24 '20 at 18:19
  • 1
    Minimum TTL [is not actually a "minimum TTL"](https://stackoverflow.com/a/26638663/1695906) in the sense you're thinking -- it's the minimum *boundary* for TTL, used to direct CloudFront to pretend, for its own internal purposes, that `max-age` and/or `s-maxage` values from the origin that are smaller than this value were actually set to this value, permitting (but not requiring) CloudFront to cache objects for longer than the origin actually specifies, without revalidating them at the origin. – Michael - sqlbot Jan 24 '20 at 20:22
  • @nidhi I added a picture of my behavior config. – Cannon Moyer Jan 24 '20 at 20:27
  • Your hit ratio does seem unusually low (though I've typically used CloudFront with larger traffic volumes), but there's nothing obviously amiss if these headers are representative. When testing, though, check the `X-Amz-Cf-Pop` response header. This identifies the specific global edge location handling your request, and they don't typically share caches (though nearby edges sometimes cascade through a common upstream regional cache). If you are hitting different edges, the hit/miss for you doesn't prove quite as much as if you repeatedly hit the same edge. – Michael - sqlbot Jan 24 '20 at 20:36
  • As you are using the setting `Use Origin Cache Headers`, a TTL of 1 year will be applied for all assets that follow this behavior. So, the other settings i.e. Min, Max and Default TTL won't be applied. Could you also paste the response headers from your Cloudfront distribution? – Nidhi Jan 25 '20 at 00:02
  • @nidhi I have tried both. Currently I have that turned on. When I originally asked the question I had a custom min, max and default TTL set. I have added the my response headers from Cloudfront. – Cannon Moyer Jan 25 '20 at 00:10
  • @Michael-sqlbot I am noticing that each time I get a cache miss the header you mentioned has changed. I have seen 4 different values for this header in the past 20 minutes. – Cannon Moyer Jan 25 '20 at 00:43
  • @CannonMoyer Did you find any solution for this? – Mehdi Khlifi Sep 30 '20 at 10:23
  • @MehdiKhlifi Yes and no. I concluded that files accessed less frequently were purged from the CDN after some time (10 minutes or so) regardless of the settings (Amazon said it might do this) and files with higher request rates were persisted longer. I was able to prove this by creating another CDN and placing files with higher request rates on that CDN and my hit rate went to 98% (same config). I also tested Fastly and I was able to get everything working as expected there. It seems that Amazon purges those lesser used files which are requested a few times per day regardless of the settings. – Cannon Moyer Sep 30 '20 at 13:44
  • @MehdiKhlifi If you're working on a site with lower traffic, I would recommend a completely different CDN. Cloudfront seems to do well on larger scales. – Cannon Moyer Sep 30 '20 at 13:45

0 Answers0