How do I make sure my users are downloading the new version of my S3 File?

Question

This is inside bash file:

s3cmd --add-header='Content-Encoding':'gzip' put /home/media/main.js s3://myproject/media/main.js

This is what I do to upload my backbone compressed file into Amazon S3. I run this command every time I make changes to my javascript files.

However, when I refresh the page in Chrome, Chrome still uses the cached version.

Request headers:

Accept:*/*
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-US,en;q=0.8,es;q=0.6
AlexaToolbar-ALX_NS_PH:AlexaToolbar/alxg-3.3
Cache-Control:max-age=0
Connection:keep-alive
Host:myproject.s3.amazonaws.com
If-Modified-Since:Thu, 04 Dec 2014 09:21:46 GMT
If-None-Match:"5ecfa32f291330156189f17b8945a6e3"
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36

Response headers:

Accept-Ranges:bytes
Content-Encoding:gzip
Content-Length:70975
Content-Type:application/javascript
Date:Thu, 04 Dec 2014 09:50:06 GMT
ETag:"85041deb28328883dd88ff761b10ece4"
Last-Modified:Thu, 04 Dec 2014 09:50:01 GMT
Server:AmazonS3
x-amz-id-2:4fGKhKO8ZQowKIIFIMXgUo7OYEusZzSX4gXgp5cPzDyaUGcwY0h7BTAW4Xi4Gci0Pu2KXQ8=
x-amz-request-id:5374BDB48F85796

Notice that the Etag is different. I made changes to it, but when I refreshed the page, that's what I got. Chrome is still using my old file.

Answer 1

It looks like your script has been aggressively cached, either by Chrome itself or some other interim server.

If it's a js file called from a HTML page (which is sounds like it is), one technique I've seen is having the page add a parameter to the file:

<script src="/media/main.js?v=123"></script>

or

<script src="/media/main.js?v=2015-01-03_01"></script>

... which you change whenever the JS is updated (but will be ignored by the server). Neither the browser nor any interim caching servers will recognise it as the same and will therefore not attempt to use the cached version - even though on your S3 server it is still the same filename.

Whenever you do a release you can update this number/date/whatever, ideally automatically if the templating engine has access to the application's release number or id.

It's not the most elegant solution but it is useful to have around if ever you find you have used an optimistically long cache duration.

Obviously, this only works if you have uploaded the new file correctly to S3 and S3 is genuinely sending out the new version of the file. Try using a command-line utility like curl or wget on the url of the javascript to check this is the case if you have any doubts about this.

Answer 2

The Invalidation Method

s3cmd -P --cf-invalidate put /home/media/main.js s3://myproject/media/main.js
       |    |
       |    Invalidate the uploaded filed in CloudFront.               
       |
      -P, --acl-public / Store objects with ACL allowing read for anyone.

This will invalidate the cache for the file you specify. It's also possible to invalidate your entire site, however, the command above shows what I would imagine you'd want in this scenario.

Note: The first 1000 requests/month are free. After that it's approximately $0.005 per file, so if you do a large number of invalidation requests this might be a concern.

The Query String / Object Key Method

CloudFront includes the query string (on origin) from the given URL when caching the object. What this means is that even if you have the same exact object duplicated, but the query strings are different, then each one will be cached as a different object. In order for this to work properly you'll need to select Yes for Forward Query Strings in the CloudFront console or specify true for the value of the QueryString element in the DistributionConfig complex type when you're using the CloudFront API.

Example:

http://myproject/media/main.js?parameter1=a

Summary:

The most convenient method of ensuring the object being served is the current would be invalidation, although if you don't mind managing the query string parameters then you should find it just as effective. Adjusting the headers won't be nearly as reliable as either method above in my opinion; clients handle caching differently in too many ways that it's not easy to distinguish where caching issues might be.

Answer 3

You need the response from S3 to include the Cache-Control header. You can set this when uploading the file:

s3cmd --add-header="cache-control:max-age=0,no-cache" put file s3://your_bucket/

The lack of whitespace and uppercase in my example is due to some odd signature issue with s3cmd. Your mileage may vary.

After updating the file with that command, you should get the Cache-Control header in the S3 response.