9

I am using Amazon S3 for storing and retrieving images for an image storing website. The trouble is that multiple users have to retrieve same image multiple times.

Is it suggested to use Redis or memcached for caching image files by storing them directly onto it.

Amazon S3 pricing for data transfer is much higher than compared to serving images via Redis cache. But storing image files directly on Redis seems to be a bad proposition because I read somewhere that Redis is not good for operating on large data files. Also I don't understand that if Redis stores data on memory how will it store so many images(unless I make many many instances).

Is it advisable to store image files directly onto Redis or is there an alternate for caching these images?

Do pinterest and imgur use Redis and memcache for storing images directly? If not why do they have so many instances?Pinterest

Mayank
  • 1,364
  • 1
  • 15
  • 29
  • 1
    Storing images in Redis seems like a terrible idea since it will quickly fill up the available RAM on the Redis server. Also your statement that "S3 pricing for data transfer is much higher than compared to serving images via Redis" sounds incorrect to me. I think you are missing something there. The standard way to cache images is to use a CDN such as CloudFront, CloudFlare or MaxCDN. – Mark B Jun 07 '16 at 14:24
  • @MarkB I looked into CloudFront and I found that pricing for it is ($0.85 - $0.25) which is more if not same for S3 pricing. While AWS says - "There is no Amazon ElastiCache Data Transfer charge for traffic in or out of the Amazon ElastiCache Node itself.". [Here](https://aws.amazon.com/elasticache/pricing/) – Mayank Jun 07 '16 at 16:49
  • @MarkB Also can you tell me if not for caching images directly, what does pinterest and imgur use Redis and Memcached for? – Mayank Jun 07 '16 at 16:51
  • Redis and Memcached are usually used for caching database data. I can't tell you exactly what they are using it for because I don't work for them, so how would I know exactly? – Mark B Jun 07 '16 at 17:14
  • *The trouble is that multiple users have to retrieve same image multiple times.* Incorrect Cache-Control headers? Why do multiple users have to retrieve the same image multiple times? – Michael - sqlbot Jun 07 '16 at 23:09

1 Answers1

13

You get credit for creativity, but you have not found a loophole, here.

First, it's entirely inappropriate to try to serve images from elasticache. It's a cache. It's volatile by definition.

Second, it's not a web server.

Third, it's not intended to be exposed to the Internet.

But even if these aren't persuasive, your question seems premised on a misunderstanding of the pricing structure on several levels.

There is no Amazon ElastiCache Data Transfer charge for traffic in or out of the Amazon ElastiCache Node itself.

https://aws.amazon.com/elasticache/pricing/

Technically, this is accurate, but it is not helpful.

This is only relevant to the transfer from elasticache to your EC2 instance. You still have to return the data to the browser, across the Internet, and this costs the same, whether you return it from/through EC2, or from S3.

Data Transfer OUT From Amazon EC2 To Internet

Up to 10 TB / month $0.09 per GB

https://aws.amazon.com/ec2/pricing/

...or...

Data Transfer OUT From Amazon S3 To Internet

Up to 10 TB / month $0.090 per GB

https://aws.amazon.com/ec2/pricing/

Meanwhile, CloudFront is $0.085/GB for traffic sent to browsers that are accessing edge locations in the lowest price class, US and Europe. And you control which edge locations are available when you select a price class other than the global one:

If you choose a price class that does not include all edge locations ... you're charged the rate for the least expensive region in your selected price class.

http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/PriceClass.html

That's $0.085 if configured correctly.

There is no charge for transfer from S3 to CloudFront or from EC2 to CloudFront. Only the charge from CloudFront to the Internet.

Michael - sqlbot
  • 169,571
  • 25
  • 353
  • 427