2

From Request Rate and Performance Guidelines - Amazon Simple Storage Service:

Amazon S3 automatically scales to high request rates. For example, your application can achieve at least 3,500 PUT/POST/DELETE and 5,500 GET requests per second per prefix in a bucket. There are no limits to the number of prefixes in a bucket. It is simple to increase your read or write performance exponentially. For example, if you create 10 prefixes in an Amazon S3 bucket to parallelize reads, you could scale your read performance to 55,000 read requests per second.

Assume the S3 bucket with the folder s3bucket/sample/. There are multiple objects present in the folder. Example : s3bucket/sample/object_1, s3bucket/sample/object_2.

What does the prefix mean in this statement? Is it the full object path like s3bucket/sample or s3bucket/sample/object_1?

Is the 5500 requests per second for the folder - s3bucket/sample or 5500 requests are allowed for every object in the folder?

We have a multiple asset types belonging to one content, would like to understand which of the below 2 option will scale better?

Option 1

s3bucket/contentId_1/assetType_1
s3bucket/contentId_1/assetType_2
s3bucket/contentId_1/assetType_3
s3bucket/contentId_2/assetType_1
s3bucket/contentId_2/assetType_2
s3bucket/contentId_3/assetType_3

or

Option 2

s3bucket/contentId_1_assetType_1
s3bucket/contentId_1_assetType_2
s3bucket/contentId_1_assetType_3
s3bucket/contentId_2_assetType_1
s3bucket/contentId_2_assetType_2
s3bucket/contentId_3_assetType_3
John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Navin GV
  • 700
  • 3
  • 10
  • 24
  • This seems to be a duplicate of https://stackoverflow.com/questions/52443839/s3-what-exactly-is-a-prefix-and-what-ratelimits-apply. – ingomueller.net Oct 30 '19 at 09:37

1 Answers1

2

The page says requests per second per prefix in a bucket, which is effectively the same as saying "per directory per bucket".

Frankly, you are unlikely to go anywhere near these performance limits. Large companies with millions of customers might want to use these performance hints, but the vast majority of AWS customers would not approach such levels of usage.

I would suggest you deploy your data in the most meaningful way for your application rather than having to conform to these techniques, at least until you scale to very large usage patterns.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470