12

I'm using aws s3 SDK in my code. In my tests I'm trying to create/drop buckets for the test.

My sdk version is 1.11.800 :

<aws-s3-sdk.version>1.11.800</aws-s3-sdk.version>
<aws-java-sdk-core.version>1.11.800</aws-java-sdk-core.version>

The creation's code is exactly the same as in aws examples :

 public void createBucket(String bucketName){
        try {
            if (!s3Client.doesBucketExistV2(bucketName)) {
                // Because the CreateBucketRequest object doesn't specify a region, the
                // bucket is created in the region specified in the client.
                s3Client.createBucket(new CreateBucketRequest(bucketName));

                // Verify that the bucket was created by retrieving it and checking its location.
                String bucketLocation = s3Client.getBucketLocation(new GetBucketLocationRequest(bucketName));
                System.out.println("Bucket location: " + bucketLocation);
            }
        } catch (AmazonServiceException e) {
            // The call was transmitted successfully, but Amazon S3 couldn't process
            // it and returned an error response.
            e.printStackTrace();
        } catch (SdkClientException e) {
            // Amazon S3 couldn't be contacted for a response, or the client
            // couldn't parse the response from Amazon S3.
            e.printStackTrace();
        }
    }

The exception :

22:32:33.949 [main] DEBUG com.amazonaws.request - Received error response: com.amazonaws.services.s3.model.AmazonS3Exception: You have attempted to create more buckets than allowed (Service: Amazon S3; Status Code: 400; Error Code: TooManyBuckets; Request ID: 4FE57EE4ACC6C7E4; S3 Extended Request ID: qIzyYNg6Il9J+ehR+JM3W5+RZWg43GJIU9v4EA5ajHG05bUydw8lN3qLZY5u/QjdXhV8tJlINMk=; Proxy: null), S3 Extended Request ID: qIzyYNg6Il9J+ehR+JM3W5+RZWg43GJIU9v4EA5ajHG05bUydw8lN3qLZY5u/QjdXhV8tJlINMk=
com.amazonaws.services.s3.model.AmazonS3Exception: You have attempted to create more buckets than allowed (Service: Amazon S3; Status Code: 400; Error Code: TooManyBuckets; Request ID: 4FE57EE4ACC6C7E4; S3 Extended Request ID: qIzyYNg6Il9J+ehR+JM3W5+RZWg43GJIU9v4EA5ajHG05bUydw8lN3qLZY5u/QjdXhV8tJlINMk=; Proxy: null), S3 Extended Request ID: qIzyYNg6Il9J+ehR+JM3W5+RZWg43GJIU9v4EA5ajHG05bUydw8lN3qLZY5u/QjdXhV8tJlINMk=
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1811)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1395)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1371)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5062)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5008)
    at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:1097)

  • I got more than 100 buckets in my account but when I try to create a bucket in the aws console I create it successfully without any problems.

  • When I create the bucket manually and run my other tests (that don't create this bucket) I can successfully put and delete content from the bucket so it isn't a credentials issue.

halfer
  • 19,824
  • 17
  • 99
  • 186
JeyJ
  • 3,582
  • 4
  • 35
  • 83
  • Have you raised an increase request before as AWS only allows you to create up to 100 buckets in your account? – Nitesh Sharma Jun 16 '20 at 20:11
  • What does `aws service-quotas list-service-quotas --service-code s3` print ? – Ersoy Jun 16 '20 at 20:28
  • { "Quotas": [] } – JeyJ Jun 16 '20 at 20:29
  • @NiteshSharma I already have more than 100 buckets and I can keep creating them via the console. I cant create more via the SDK. – JeyJ Jun 16 '20 at 20:31
  • 1
    Perhaps AWS has a hard limit on the number of buckets created with code. Code can create dozens or hundreds of buckets at a time. You can only create one bucket at a time with the console. Reading @Gagan Bhat's answer, I suspect this is the case. – Gilbert Le Blanc Jun 16 '20 at 20:39
  • Take your app and the Java SDK out of the picture. Using the same credentials, can you create a new bucket using the awscli? – jarmod Jun 17 '20 at 00:30

1 Answers1

18

By default, an AWS account is soft limited to 100 S3 buckets.

You need to submit a service limit increase request by creating a new case through the support portal.. Once they override the default soft limit on your account you may create up to 1000 S3 buckets.

Read more about bucket restrictions here

Gagan Bhat
  • 352
  • 3
  • 11