198

I'm getting this error intermittently.

I have a program that uses the java aws sdk and loads over the 10s of thousands of small files to s3. I see this error intermittently.

Could not find any helpful answer after doing a quick search on the internet.

Note the calling program is single threaded. The underlying aws java sdk does seem to use worker threads.

Status Code: 409, AWS Service: Amazon S3, AWS Request ID: 75E16E8DE2193CA6, AWS Error Code: OperationAborted, AWS Error Message: A conflicting conditional operation is currently in progress against this resource. Please try again., S3 Extended Request ID: 0uquw2YEoFamLldm+c/p412Lzd8jHJGFBDz3h7wN+/4I0f6hnGLkPMe+5LZazKnZ
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:552)
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:289)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:170)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:2648)
    at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:578)
    at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:503)
Bhavik Ambani
  • 6,557
  • 14
  • 55
  • 86
user1172468
  • 5,306
  • 6
  • 35
  • 62
  • 1
    Are you trying to put objects into S3 and then immediately read them? – jamieb Dec 16 '12 at 02:31
  • @jamieb, actually no, not at this time. – user1172468 Dec 16 '12 at 03:27
  • 1
    I wish I had an answer for you. Are you trying to rewrite the same keys in rapid succession? S3 is built on an "eventually consistent" model where PUTs need time to settle. – jamieb Dec 16 '12 at 05:58
  • @jamieb, thanks for spending brain cycles but the answer provided by Guy is correct I think. I think in my code I retry to create a bucket -- expecting to fail gracefully -- it does like 99.99% of the time and sometimes it throws this error. Again really appreciate you taking the time to help me out. Cheer! – user1172468 Dec 17 '12 at 07:46
  • in NodeJS, my code was wrapped in a createBucket call as just part of the cycle -- expecting a graceful fail as well. I quickly started getting this error. If you must check if the bucket is created, do it once (maybe on server start). – Senica Gonzalez Aug 10 '17 at 14:45
  • I had the same issues. To be fair, I had more luck when deleting and quickly creating a new bucket if server-side encryption was off, in that case I could create a new one just a couple of minutes after. – Francesco Jan 21 '20 at 21:33

10 Answers10

502

I got the same error message, when I did following:

  1. created a bucket - it went by default to US region (used AWSCLI)

  2. realized, the bucket shall go to EU region and deleted it (used AWS console)

  3. (few minutes later) tried to create the bucket, specifying the EU region

At step 3, AWS console has shown me the error message from title of your question.

So I guess, that the bucket in US was deleted, but there are possibly some synchronization processes, which are taking time. And I hope, that waiting few hours I will find the bucket name again available for creation - this time in proper (EU) region.

FIX :- Edit: About an hour later, my attempt to create the bucket (in EU region) succeeded.

Faisal Mansoor
  • 2,041
  • 1
  • 21
  • 25
Jan Vlcinsky
  • 42,725
  • 12
  • 101
  • 98
  • 4
    I just ran into the same situation. I'll post back if I can create my bucket within the next hour or so. – AJB May 17 '14 at 19:42
  • 32
    I can confirm this. It took about 1.5hrs, and there's no guarantee someone else won't scoop the bucket name in the meantime, but you can get it back it seems within a reasonable period of time, and with a little luck. – AJB May 17 '14 at 21:49
  • Just ran into the same problem. Moved the bucket once, that worked. Tried it again, now it locks me out of doing so with the above message. Not impressed... – Daniel Nov 19 '15 at 17:10
  • 1
    I agree with @jan To solve this: Create a new bucket with new name, Don't use the name of the deleted bucket while naming your new bucket. – Dinesh Sunny May 08 '16 at 20:01
  • 6
    For me it took less than 70 minutes. – offby1 Jul 16 '16 at 19:54
  • 5
    Amazon says that it could take anywhere up to 10hrs https://forums.aws.amazon.com/thread.jspa?threadID=37532 – TomDotTom Mar 06 '17 at 15:59
  • Your code should not depend on the availability of a bucket name and should handle these situations (by creating random prefix or suffix bucket names). – Kamran Bigdely Dec 28 '17 at 17:22
  • Same exact issue. Very concerned for a second there that someone had that bucket name in the EU. – Matthieu McLaren Apr 11 '18 at 12:19
  • For me it didnt allow me to create a bucket name with additional suffix of a number at the end too. Not sure why that happened. – Nirojan Selvanathan Sep 28 '18 at 04:15
  • @NirojanSelvanathan Can you provide example of original and suffixed bucket name? To me it seems this is completely different issue. – Jan Vlcinsky Sep 30 '18 at 16:08
  • this is a bit ridiculous because to host a static site you'd use the public domain, which could be squatted! – scape May 07 '19 at 22:17
  • on the same boat! seriously ? Why it can't be a simple broadcast push action based on delete? – kahmed May 11 '19 at 02:26
  • creating same bucket in same region is quick in case someone is stucked with the error – Ajaya Mandal May 31 '20 at 17:09
  • I guess this is due to s3 replicas across the AZs for that particular region and it takes time to reflect changes in all replicas – Jeremias Moraes Oct 02 '20 at 21:22
44

For all others who stumble upon this thread from google, as 1st result in search for this error message:

If you deleted bucket, to recreate in new region, do not wait "manually" until this background sync will be complete, instead put a small bash script to run and retry your needed bucket creation every 5 seconds or so.

Example:

#!/bin/bash 
RESULT=2 
until [  $RESULT -eq 0 ]; do
    aws s3 mb s3://your.bucket.name --region us-west-2
    RESULT=$?
    sleep 5 
done 
echo "Bucket created!"

it will retry the "create bucket" operation for you, every few seconds (depend on 'sleep' ) and as soon as it's possible - will create it for you, so no one can steal your bucket name by mistake :)

hope it helps :)

Dmitry Shmakov
  • 738
  • 7
  • 9
  • 2
    Thanks for this! In my instance, I already had the region defined in the aws configuration and running the script as is resulted in the following error: `UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 17: ordinal not in range(128)`. The solution was to remove the `--region` flag. – MagentoAaron Apr 10 '19 at 14:17
  • Horosho!!!!!!!!! – Kaguei Nakueka Dec 15 '21 at 16:43
  • 1
    Simpler :-) `until aws s3 mb aws s3 mb s3://your.bucket.name --region us-west-2; do echo "Trying again.."; sleep 5; done` – James Andres Mar 15 '22 at 11:51
23

The request in your example is to create a bucket. If you are trying to create too many buckets or replace buckets, it is not healthy.

Note that you have a limit of 100 buckets for an account (see here). EDIT: Now this limit is a "soft limit" and you can increase it if needed.

Also note that a creation of a bucket takes time and:

...it is not appropriate to make bucket create or delete calls on the high availability code path of your application...

It is better to create your buckets once and then, you can put as many objects that you like in your existing buckets (or even a single one).

Anyway, when working with a system that is bound to fail any now and then, you should be able to handle errors and slow down your process when receiving such an error. See more details in AWS Docs.

Guy
  • 12,388
  • 3
  • 45
  • 67
13

This error usually occurs when a Bucket is deleted and a new bucket is created in the same name as the old bucket.

I believe we would need to wait for certain amount of time until we can create a new bucket in the same name.

Srivignesh KN
  • 452
  • 8
  • 22
10

Simple and straightforward: Change the new bucket name or wait 1 hour.

sigmaxf
  • 7,998
  • 15
  • 65
  • 125
6

Finally, AWS now has a clear answer in their own knowledge center. See this: Why am I getting the error "A conflicting conditional operation is currently in progress against this resource" from Amazon S3 when I try to re-create a bucket?.

I've just deleted a bucket (since S3 bucket names are globally unique and I need to create the same bucket in another AWS account) and my script that creates a bucket with the same name fails with the same error. So I have to be patient and wait for an hour.

Just as a confirmation: After around one hour, I can successfully create a bucket with the same name as the deleted bucket.

Abdullah Khawer
  • 4,461
  • 4
  • 29
  • 66
Dmitriy Popov
  • 2,150
  • 3
  • 25
  • 34
1

You also get this message when you have more than 100 buckets in the account already. There is a soft limit of 100 buckets.

John Mee
  • 50,179
  • 34
  • 152
  • 186
0

Change your bucket name or wait. I waited less than 30 minutes for mine. I did not need to change the name and I couldn't change it as I was set on it - it had to match my already existing domain name for a static website that I was looking to host. Anyway, in less than 30 minutes it was created automatically.

vel12171
  • 21
  • 2
0

I got the error running a Cloud Formation template.

Turns out the IAM role the EC2 assumed that I was running the CFN template on didn't have permission to create S3 buckets.

Either provide IAM access, or use a different IAM role by doing an AWS Configure or programmatically assume a role with necessary permission to create an S3 Bucket.

Jeremy Thompson
  • 61,933
  • 36
  • 195
  • 321
0

I ran into this. I created a bucket and then realized I created the bucket in the wrong region: us-west-2

However, it should have been in us-east-1. I deleted the errored bucket and then tried to recreate it in the correct region.

This command runs the bucket create command every 10 seconds (MacOS or Linux). It took a little over an hour to succeed.

watch -n 10 aws s3api create-bucket --bucket com.xenial.deploy.uat-us-east-1 --region us-east-1
William Crossland
  • 1