9

Using multipart uploads, Amazon S3 retains all the parts until the upload is either completed or aborted. In an anonymous drop situation, it would be good for abandoned uploads to be automatically aborted after a timeout to reclaim the space and avoid the cost of holding any parts that made it.

It would be possible to create some external monitor using ListMultipartUploads, but it would be better if S3 did it automatically.

If you initiate an upload and maybe upload some parts, but then do nothing further, will S3 eventually abort it for the bucket owner?

Alister Lee
  • 2,236
  • 18
  • 21
  • I was stuck with a bunch of incompletes that were tricky to delete (regular SDK scripts didn't work) came up with this solution http://stackoverflow.com/questions/39457458/howto-abort-all-incomplete-multipart-uploads-for-a-bucket – Refael Ackermann Sep 12 '16 at 19:38

2 Answers2

8

You can set this here:

https://media.amazonwebservices.com/blog/2016/con_s3_life_multi_clean_1.png

https://aws.amazon.com/blogs/aws/s3-lifecycle-management-update-support-for-multipart-uploads-and-delete-markers/

and say afer 7 days or so, just delete them.

Andrew Arrow
  • 4,248
  • 9
  • 53
  • 80
  • Andrew one question: If today(13 August) I set multipart cleanup rule (7 days). Then will this rule remove the object that is created before 7 days (1 August)? – Himanshu Shekhar Aug 13 '20 at 10:35
  • 1
    my guess is, only uploads started after you set the cleanup rule will be affected. But not 100% sure, u'd have to test both cases. – Andrew Arrow Aug 13 '20 at 14:04
4

Note the newer setting mentioned in Andrew's answer

Original response:

No. From the doc page you linked, "Once you initiate a multipart upload, Amazon S3 retains all the parts until you either complete or abort the upload."

If you do not either complete or abort the upload, any parts that have been uploaded will just kinda hang around and cause you storage charges until you do so.

Josh Hancock
  • 795
  • 10
  • 15
  • I agree the documentation doesn't indicate that it aborts automatically. However, that's probably the normal case they are documenting. I'm interested in S3's behaviour in an extraordinary case. For a final answer, this probably needs either an experiment or a comment from an AWS developer. A stackoverflow user with premium support could raise a support request with amazon if this issue concerned them enough. – Alister Lee Apr 30 '15 at 03:07
  • 1
    I'll provide you with another piece of documentation, if you like: http://docs.aws.amazon.com/AmazonS3/latest/dev/uploadobjusingmpu.html - "Once you initiate a multipart upload there is no expiry; you must explicitly complete or abort the multipart upload" – Josh Hancock Apr 30 '15 at 21:06