What permissions would let me upload with a MultiPartUploadRequest but not with TransferUtility.Upload?

Question

I am trying to upload a file to an S3 bucket using AWSSDK.S3. I am trying to use the TransferUtility.UploadAsync() method, as this is what we are using to upload files to other buckets, using other AWS credentials. However, when I use that here I am getting AccessDenied.

            var credentials =
                new BasicAWSCredentials(accessKey, secretKey);
            var s3Client = new AmazonS3Client(credentials, RegionEndpoint.USEast1);

            // Initiate the upload.
            try
            {
                var transferUtility = new TransferUtility(s3Client);
                
                await transferUtility.UploadAsync(filePath, bucketName, keyName+"_2.mpg",
                    CancellationToken.None);
            }
            catch (Exception e)
            {
                Console.WriteLine(e);
            }

This get's AccessDenied.

However, if I attempt to use a MultiPartUpload, the file was successfully uploaded.

 var credentials =
                new BasicAWSCredentials(accessKey, secretKey);
            var s3Client = new AmazonS3Client(credentials, RegionEndpoint.USEast1);

            long _fileSizeMbGrowth = 0;
            long _fileSizeTotal = 0;
            DateTime _startTime = DateTime.Now;
            // Create list to store upload part responses.
            List<UploadPartResponse> uploadResponses = new List<UploadPartResponse>();
            
            // Setup information required to initiate the multipart upload.
            InitiateMultipartUploadRequest initiateRequest = new InitiateMultipartUploadRequest
            {
                BucketName = bucketName,
                Key = keyName+"_3.mpg",
                CannedACL = S3CannedACL.BucketOwnerFullControl
            };
            
            InitiateMultipartUploadResponse initResponse =
                 await s3Client.InitiateMultipartUploadAsync(initiateRequest, cancellationToken);

            // Upload parts.
            _fileSizeTotal = new FileInfo(filePath).Length;
            long _fileSizeTotalDisplay = _fileSizeTotal / 1048576;
            string _fileName = new FileInfo(filePath).Name;
            long partSize = 5 * (long)Math.Pow(2, 20); // 5 MB
            
            try
            {
                string hashMD5;
                Console.WriteLine("Uploading parts");
#pragma warning disable SCS0006 // Weak hashing function
                using (var md5 = MD5.Create())
#pragma warning restore SCS0006 // Weak hashing function
                {
                    using (var stream = File.OpenRead(filePath))
                    {
                        hashMD5 = Convert.ToBase64String(md5.ComputeHash(stream));
                    }
                }
                long filePosition = 0;
                for (int i = 1; filePosition < _fileSizeTotal; i++)
                {
                    UploadPartRequest uploadRequest = new UploadPartRequest
                    {
                        BucketName = bucketName,
                        Key = keyName+"_3.mpg",
                        UploadId = initResponse.UploadId,
                        PartNumber = i,
                        PartSize = partSize,
                        FilePosition = filePosition,
                        FilePath = filePath,
                        ServerSideEncryptionCustomerProvidedKeyMD5 = hashMD5
                    };
                    
                    uploadResponses.Add(await s3Client.UploadPartAsync(uploadRequest, cancellationToken));

                    filePosition += partSize;
                }

                // Setup to complete the upload.
                CompleteMultipartUploadRequest completeRequest = new CompleteMultipartUploadRequest
                {
                    BucketName = bucketName,
                    Key = keyName,
                    UploadId = initResponse.UploadId
                };
                completeRequest.AddPartETags(uploadResponses);

                // Complete the upload.
                CompleteMultipartUploadResponse completeUploadResponse =
                    await s3Client.CompleteMultipartUploadAsync(completeRequest, cancellationToken);

                Console.WriteLine($" : Completed in {DateTime.Now.Subtract(_startTime).TotalSeconds} Second(s)");
            }
            catch (Exception exception)
            {
                Console.WriteLine("An AmazonS3Exception was thrown: { 0}", exception.Message);

                // Abort the upload.
                AbortMultipartUploadRequest abortMPURequest = new AbortMultipartUploadRequest
                {
                    BucketName = bucketName,
                    Key = keyName,
                    UploadId = initResponse.UploadId
                };
                await s3Client.AbortMultipartUploadAsync(abortMPURequest, cancellationToken);
            }

Is there a bucket policy, or set of access permissions that would allow a multipartupload request but not a PutObject request?

score 0 · Answer 1 · answered Mar 24 '22 at 20:59

In this case I ended up asking the wrong question here. It isn't an issue of access permissions or bucket policy, but that since we are uploading to someone else's bucket we need to set a ACL.

            try
            {
                var transferUtility = new TransferUtility(s3Client);

                var transferUtilityRequest = new TransferUtilityUploadRequest()
                {
                    BucketName = bucketName,
                    Key = keyName+"_5.mpg",
                    FilePath = filePath,
                    **CannedACL = S3CannedACL.BucketOwnerFullControl**
                };

                await transferUtility.UploadAsync(transferUtilityRequest, cancellationToken);
            }
            catch (Exception e)
            {
                Console.WriteLine(e);
            }

Adding the CannedACL allowed the upload request to work.

What permissions would let me upload with a MultiPartUploadRequest but not with TransferUtility.Upload?

1 Answers1