0

I have a UWP app that Upload a file to an S3 Bucket using the AWS SDK for .NET (high-level API) that is TransferUtility.
This API works very well when I am uploading small size recording files but when I upload larger files the file does not seem to be uploaded and it throws following errors: An error occurred while sending the request. // in some tests A task was canceled.

To fix this error, I referred multiple resources from the internet but could not get the solution. I also tried adding different parameters to the TransferUtilityUploadRequest i.e PartSize to determine the size of the multipart segment, added TransferUtilityConfig as an additional parameter to transferUtility, Configured various RegionEndPoints used while sending upload request. Adding these changes did not make any difference while uploading large files to s3 bucket.

Below is the code snippet I am using

 int MB_SIZE = (int)Math.Pow(2, 20);

 var s3Client = new AmazonS3Client(objUploadRecording.AccessKeyId, objUploadRecording.SecretAccessKey,objUploadRecording.Token, region);

 var transferUtilityConfig = new TransferUtilityConfig
  {
     // Use 5 concurrent requests.
      ConcurrentServiceRequests = 5,

       // Use multipart upload for file size greater 1 MB.
        MinSizeBeforePartUpload = 1 * MB_SIZE,
      };
   using (var transferUtility = new TransferUtility(s3Client, transferUtilityConfig))
    {
      var uploadRequest = new TransferUtilityUploadRequest
       {
        BucketName = objUploadRecording.BucketName,
        Key = objUploadRecording.S3Filename,
        FilePath = objUploadRecording.FilePath,
        // Set size of each part for multipart upload to 1 MB
        PartSize = 1 * MB_SIZE
         };
        uploadRequest.UploadProgressEvent += OnUploadProgressEvent;
         CancellationToken cancellationToken = default(CancellationToken);
        try
         {
          await transferUtility.UploadAsync(uploadRequest, ancellationToken);
         }
        catch (Exception ex)
        {                                                    
         LoggingServices.Instance.WriteLine<RecordingListPage>("AWS Upload         
         Error: " + ex.Message.ToString(), MetroLog.LogLevel.Error);
        }

Can anyone please guide me where I am doing mistake or what additional parameters I will have to add here to make this work?

Shakita
  • 99
  • 2
  • 12
  • Honestly I think you'll have better luck finding an answer to your question on the [aws-sdk-net GitHub](https://github.com/aws/aws-sdk-net/issues). Every time I get an issue with that SDK, I don't get any help on StackOverflow but their GitHub is quiet active – iam.Carrot May 04 '20 at 17:16
  • Out of curiosity what is the behavior when you set ConcurrentServiceRequests to 1? – Norm Johanson May 05 '20 at 06:38
  • Not a C# guy, but my approach for debugging this would be by probing into a few questions like - Can I increase the logging level to get more info on error? Do I already have the request-id which I can search in Cloud Trail? What is the exact size from which I start to get error (may be find using Binary Search)? Is there an API Gateway in middle which causes this issue? – Siddharth May 07 '20 at 03:17
  • Does it work manually? Any file over 80MB-100MB is supposed to be uploaded using MultiPart and 1MB seems a bit small. Even the documentation states 16MB is the default, please follow their instructions https://docs.aws.amazon.com/mobile/sdkforxamarin/developerguide/s3-integration-transferutility.html – Jeremy Thompson May 08 '20 at 03:32

1 Answers1

2

Thanks all, as I mentioned I was using aws High level SDK to upload a file but it was not working. Also, I have not found any other implementation or solution that would get rid of the issues causing in my application Therefore, I have used a low level API here

and this uploads file successfully. There are drawback like it uploads file part by part but thats fine for me.

Community
  • 1
  • 1
Shakita
  • 99
  • 2
  • 12