2

I have a small ASP.NET Core site that I am hosting in AWS lambda. I have a simple Excel Workbook upload that is functioning correctly when hosted in an IIS Express instance (running from Visual Studio locally) but once my site is published to AWS Lambda, the binary file data is being corrupted. My code is below:

Upload.cshtml

<h3>Please pick the workbook to upload...</h3>
<div class="row">
    <div class="col-md-4">
        <form method="post" enctype="multipart/form-data">
            <div class="form-group">
                <label asp-for="WorkbookFile" class="control-label"></label>
                <input asp-for="WorkbookFile" type="file" class="form-control" style="height:auto" />
                <span asp-validation-for="WorkbookFile" class="text-danger"></span>
            </div>

            <input type="submit" asp-page-handler="Preview" value="Preview Upload" class="btn btn-default" />
        </form>
    </div>
</div>

Upload.cshtml.cs

    [BindProperty]
    [Required]
    [Display(Name = "Workbook")]
    public IFormFile WorkbookFile { get; set; }


     public async Task<IActionResult> OnPostPreview()
     {
            // Perform an initial check to catch FileUpload class
            // attribute violations.
            if (!ModelState.IsValid)
            {
                return Page();
            }
            byte[] fileContents;
            Guid workbookId;
            using (var memoryStream = new MemoryStream())
            {
                await WorkbookFile.OpenReadStream().CopyToAsync(memoryStream);
                fileContents = memoryStream.ToArray();
                workbookId = SaveWorkbook(fileContents);
            }

            return RedirectToPage("./UploadReview", new { id = workbookId });
        }

As is outlined by the AWS documentation (https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-payload-encodings.html) I have added "multipart/form-data" to my Api Gateway's "binary" types list but this does not seem to be having any effect.

API Gateway Settings Screenshot

Based on everything I can find this should allow my uploaded files to be passed as binary straight to my service, however the resulting files are 2x the size of the files when uploaded locally, so I am assuming the base64 encoding is still being applied.

All of the Request headers appear to either be multipart/form-data or application/octet-stream so I'm really at the end of my rope as I am neither a Web nor an AWS expert. Any suggestions would be greatly appreciated. Thanks!

EDIT 1. The picture (link) below shows log messages that write out the byte[] sizes I'm receiving in the ASP.NET core back-end when running local vs. running AWS.

Log messages with array sizes

The picture (link) below shows the Chrome dev tools showing my content-type is multipart/form-data and the length lines up with the expected size.

Chrome Dev Tools Information

jasonu
  • 61
  • 1
  • 6
  • 1
    where are you storing these files? S3? – Thales Minussi Mar 02 '19 at 23:25
  • I'm storing the byte[] in a SQL Server instance. Eventually I was thinking of storing the files in S3 but regardless of where my Page stores the file I think I need to get the byte[] "un-encoded" in my Page Model in order to be able to process it\return it back to the user at a late date. If there's an ASP.NET Core way to save it "direct" to S3 without going through the API Gateway I would definitely be up for trying it. Thanks! – jasonu Mar 03 '19 at 16:25
  • Base64 encoded bodies add an extra [33]% overhead to the original file, not 100%. However, that's only when receiving a request. Once a file (binary) has been saved successfully, it doesn't matter whether it was uploaded via base64 or binary as they should have the same size since the content is the same. I'd try converting your bytes stream to base64 (this is what I usually do to save on S3). In NodeJS it is like this: new Buffer(body.replace(/^data:image\/\w+;base64,/, ""),'base64'), use the C# equivalent and try it out. – Thales Minussi Mar 03 '19 at 18:23
  • please check this answer: https://stackoverflow.com/questions/54840576/aws-serverless-function-is-not-responding-with-an-image/54844893#54844893 the user was facing the same issue with API Gateway when uploading binary files. – Thales Minussi Mar 03 '19 at 18:26

1 Answers1

4

After scouring much of the internet, my problem was "Save" doesn't mean "Use". The blog below gave me the hint I needed:

https://github.com/krisgholson/serverless-thumbnail

After changing the API Gateway's "Settings" to include the multipart/form-data and clicking "Save Changes", the changes are SAVED but are not being USED. In order to USE the new settings you must go to the Resources tab, "Actions" drop-down > Deploy API. Once the API has been deployed now the Saved configuration will be Used (in my scenario the binary data gets passed as expected). Many Thanks!

jasonu
  • 61
  • 1
  • 6