Powershell is unable to upload zip file over ~50MB using Multipart form-data to REST API

Question

I've got a REST API endpoint in .net core 3.1 that looks like this:

        [Route("Install")]
        [HttpPost]
        [RequestSizeLimit(524288000)]
        public async Task<IActionResult> AddInstallFile(List<IFormFile> files)

I can upload a file using cURL with this command:

curl -X POST MYURL Content-Type: multipart/form-data" -F "files=@filename.zip;type=application/x-zip-compressed"

It also works via swagger

In powershell though Multipart/form-data is not natively supported so I'm using the following code to upload there (essentially I used the working swagger upload with 'Copy as Powershell' function in chrome dev tools to create this code):

$fileBytes = [System.IO.File]::ReadAllBytes("C:\Temp\filename.zip");
$enc = [System.Text.Encoding]::GetEncoding('utf-8')
$fileEnc = $enc.GetString($fileBytes)
    

Invoke-WebRequest -Uri "MYURL" `
-Method "POST" `
-Headers @{
 "method"="POST"
 "accept"="application/octet-stream"
 "accept-encoding"="gzip, deflate, br"
} `
-ContentType "multipart/form-data; boundary=----WebKitFormBoundaryT2XycANuthCIUwGk" `
-Body ([System.Text.Encoding]::UTF8.GetBytes("------WebKitFormBoundaryT2XycANuthCIUwGk$([char]13)$([char]10)Content-Disposition: form-data; name=`"files`"; filename=`"filename.zip`"$([char]13)$([char]10)Content-Type: application/x-zip-compressed$([char]13)$([char]10)$([char]13)$([char]10)$fileEnc$([char]13)$([char]10)------WebKitFormBoundaryT2XycANuthCIUwGk--$([char]13)$([char]10)"))

This Powershell code works fine for a zip file that I have that is under 2MB but fails for another zip file that is over 80MB

Error on the server is here:

An unhandled exception has occurred while executing the request.

Exception: 
System.ArgumentNullException: Value cannot be null. (Parameter 'source')
   at System.Linq.ThrowHelper.ThrowArgumentNullException(ExceptionArgument argument)
   at System.Linq.Enumerable.Sum[TSource](IEnumerable`1 source, Func`2 selector)
   at netproject.ControllerClass.AddInstallFile(List`1 files) in ControllerClass.cs:line 44

Essentially the files variable is coming through as null when called from powershell for a file over ~50MB. I've tried both Invoke-WebRequest and Invoke-RestMethod with the same results

Did you check: [Upload BIG files via HTTP](https://stackoverflow.com/a/33253233/1701026)? — iRon, Sep 03 '21 at 07:01
Shouldn't be necessary for a file under 100MB. They're talking about files in the Gigabytes. Starting to think I've found a PowerShell bug to be honest. The fact that the code works for files under 50MB, then inexplicably breaks for files over about 80MB is very strange. I've managed to use Postman, swagger, cURL and a web form now, just not powershell. — Ralph, Sep 03 '21 at 09:55
I've run some experiments and it starts failing at around 80MB. The only difference is the actual file. I've used the same filenames and paths for tests — Ralph, Sep 03 '21 at 09:57

score 0 · Accepted Answer · answered Sep 08 '21 at 05:44

For anyone who comes across this in the future - if you get a working solution in powershell I would still be interested in hearing about it. To work around this for the moment I've used the .net framework to upload the file instead. Code is here:

    Write-Host "Uploading $name"
    $url = "URL_TO_UPLOAD"
    $file = "FULL_PATH_ZIP_FILE"

    $fs = [System.IO.FileStream]::New($file, [System.IO.FileMode]::Open)
    $f1 = New-Object System.Net.Http.StreamContent $fs

    $client = New-Object System.Net.Http.HttpClient 
    $client.DefaultRequestHeaders.ConnectionClose = $true 
    $client.DefaultRequestHeaders.Add("Authorization", $auth);

    $form = New-Object System.Net.Http.MultipartFormDataContent
    $form.Add($f1, 'files', $name)

    $rsp = $client.PostAsync($url, $form).Result
    Write-Host "Upload status: " + $rsp.StatusCode

    $fs.Close(); 
    $fs.Dispose()

Powershell is unable to upload zip file over ~50MB using Multipart form-data to REST API

1 Answers1