Why does NGINX authverify of multipart file upload request receive an immediate closed connection?

Question

NGINX serves our gateway on 8080, authenticating via our AAA server at 8100 and then passing authenticated requests to NGINX on 8081 for routing. Everything works fine straight out of the box with the exception of File Upload requests (of ANY size) directed at our File Server. Sent to the Gateway at 8080, a 500 Internal Server Error occurs when the connection to AAA at 8100 is peremptorily closed. Sent to 8081, NGINX routes the multipart File Upload content just fine.

Changing client_max_body_size does not help; nor does changing keep-alive values. What is the trick to perform the authverify step for multipart in NGINX?


server {
    listen 8080;
    listen [::]:8080;
    server_name _;
    access_log /var/log/nginx/api.auth.log;

    location = /authverify {
        internal;
        proxy_pass http://127.0.0.1:8081/api/v1/aaa/authentication/validate;
        proxy_pass_request_body off;
        proxy_set_header Content-Length "";
        proxy_set_header X-Original-URI $request_uri;
    }

    location /api/v1/aaa/authentication {
    proxy_pass http://127.0.0.1:8081;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /api {
        auth_request /authverify;
        auth_request_set $auth_header $upstream_http_authorization;
        more_set_headers "Authorization: $auth_header";

        proxy_pass http://127.0.0.1:8081;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Any help much appreciated! (We're running nginx 1.6.2 and have installed no extra modules.)

UPDATE: It appears that WEB API is unhappy about the request header still indicating "multipart" when the content has been suppressed. I had investigated suppressing the Content-Type in the auth request but had not discovered a satisfactory way to achieve it...

2019-11-22 05:20:41.896 +00:00 [Error] The server encountered an internal error and was unable to process your request. Try it again. System.IO.IOException: Unexpected end of Stream, the content may have already been read by another component. at Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken) at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool1 bytePool, Nullable1 limit, CancellationToken cancellationToken) at Microsoft.AspNetCore.WebUtilities.MultipartReader.ReadNextSectionAsync(CancellationToken cancellationToken) at Microsoft.AspNetCore.Http.Features.FormFeature.InnerReadFormAsync(CancellationToken cancellationToken) at Microsoft.AspNetCore.Mvc.ModelBinding.FormValueProviderFactory.AddValueProviderAsync(ValueProviderFactoryContext context) at Microsoft.AspNetCore.Mvc.ModelBinding.CompositeValueProvider.CreateAsync(ActionContext actionContext, IList`1 factories) at Microsoft.AspNetCore.Mvc.ModelBinding.CompositeValueProvider.CreateAsync(ControllerContext controllerContext) at Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<g__Bind|0>d.MoveNext()

Have you tried setting [error_log](http://nginx.org/en/docs/ngx_core_module.html#error_log)? It might tell you some more details... — boppy, Nov 21 '19 at 08:57
Thanks, yes, with 'debug' level. I'll add the clarification also that we, of course, had a breakpoint standing by in the AAA server for the request despatch, but something closed the connection even beforehand. I'll go back to the error log and copy the exact text (not much more than connection closed) so the Question posed is a kind of long-shot that there is some protocol we missed such as clearing the multi-part indication for the auth request... — koan911, Nov 21 '19 at 09:29
Just as an update, we have now tried this with NGINX v1.16.1 and the problem still exists. — koan911, Dec 10 '19 at 01:40

score 2 · Accepted Answer · answered Jun 16 '20 at 12:39

In my case, re-setting Content-Type fixed the issue:

    location = /authverify {
        internal;
        proxy_pass http://127.0.0.1:8081/api/v1/aaa/authentication/validate;
        proxy_pass_request_body off;
        proxy_set_header Content-Length "";
        proxy_set_header Content-Type "";
        proxy_set_header X-Original-URI $request_uri;
    }

But this might depend on how your API is handling the request.

Why does NGINX authverify of multipart file upload request receive an immediate closed connection?

1 Answers1