Streaming Multiple files as zip stream on .net6.0 api fails on AWS EBS with Load Balancer

Question

Have created a .net (6.0) api to read files from Amazon S3 Bucket and zip files on fly and send it to the client as download stream via ZipStream.

We have made use of the following method to zip files on fly. Reference : Blog

[HttpGet]
        public async Task ZipAFileS3()
        {
            var s3Keys = this.ListFiles();
            var bucketname = "myBucket";

            Response.ContentType = "application/octet-stream";
            Response.Headers.Add("Content-Disposition", "attachment; filename=\"files.zip\"");

            using (var s3 = CreateS3Client())
            {
                using (var archive = new ZipArchive(Response.BodyWriter.AsStream(), ZipArchiveMode.Create))
                {
                    foreach (var s3Key in s3Keys)
                    {
                        var entry = archive.CreateEntry(Path.GetFileName(s3Key), CompressionLevel.NoCompression);
                        using (var entryStream = entry.Open())
                        {
                            var request = new GetObjectRequest { BucketName = bucketname, Key = s3Key };
                            using (var getObjectResponse = await s3.GetObjectAsync(request))
                            {
                                await getObjectResponse.ResponseStream.CopyToAsync(entryStream);
                            }
                        }
                    }
                }
            }
        }


    public static AmazonS3Client CreateS3Client()
        {
                var AWSAccessKeyId = "";
                var AWSSecretAccessKey = "";
                var Token = "";
                var awsCredentials = new SessionAWSCredentials(AWSAccessKeyId, AWSSecretAccessKey, Token);
                return new AmazonS3Client(awsCredentials, RegionEndpoint.EUWest1);
        }

This api works efficiently on IIS and Nginx Server while testing on Local and Elastic Beanstalk with Single Instance.

While the api fails when Elastic beanstalk is configured with Load Balancer. The stream closes returning Failed - Network error while the zip file is being transmitted.

I have also checked logs on Cloudwatch to verify if the error is from the application side. I see that the file transmission is happening even after the connection to the client is closed.

Not sure, what is causing the issue.

Proxy Server : Nginx

Following is the Nginx configuration, which is modified whenever a new instance is created.

user                    nginx;
error_log               /var/log/nginx/error.log warn;
pid                     /var/run/nginx.pid;
worker_processes        auto;
worker_rlimit_nofile    32368;

events {
    worker_connections  1024;
}

http {
    server_tokens off;

    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    include       conf.d/*.conf;

    map $http_upgrade $connection_upgrade {
        default     "upgrade";
    }

    server {
        listen        80 default_server;
        access_log    /var/log/nginx/access.log main;

        client_header_timeout 864000;
        client_body_timeout   864000;
        keepalive_timeout     864000;
        gzip                  on;
        gzip_comp_level       4;
        gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;

        add_header Connection "keep-alive";
        add_header Accept-Encoding gzip;
        add_header Accept-Ranges "bytes";
        add_header Cache-Control "private, no-transform, no-store";
        add_header Content-Transfer-Encoding "binary";
        
        

        # Include the Elastic Beanstalk generated locations
        include conf.d/elasticbeanstalk/*.conf;
    }
}