Getting "server keep idle connection" errors using MinIO with Go as I open many connections in a row, how can I solve the issue?

Question

I have a service which wants to open 28 connections to different S3 objects to save them there for later reuse.

At this time, I'm testing my code against localstack to make sure it will work and avoid potential confusion in our real S3 environment.

When I open up to about 8 connections in a raw, things work as expected. Nearly every time I go over that number, it starts creating "server keep idle connection" errors. The following is an example of the error in its entirety:

Put http://192.168.26.95:4572/bucket-name/test-12.mp3?partNumber=1&uploadId=ofutQ7QNwlpy0q4jYAofwhZQ130mMqk5hKkKsI2b50bBNHB1CfQI2IuUg: http: server closed idle connection

Just in case, I tried with one connection, left it open for some time without sending anything to it and it worked just fine. No "idle connection" errors. So I'm pretty sure the error label is wrong. It is more likely an error due to an overwhelmed listener (i.e. too many connections in a row, the connection pipe is full).

I have two main questions:

1. Is there a way to fix the problem on the client side? I tried to pace the connections, but even with a sleep of 1 second between each connection (which is way too long) I still get the error.

2. Would there be a way to setup localstack to allow for more connections to happen in a row? I'm using the Docker version.

For (1), I looked a bit in MinIO, the PutObject() does not seem to have any way to increase the timeout for a connection to succeed, or a number of repeat attempts, which would make my life easier.

For (2), it would need to match what a real S3 environment allows. So if the existing settings of localstack already match a real S3, I'm afraid we need to make (1) work.

---

The following is a sample which doesn't fail as bad as the main code, but with a count of 28, it still fails most of the time. The length of data or waiting before writing to the pipes or not doesn't seem to affect the PutObject() which will generate that "idle connection" error. The number of connections (calls to PutObject()) does have a strong effect.

package main

import (
    "fmt"
    "io"
    "os"
    "time"

    "github.com/minio/minio-go"
)

func main() {
    var mc *minio.Client
    var err error
    var size int64
    var count int = 28
    var outputRC []io.ReadCloser
    var inputPipe []*os.File

    mc, err = minio.New("192.168.2.95:4572", "localstack", "localstack", false)
    if err != nil {
        fmt.Printf("error: minio.New(): %v\n", err)
        os.Exit(1)
    }

    outputRC = make([]io.ReadCloser, count)
    inputPipe = make([]*os.File, count)

    // create pipes which are used to send data to the S3 objects
    for i := 0; i < count; i++ {
        // reader, writer, err = os.Pipe()
        outputRC[i], inputPipe[i], err = os.Pipe()
        if err != nil {
            fmt.Printf("error: os.Pipe() #%d: %v\n", i, err)
            os.Exit(1)
        }
    }

    // now setup the readers (read pipe output and save in S3 objects)
    for i := 0; i < count; i++ {
        time.Sleep(100 * time.Millisecond)
        idx := i
        go func() {
            filename := fmt.Sprintf("whatever-%d.mp3", idx)
            size, err = mc.PutObject("bucket-name", filename, outputRC[idx], -1, minio.PutObjectOptions{ContentType:"audio/mpeg"})
            if err != nil {
                fmt.Printf("error: mc.PutObject() #%d: %v\n", idx, err)
                //os.Exit(1)
            }
        }()
    }

    //time.Sleep(5 * time.Second)

    for i := 0; i < count; i++ {
        fmt.Fprintf(inputPipe[i], "FILE #%d\n", i)
    }

    //time.Sleep(10 * time.Second)

    for i := 0; i < count; i++ {
        inputPipe[i].Close()
    }

    time.Sleep(60 * time.Second)
}