How to solve write: broken pipe on websocket?

Question

I'm making a progress bar for my program, and send it to client. The IPC I used is websocket. But I'm facing write: broken pipe when updating progress bar to the client. This error occurs after the first request. For example, on initial run of my app, when i perfrom request to the server, there is no error. But, when I perform another request, this error will occur.

Even though there is this error, but the progress bar is actually working despite the error (the progress bar is successfully sent to the client until 100%)

I'm using go fiber websocket as my server

// middleware
app.Use("/ws", func(c *fiber.Ctx) error {
  if websocket.IsWebSocketUpgrade(c) {
    c.Locals("allowed", true)
      return c.Next()
    }

    return fiber.ErrUpgradeRequired
})

// handler
func (s *downloaderService) progressBar(c *websocket.Conn) {
    channel := api.CreateChannel(c.Params("client"))
    done := make(chan bool)

    go func() {
        for {
            t, _, err := c.ReadMessage()
            if err != nil {
                log.Println("Error reading message:", err)
                return
            }

            if t == websocket.CloseMessage {
                done <- true
            }
        }
    }()

    for {
        select {
        case <-done:
            return
        case data, ok := <-channel.Subscribe():
            if !ok {
                return
            }

            progressBar := data.(downloader.Progressbar)

            payload, err := json.Marshal(progressBar)
            if err != nil {
                log.Println("Error marshalling data:", err)
                break
            }

            c.SetWriteDeadline(time.Now().Add(10 * time.Second))
            if err := c.WriteMessage(websocket.TextMessage, payload); err != nil {
                log.Println("Error sending progress data:", err)
                done <- true
            }
        }
    }
}

On the client, I'm using gorilla websocket


func main() {
    interrupt := make(chan os.Signal, 1)
    signal.Notify(interrupt, []os.Signal{syscall.SIGINT, syscall.SIGKILL, syscall.SIGTERM, syscall.SIGSTOP, os.Interrupt}...)

    ctx, cancel := context.WithCancel(context.Background())

    conn, res, err := websocket.DefaultDialer.DialContext(ctx, ws, nil)
    if err != nil {
        log.Fatalf("Error dialing websocket: %v. Status courlde %d", err, res.StatusCode)
        return
    }

    executeCommand(ctx)

    progressBar := progressbar()

    go func() {
        for {
            _, message, err := conn.ReadMessage()
            if err != nil {
                log.Println("Error reading message:", err)
                break
            }

            var progress progress
            if err := json.Unmarshal(message, &progress); err != nil {
                log.Println("Error unmarshalling message:", err)
                break
            }

            if progress.Done {
                truncateStore()
                cancel()
                break
            }

            progressBar.update(progress.Index, progress.Downloaded, progress.Size)
        }
    }()

    for {
        select {
        case <-ctx.Done():
            closeConn(ctx, conn)
            return
        case <-interrupt:
            stopDownload()
            closeConn(ctx, conn)
            return
        }
    }
}


func closeConn(ctx context.Context, conn *websocket.Conn) {
    if err := conn.WriteMessage(websocket.CloseMessage, websocket.FormatCloseMessage(websocket.CloseNormalClosure, "")); err != nil {
        log.Println("Error sending close signal to server:", err)
        return
    }

    select {
    case <-ctx.Done():
    case <-time.After(time.Second):
    }
}

The client is a CLI that will create request to the server. And when the progress bar is complete, it will be terminated.

Here is how it looked like

Update

After reading the close message from client, I manage to get rid the broken pipe error. Instead, i got write: close sent

The peer has closed the connection, possibly without having read all the pending incoming data. — user207421, Sep 03 '23 at 04:57

score -1 · Answer 1 · answered Sep 03 '23 at 04:58

The error write: broken pipe generally indicates that the receiver (in this case, the client) closed the connection before the server could write to it. Even if you've noticed that the progress bar is working on the initial run, subsequent requests might encounter situations where there's a race condition between writing messages on the server side and reading/closing the connection on the client side.

Here are some steps you can take to diagnose and potentially solve the problem:

Ensure Persistent Connection: Check if the client maintains a persistent WebSocket connection across multiple requests. If the client closes the WebSocket connection and then tries to use the same closed connection for subsequent requests, you will encounter such errors.
Graceful Shutdown: Ensure both the server and the client are shutting down the connection gracefully. You're using websocket.CloseMessage to indicate a closure on the client side, but you should also ensure the server acknowledges this and closes the connection gracefully from its side.
Error Handling: Your current error handling in the progressBar function exits the loop as soon as there's an error. This might be problematic if you want to ensure persistent communication, even in the face of temporary errors. Consider adding more refined error handling.
Context Handling: Ensure the ctx you're using with websocket.DefaultDialer.DialContext(ctx, ws, nil) isn't prematurely canceled, leading to connection termination. Your code does suggest that you cancel the context when progress.Done is true, but just double-check the rest of the application to make sure there's no unintentional context cancellation.
Logs & Debugging: Add more logs on both the server and client sides, especially around connection initiation, message send/receive, and connection termination. This will give you a clearer picture of the order of operations and might help pinpoint where things go wrong.
Multiple Connections: Ensure that you're not unintentionally creating multiple WebSocket connections from the client side, which could lead to unexpected behavior.
Handle Connection Closure: Ensure that you're handling the WebSocket's CloseHandler on both the client and server sides to manage the connection termination gracefully.
Check for Resource Limits: Ensure your server isn't hitting any open file descriptor limits or similar resource limits, which might cause connection disruptions.
Check Middleware: Double-check any middleware you're using with Go Fiber. Ensure they're not interfering with the WebSocket connection.

If you're using the CLI client in a way that you're making rapid consecutive requests (like quickly re-running the CLI command), you might want to add a slight delay or ensure that the previous WebSocket connection has fully closed before initiating a new one.

How to solve write: broken pipe on websocket?

1 Answers1