WebSocket inside express route works only for the first call

Question

I have an expressJS server and I would like to call a WebSocket (I use the ws package) inside it.

Here is what I do on the server side:

import express from 'express';
import { WebSocketServer } from 'ws';

const app = express();

const server = app.listen(3000);

const wss = new WebSocketServer({ server });

// Set the wss to be able to retrieve it in the route
app.set('wss', wss);

app.get('/download', (req, res) => {
  // Retrieve the WebSocket server
  const wss = req.app.get('wss');
  wss.on('connection', async (ws) => {
    ws.send('hello world');
    // Do some async stuff here to gather data and send some messages
    const fileContents = Buffer.from(myData);

    const readStream = new stream.PassThrough();
    readStream.end(fileContents);

    res.set('Content-disposition', `attachment; filename=test.csv`);
    res.set('Content-Type', 'text/csv');

    readStream.pipe(res);
    ws.close()
  })
})

In my client:

axios.get('http://'+ window.location.host +'/download', {
  responseType: 'blob',
}).then(response=>{
  // create file link in browser's memory
  const href = URL.createObjectURL(response.data);

  // create "a" HTML element with href to file & click
  const link = document.createElement('a');
  link.href = href;
  link.setAttribute('download', `file.csv`); 
  //or any other extension
  document.body.appendChild(link);
  link.click();

  // clean up "a" element & remove ObjectURL
  document.body.removeChild(link);
  URL.revokeObjectURL(href);
})
const ws = new WebSocket('ws://' + window.location.host);

ws.addEventListener('message', function(m) {
  // Do some stuff with messages
})

This works properly the first time /download is called (the messages are sent and the file gets downloaded). But the second time (when I refresh the page of the client) the messages are sent but the download fails. I get the following error in the server and the server crashes:

node:internal/errors:478
    ErrorCaptureStackTrace(err);
    ^

Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client

Express seems to consider the second request part of the first. Why is express not interpreting the second call as an independent call like it does for any endpoint call? What should I do to avoid this error?

You have to move the `wss.on('connection', ...)` outside your route. The way you have it, you install another one of those listeners everytime that route is hit and they accumulate duplicates over and over. There is no webSocket model where you listen only for connections from one specific page. You install the listener globally and then have the client connect only from pages that it wishes to. — jfriend00, May 09 '23 at 19:24
The ERR_HTTP_HEADERS_SENT issue is because you have more than one `wss.on('connection', ...)` so each duplicate is attempting to do `readStream.pipe(res);` again which causes that error. — jfriend00, May 09 '23 at 19:31
I honestly don't understand what the client code is trying to do so you'll have to back up a few steps and describe what the actual problem is you're trying to solve and then we can come up with an approach for that. — jfriend00, May 09 '23 at 19:54
@jfriend00 My initial problem was this: https://stackoverflow.com/questions/76186270/download-dialog-prompt-from-websocket-endpoint and I came up with the solution above. All I am trying to do is to show a progress bar for the time it takes to extract the data from the database and at the end send the file to the client. I can't really save the data into the server since I scale horizontally and it would require to manage that (looking for an easy solution). — Nate, May 09 '23 at 20:02
That other question probably didn't get any traction because it's abstract/theoretical. It doesn't show what you're actually trying to accomplish. Are you just trying to make an Ajax call to the server and show progress in the client while that is processing? Or are you trying to do giant uploads to your server from the client? Or what? What data is flowing which directions? — jfriend00, May 09 '23 at 20:10
@jfriend00 The data is flowing from the server to the client. The client triggers the extraction. The server extracts the data from the database and at the same time sends the progress of the extraction to the client and once the extraction is done, it sends the file to the client (it should prompt the download dialog like it does for anything you would download online). — Nate, May 09 '23 at 20:14
How big is the data you're sending to the client after the server did its database work and what is the client going to do with it? — jfriend00, May 09 '23 at 20:21
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/253574/discussion-between-nate-and-jfriend00). — Nate, May 09 '23 at 20:36
I'm just trying to figure out if the data can be sent over webSocket or if it has to go via an HTML response. — jfriend00, May 09 '23 at 20:39
@jfriend00 from what I understand, sending the data through a websocket will require to manage the download process. Which requires significant work. — Nate, May 09 '23 at 21:28
Please answer my previous question. How big is the data and what is the client trying to do with it? — jfriend00, May 09 '23 at 21:49
@jfriend00 Sorry, I answered in the chat and thought you saw it... up to 100-200MB but can be as low as few MB — Nate, May 09 '23 at 21:53
@jfriend00 I actually solved my problem with socket.io. I listen to the connections outside of my route and emit to the corresponding user within the route. Thanks for your help! — Nate, May 10 '23 at 20:01
You could post an answer to show your solution (in an answer, not in the question). — jfriend00, May 10 '23 at 22:55

score 0 · Answer 1 · answered May 10 '23 at 18:55

If you decided to use WebSocket, then using REST API is a bit obsolete. Why don't you communicate with WS only? It's pretty simple and very swift solution, where you can build something like router based on data, you send in the ws requests.

server.js

const wss = new WebSocketServer({ server });

wss.on('connection', async (ws) => {
    ws.send('CONNECTION SUCCESFUL')
    ws.on('message', (mes, isBinary) => {
      const m = isBinary ? mes : mes.toString();
      // do whatever you want with the message
      // I am sending Object, which helps me
      // routing the ws requests:
      const fullMessage = JSON.stringify(m)
      if (m.action) === 'download' {
           // do the download stuff
      } else if (m.action === 'processingTime') {
          ws.send(JSON.stringify({action: 'processingTime', time: '00:00:00'}
      }
    })
})

On the client side just process it similarly:

const ws = new WebSocket('ws://' + window.location.host);

ws.addEventListener('message', function(m) {
  // Parse the message
  if (m.action === 'download') {
     // receive downloaded file
  } else if (m.action === 'processingTime') {
    console.log('TIME: ', m.time)
  }
})

How to send files with ws, see my previous answer here.

This doesn't solve the problem, you have to implement the whole download mechanism with websockets which is not trivial to do... — Nate, May 10 '23 at 20:11
I do not want to convince you to use websocket for sending files, don't be afraid, just see the link from my answer. There are only few lines of code. Just give it a try. If I did it, you will. — Fide, May 11 '23 at 10:09

WebSocket inside express route works only for the first call

1 Answers1