0

I am implementing a websocket chat application where I want to gracefully shutdown all clients when the server stop because of the ctrl+c signal.

I am listening to incoming events using mio poll and tokens. Any new socket connection is registered with mio poll and any event received on the socket is successfully captured on the polling.

My initial idea was to use tokio::select with listen events and shutdown as 2 branches. But, I guess this design needs some modification to enable graceful shutdown.

// get_events function
fn get_events(poll, conn) -> Option<mio::Event>{
    let shutdown = tokio::signal::ctrl_c();
    let res = tokio::select!{
        res = get_poll_events(poll) => { // returns events asynchronously, its an async method
           // ... handler when events are received
           res
        }, 
        _ = shutdown => {
            // ... handler when ctrl+c signal is received
            // send close connection message to TcpStream
            return None;
        }
    };
    Some(res)
}

// main function
let poll = Poll::new();
let shared_poll = Arc::new(Mutex::new(poll));
let conn: Arc<Mutex<HashMap<Token, WebSocketClient>>> = Arc::new(Mutex::new(HashMap::new()));
let (shutdown_notifier, shutdown_receiver) = mpsc::channel(10);

loop {
      let res = WebSocketServer::get_events(shared_poll.clone(), conn.clone()).await;
      if let None = res {
          drop(shutdown_notifier); // drop the original mpsc::Sender, cloned in all clients.
          let _ = shutdown_receiver.recv().await; // mpsc::Receiver only returns error when all clients are dropped, thus dropping the senders inside them.
          break; // stop the program
      }

      // use event to register a new client with mpsc::sender
      // if readable; accept incoming message
      // broadcast message to other subscribers
      // ... other tasks
   ...
}

On Ctrl+c signal received the shutdown handler gets activated. Post that, close connection message is sent on the TcpStream to all active clients. At this point, everything is as expected. All the clients receive close messages. But the real issue starts here, since the shutdown handler executes, the get_poll_events branch of tokio::select gets dropped. Meaning, no more polling for the incoming events on the registered sources (TcpStreams).

Ideally, the server should only close when it has received back the close message response from all the clients. The clients communicate back on the TcpStream, but since there is no active mechanism to listen to these events, I am unable to capture them and hence not able to drop the clients. However, instead of sending close message to clients if I dropped all the clients manually, then there was no need to listen to close message response and things worked albeit an incorrect implementation.

Tokio::select isn't an ideal choice as far as I can guess, but I am unable to come up with a solution on how to implement this case, where the server is still active to listen to all the close message responses from clients. It can then close when all the clients have gone out of scope in the received close messages.

What would be a way to achieve this functionality? TIA.

Ashish Singh
  • 739
  • 3
  • 8
  • 21
  • If the server's controlling user types Ctrl-C, I see this quite equivalent to a pulled Ethernet cable or any other fatal server error. All clients should be able to cope with a sudden dead of the server. What problem do you try to solve with a more graceful shutdown? – the busybee Sep 05 '22 at 13:08
  • @thebusybee I am trying to emulate a server close from command line. When a server decides to terminate, the next steps is informing the clients and waiting for their response before closing down. – Ashish Singh Sep 05 '22 at 13:17
  • try putting shutdown into another tokio::select!{} or try making other function for a shutdown, so it will listen to the signals separately from get_poll_events(poll) – micegan Sep 05 '22 at 13:19
  • Well, then do this in the signal handler, which you might provide yourself. – the busybee Sep 05 '22 at 13:19
  • @micegan the thing is `get_poll_events()` function has `.await` where the execution is suspended listening for new events on the poll. If I create another method for shutdown lets then also, the `shutdown` signal will be captured inside the `get_poll_events` method only. Did I misunderstand your suggestion somehow? – Ashish Singh Sep 05 '22 at 13:26
  • @thebusybee did you mean the handler, inside the `tokio::select` block? In that case, the get_poll_events branch gets dropped, hence stopping any future events received for the ack of close connection messages sent to the clients. – Ashish Singh Sep 05 '22 at 13:30
  • I have no insights to tokio, but I would expect that one could set up an own signal handler, exactly for the purpose you describe. – the busybee Sep 05 '22 at 14:06
  • @thebusybee *"quite equivalent to a pulled Ethernet cable"* - many servers implement Ctrl-C (SIGTERM) as graceful-shutdown signal. So it's not that surprising to implement something like this, in my opinion. – Finomnis Sep 05 '22 at 16:58
  • @AshishSingh *"I am listening to incoming events using mio poll and tokens"* - are you sure you want to go this route? mio is not async and writing an async wrapper around it is a major and error-prone task. There is already [`tokio-tungstenite`](https://crates.io/crates/tokio-tungstenite), which is an excellent async websocket implementation. Even if you still want to continue with your own mio based approach, then maybe there are answers to your questions in its codebase. – Finomnis Sep 05 '22 at 17:05

0 Answers0