Tokio receiver cannot read from channel unless sender task task has any "await" in its closure. Potential bug or a quirk in tokio?

Question

Code:

use std::thread;
use chrono::Utc;
use tokio::sync::watch::Receiver;
use std::time::Duration;
use tokio::sync::watch;
use tokio::time::sleep;

pub fn get_now_formatted() -> String {
    Utc::now().format("%Y-%m-%d %H:%M:%S.%3f").to_string()
}

#[tokio::main]
async fn main() {
    let mut rx = subscribe_orderbook().await;

    // simple thread that reads from the stream
    tokio::spawn(async move {
        loop {
            println!("{}: Trying to read..", get_now_formatted());
            rx.changed().await.expect("TODO: panic message");
            println!("DOESN'T READ PROPERLY UNLESS WE COMMENT OUT AWAIT LINE IN SENDER");
            println!("{}: Received: {:?}", get_now_formatted(), rx.borrow().clone());
        }
    });

    loop {
        sleep(Duration::from_millis(1000)).await;
    }
}

pub async fn subscribe_orderbook() -> Receiver<i32> {
    // this is where the issue will happen:
    let (channel_tx, channel_rx) = watch::channel(1);

    tokio::spawn( async move {
        let mut counter = 0;
        loop {
            counter += 1;
            let result = channel_tx.send(counter);
            if result.is_err() {
                eprintln!("{} Failed to send fair value to main thread: {}", get_now_formatted(), result.err().unwrap());
            }
            else {
                println!("{} SENT {:?}", get_now_formatted(), counter);
            }

            // NOTE: commenting out this pointless await fixes the problem!
            // sleep(Duration::from_millis(0)).await;
        }
    });
   channel_rx
}

It prints only one "Received.." despite printing many "SENT".
I can fix it by commenting out the line sleep(Duration::from_millis(0)).await

Essentially, this only works if the sender task has an "await" somewhere in its closure. This is what I discovered after many hours of digging and distilled it into this example code.

Note that this is specific to the sender task. If I create a separate task that loops forever, it does not cause any issues. Meaning, adding the following to the main function does not cause problems:

    // NOTE: Let's add a task that blocks forever... does not make any difference to either case
    tokio::spawn(async move {
        loop {
            thread::sleep(Duration::from_millis(1000));
        }
    });

Why does adding any "await" in the sender work, and is it "correct" to simply add that pointless sleep for 0 ms line as a workaround?

Thank you

EDIT:

It seems like just adding an "await" is not enough because it introduces strange delays.

Does this answer your question? [Tokio subtask within task not executing as expected](https://stackoverflow.com/questions/72870125/tokio-subtask-within-task-not-executing-as-expected) — cafce25, Apr 08 '23 at 05:11
@cafce25 I can remove the blocking operation thread::sleep from the sender loop and the issue persists. It only goes away if I "await" anything in the sender loop. — SpaceMonkey, Apr 08 '23 at 09:13
Right now the code doesn't make logical sense. What are you counting? Answering that will determine how to fix your code. — drewtato, Apr 08 '23 at 17:53
@drewtato it's just dummy sequential data to send on the channel to let me reproduce the problem in a simple example — SpaceMonkey, Apr 08 '23 at 19:02
The answer depends on what you actually want to do in the loop. — drewtato, Apr 08 '23 at 19:46

score 0 · Answer 1 · answered Apr 08 '23 at 02:23

Tokio tasks should never block, or you can get situations like this. You should use tokio's sleep instead of std::thread::sleep:

// thread::sleep(Duration::from_millis(1000));
sleep(Duration::from_millis(1000)).await;

If you are doing a computationally expensive or otherwise blocking task, you should use spawn_blocking:

tokio::task::spawn_blocking(|| expensive_task())
    .await
    .unwrap();

Another thing that is sometimes useful is inserting calls to yield_now. This is the direct replacement for the zero-duration sleep, but the other solutions are better if it's possible to use them.

for _ in 0..100 {
    tokio::task::yield_now().await;
    // Something that takes some time
    thread::sleep(Duration::from_millis(10));
}

If I remove `std::thread::sleep`, the same problem happens. I suppose I'm still "blocking" this way because it's an infinite loop? So, I have to have an "await" somewhere in the sender task? — SpaceMonkey, Apr 08 '23 at 09:09

SpaceMonkey · Answer 2 · 2023-04-08T13:14:41.360

As I suspected, the issue was the lack of any "await" in the sender task.

The problem is not the use of thread::sleep. Removing that line (which means I'll loop forever), doesn't resolve the problem. I've edited my question to remove that line entirely.

It seems like the sender loop that is using the send channel has to "give back control" at some point which happens when await is used. Otherwise, no other task can read from the receiver part of the channel.

The solution is:

use tokio::task::spawn_blocking
comment out the line in my code that does a pointless await: sleep(Duration::from_millis(0)).await;

However, only the first solution seems to be the correct one. The second solution introduces strange delays.

Tokio receiver cannot read from channel unless sender task task has any "await" in its closure. Potential bug or a quirk in tokio?

2 Answers2