5

I'm currently trying to achieve data streaming trough a simple HTTP get request in rust with hyper (0.13). The idea is simple. If the client sends a request the server responds in "chunks" every 5s leaving the connection open the whole time.

I'm trying to implement the trait Stream from futures crate

impl Stream for MonitorString {
    type Item = Result<String, serde_json::Error>;

    fn poll_next(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<Option<Result<String, serde_json::Error>>> {
        thread::sleep(Duration::from_secs(5)); 

        Poll::Ready(Some(self.data.to_string()))
    }
}

Then build a response like

type StreamingServiceResult = Result<Response<Body>, Box<dyn std::error::Error + Sync + Send>>;

pub async fn handle_request(_: Request<Body>, params: Params, _: Query) -> StreamingServiceResult {
     Ok(Response::builder()
        .header(hyper::header::CONTENT_TYPE, "application/json")
        .header(hyper::header::ACCESS_CONTROL_ALLOW_ORIGIN, "*")
        .header(hyper::header::TRANSFER_ENCODING, "chunked")
        .body(Body::wrap_stream(MonitorString::new(...)))
}

When I send a request the server, it opens the connection and hangs. With some debugging I saw that the poll_next function is called, but the response is only sent when the Stream has nothing more to yield (Poll::Ready(None) is returned by poll_next) and is sent in the whole.

I can see that my initial understanding of the Stream trait and Body::wrap_stream was wrong. Sadly I couldn't find any examples that works for my use-case. Can you put me on the right track with this one?

Adrián Nagy
  • 51
  • 3
  • 1
    That probably unrelated but you should avoid blocking operations like `thread::sleep` inside `Stream::poll_next`/`Future::poll`. Consider async versions like [`tokio::time::delay_for`](https://docs.rs/tokio/0.2.21/tokio/time/fn.delay_for.html) instead. – Kitsu Jul 17 '20 at 08:50
  • @Kitsu Good point, but is it somehow possible to await for the future returned by `tokio::time::delay_for` in a non async function like `Stream::poll_next`? Actually I tested my example without the sleep and the streaming part seems to work :) Now just have to figure out how to use some kind of delay – Adrián Nagy Jul 17 '20 at 10:23
  • Manual `Stream` implementation is fairly complex indeed, you might want to use [async-stream](https://crates.io/crates/async-stream) instead. Otherwise you need to store the `Delay` future, poll until it ready and then reset. – Kitsu Jul 17 '20 at 11:11

0 Answers0