1

I have a service which updates cache data on a fixed interval. Every N seconds it will trigger a future using a loop (tokio::run(future_update(http_client.clone()))), but it is not returned to the parent function where the future resolved. The loop blocks and I get only one iteration.

When I create a new hyper HTTP client instead of passing a cloned one then everything works correctly. It's does not working Arc<Client> either.

pub fn trigger_cache_reload(http_client: Arc<Client<HttpConnector, Body>>) {
    let load_interval_sec = get_load_interval_sec(conf.load_interval_seconds.clone());

    std::thread::spawn(move || loop {
        let http_client = http_client.clone();

        info!("Woke up");
        tokio::run(pipeline(http_client));
        info!(
            "Pipeline run complete. Huuhh Now I need sleep of {} secs. Sleeping",
            load_interval_sec
        );
        std::thread::sleep(std::time::Duration::from_secs(load_interval_sec));
    });
}

fn pipeline(
    client: Arc<Client<HttpConnector, Body>>,
) -> Box<dyn Future<Item = (), Error = ()> + Send> {
    let res = fetch_message_payload() //return type of this call is Box<dyn Future<Item = (), Error = Error> + Send>
        .map_err(Error::from)
        .and_then(|_| {
            //let client = hyper::Client::builder().max_idle_per_host(1).build_http();
            //if i create new client here every time and use it then all working is fine.
            refresh_cache(client) //return type of this call is Box<dyn Future<Item = (), Error = Error> + Send>
                .map_err(Error::from)
                .and_then(|arg| {
                    debug!("refresh_cache completed");
                    Ok(arg)
                })
        });

    let res = res.or_else(|e| {
        error!("error {:?}", e);
        Ok(())
    });
    Box::new(res)
}

After calling of trigger_cache_reload once, I get the "woke up" log message. I also get "refresh_cache completed" log message after some time on successful completion of future. I do not get the "sleeping" log message with or without Arc.

If I create a new client inside the future every time, I am able to get "sleeping" log messages.

Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
Jarvis42
  • 61
  • 1
  • 7

1 Answers1

1

tokio::run creates a completely new event loop and thread pool (reactor + executor) every time you call it. This is really not what you want to do.

The hyper client will bind it's state to the previous event loop and can't make progress if polled on the new one, since the old event loop will be destroyed after run finishes. That's why a new client works, but you can't reuse the old one.

There are two solutions here:

  • If the rest of your application is not using tokio, I would just use the synchronous reqwest::Client. If you don't need a lot of concurrency, a synchronous solution is much easier here.

  • if you are using tokio, use tokio::spawn inside another Future together with tokio_timer::Timeout to run the checks and then wait for the specified amount of time on the event loop.

async/await Example

The new async/await support makes code like this much easier to write.

This example currently only works on the nightly compiler with tokio-0.3.0-alpha.2 and the current hyper master branch:

[dependencies]
tokio = "0.3.0-alpha.2"
tokio-timer = "0.3.0-alpha.2"
hyper = { git = "https://github.com/hyperium/hyper.git" }

use tokio::timer::Interval;
use hyper::{Client, Uri};

use std::time::Duration;

#[tokio::main]
async fn main() {
    let client = Client::new();
    let second_interval = 120;
    let mut interval = Interval::new_interval(Duration::from_secs(second_interval));
    let uri = Uri::from_static("http://httpbin.org/ip");

    loop {
        let res = Client.get(uri.clone()).await.unwrap();
        // Do what you need to with the response...
        interval.next().await;
    }
}
theduke
  • 3,027
  • 4
  • 29
  • 28