0

I'm just getting started with Rust and am trying to work with concurrent requests. My aim is to have an asynchronous function that returns a vector of responses from a number of GET requests. What I have currently does successfully execute the requests concurrently but does not return anything from the function:

// main.rs

mod api_requester;

#[tokio::main]
async fn main() {
    let values = vec!["dog".to_string(), "cat".to_string(), "bear".to_string()];

    let responses = api_requester::get_data(values).await;

    println!("{:?}", responses)
    // do more stuff with responses
}
// api_requester.rs

use serde::Deserialize;
use futures::{stream, StreamExt};
use reqwest::Client;

const CONCURRENT_REQUESTS: usize = 2;
const API_ENDPOINT: &str = "https://httpbin.org/response-headers";

#[derive(Deserialize, Debug)]
struct ApiResponse {
    #[serde(rename(deserialize = "Content-Length"))]
    content_length: String,

    #[serde(rename(deserialize = "Content-Type"))]
    content_type: String,

    freeform: String
}


pub async fn get_data(values: Vec<String>) {
    let client = Client::new();

    let bodies = stream::iter(values)
        .map(|value| {
            let client = &client;

            async move {
                let resp = client.get(API_ENDPOINT)
                    .query(&[("freeform", value)])
                    .send().await?;

                resp.json::<ApiResponse>().await
            }
        })
        .buffer_unordered(CONCURRENT_REQUESTS);

    bodies
        .for_each(|body| async {
            match body {
                Ok(body) => println!("Got {:?}", body),
                Err(e) => eprintln!("Got an error: {:?}", e),
            }
        })
        .await;

}

My goal is to return a vector of the responses received from the GET requests back to the main function for further use. But this is where I'm having some serious confusion. I thought I would be able to just await the value in the function and return the vector when the futures have been resolved. something like this:

// api_requester.rs

...

pub async fn get_data(values: Vec<String>) -> Vec<ApiResponse> {
    let client = Client::new();

    let bodies = stream::iter(values)
        .map(|value| {
            let client = &client;

            async move {
                let resp = client.get(API_ENDPOINT)
                    .query(&[("freeform", value)])
                    .send().await?;

                resp.json::<ApiResponse>().await
            }
        })
        .buffer_unordered(CONCURRENT_REQUESTS);

    bodies
}

This produces the following error:

error[E0308]: mismatched types
  --> src/api_requester.rs:37:5
   |
24 |           .map(|value| {
   |                ------- the found closure
...
27 |               async move {
   |  ________________________-
28 | |                 let resp = client.get(API_ENDPOINT)
29 | |                     .query(&[("freeform", value)])
30 | |                     .send().await?;
31 | |
32 | |                 resp.json::<ApiResponse>().await
33 | |             }
   | |_____________- the found `async` block
...
37 |       bodies
   |       ^^^^^^ expected struct `Vec`, found struct `BufferUnordered`
   |
  ::: /home/seraub/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:72:43
   |
72 |   pub const fn from_generator<T>(gen: T) -> impl Future<Output = T::Return>
   |                                             ------------------------------- the found opaque type
   |
   = note: expected struct `Vec<ApiResponse>`
              found struct `BufferUnordered<futures::stream::Map<futures::stream::Iter<std::vec::IntoIter<std::string::String>>, [closure@src/api_requester.rs:24:14: 24:21]>>`

For more information about this error, try `rustc --explain E0308`.

I'm guessing that the BufferUnordered<futures::stream::Map<futures::stream::Iter<std::vec::IntoIter<std::string::String>>, [closure@src/api_requester.rs:24:14: 24:21]>> struct found by the compiler needs to be realized/completed?

How can I turn this BufferUnordered object into a simple vector of responses and return it back to the main function?

Xylot
  • 55
  • 3
  • 2
    You probably want something like [`.try_collect().await`](https://docs.rs/futures/latest/futures/stream/trait.TryStreamExt.html#method.try_collect), which will return a result either containing the collection of your choice or the first error received during the stream. Note you will need to import `TryStreamExt`. – PitaJ Oct 10 '22 at 22:12
  • Also note that your first `for_each().await` based approach does **not** actually send concurrent requests. They are all sent in sequence. Rust futures are **lazy** and only run once awaited or spawned. – Finomnis Oct 10 '22 at 22:59

0 Answers0