5

I'm attempting to copy a file downloaded with reqwest in to a tokio file. This file is too large to store in memory so it needs to be through the bytes_stream() rather than bytes()

I had attempted the following

let mut tmp_file = tokio::fs::File::from(tempfile::tempfile()?);
let byte_stream = reqwest::get(&link).await?.bytes_stream();
tokio::io::copy(&mut byte_stream, &mut tmp_file).await?;

This fails due to

    |
153 |     tokio::io::copy(&mut byte_stream, &mut tmp_file).await?;
    |     --------------- ^^^^^^^^^^^^^^^^ the trait `tokio::io::AsyncRead` is not implemented for `impl Stream<Item = Result<bytes::bytes::Bytes, reqwest::Error>>`
    |     |
    |     required by a bound introduced by this call

Is there any way I can get the trait AsyncRead on the Stream or otherwise copy this data in to the file? The reason I'm using a tokio file is I later need to AsyncRead from it. Perhaps it would make sense to copy in to a regular std::File and then convert it to a tokio::fs::File?

Qwertie
  • 5,784
  • 12
  • 45
  • 89
  • I thing you have to call [`StreamExt::for_each`](https://docs.rs/futures/0.3.25/futures/stream/trait.StreamExt.html#method.for_each) on your stream and for each packet write it to the file. – Aleksander Krauze Nov 02 '22 at 00:31

1 Answers1

4

This method works. Loosely based on the example for bytes_stream()

Note: You probably want to buffer the writes here

use futures::StreamExt;

let mut tmp_file = tokio::fs::File::from(tempfile::tempfile()?);
let mut byte_stream = reqwest::get(&link).await?.bytes_stream();

while let Some(item) = byte_stream.next().await {
  tokio::io::copy(&mut item?.as_ref(), &mut tmp_file).await?;
}
Qwertie
  • 5,784
  • 12
  • 45
  • 89