Unclear expected type in Rust web server

Question

I am trying to build a simple web server to upload and save images using rust.

Here is my code:

use std::convert::Infallible;
use std::path::PathBuf;
use warp::multipart::FormData;
use warp::{http::StatusCode, Filter};
use futures::stream::TryStreamExt;

#[tokio::main]
async fn main() {
    let upload_route = warp::path("upload")
        .and(warp::post())
        .and(warp::multipart::form().max_length(3 * 1024 * 1024))
        .and_then(upload_handler);

    let hello_route = warp::path!("hello")
        .and(warp::get())
        .and(warp::fs::file("./static/index.html"));

    let routes = upload_route.or(hello_route);

    println!("Server started at http://localhost:8080");
    warp::serve(routes).run(([127, 0, 0, 1], 8080)).await;
}

async fn upload_handler(mut form: FormData) -> Result<impl warp::Reply, Infallible> {
    while let Ok(Some(part)) = form.try_next().await {
        let filename = match part.filename() {
            Some(filename) => filename.to_string(),
            None => return Ok(StatusCode::BAD_REQUEST),
        };
        println!("Receiving file: {}", filename);

        let mut data = Vec::new();
        let mut stream = part.stream();
        
        while let Ok(Some(chunk)) = stream.try_next().await {
            data.extend_from_slice(&chunk);
        }

        let save_path = PathBuf::from(format!("./uploads/{}", filename));
        if let Err(_) = tokio::fs::write(save_path, data).await {
            return Ok(StatusCode::INTERNAL_SERVER_ERROR);
        }
    }
    Ok(StatusCode::CREATED)
}

My problem is in line data.extend_from_slice(&chunk);, rust says

expected `&[_]`, found `&impl Buf`

This confuses me as a beginner rust user. I am not exactly clear what [_] means and why data.extend_from_slice does not accept the chunk bytes. I appreciate your help.

`[_]` is a slice (extend_from_**slice**), `chunk` is not a slice. (Aside: saving to a user-provided filename is usually best avoided when possible. From a quick look through the warp/multer source code, I didn’t see anything that would stop `part.filename()` from being `../../etc/passwd`, i.e. a path traversal vulnerability.) — Ry-, May 19 '23 at 03:25
Looks like you can call `reader` to get something that implements `Read`. Then you can call `read_to_end(&mut data)`. If that works, I'll write an answer. — drewtato, May 19 '23 at 05:10
@Ry Thanks for that clarification, I'm still not quite clear on how I can convert my chunks to a slice? It's my understanding that chunks are an iterator over a slice, so I guess I'm confused why chunk here is a Buf? Thanks, the path traversal vulnerability is clear to me now. — tonyd629, May 19 '23 at 14:40
@drewtato can you give me a little more guidance on that? Are you suggesting I use BufReader to read each chunk? `data` is already read and ready to be written to disk, so do you mean I call read_to_end on stream? — tonyd629, May 19 '23 at 14:43

score 0 · Answer 1 · answered May 19 '23 at 16:04

0

I was able to address this type error using the Buf chunk method like so

while let Ok(Some(mut chunk)) = stream.try_next().await {
    let slice: &[u8] = chunk.chunk();
    data.extend_from_slice(slice);
    chunk.advance(slice.len());
}

answered May 19 '23 at 16:04

tonyd629

75
7

score 0 · Answer 2 · answered May 19 '23 at 17:53

You can use reader to create an implementer of Read, and then call read_to_end.

while let Ok(Some(mut chunk)) = stream.try_next().await {
    chunk.reader().read_to_end(&mut data).unwrap();
}

The issue with chunk is that it doesn't have to return the entire buffer.

Note that this can return shorter slice

If you want to use chunk, you have to loop until remaining is zero.

while let Ok(Some(mut chunk)) = stream.try_next().await {
    while chunk.remaining() > 0 {
        let slice: &[u8] = chunk.chunk();
        data.extend_from_slice(slice);
        chunk.advance(slice.len());
    }
}

Unclear expected type in Rust web server

2 Answers2