How do I pass multipart/form-data stream from client to third-party server, using Actix client?

Question

I'm creating my own frontend client to upload videos to a video hosting service. According to the API docs, I am to POST a multipart/form:

curl -X POST \
     -H "Key: YOUR_KEY" \
     -F "file=@hello_world.mp4" \
     https://muse.ai/api/files/upload

I could setup my client to be allowed to make that cross origin request, however, the request requires a secret API key, so I need my server to proxy the request on behalf of my client.

As minimally follows:

use actix_web::{Error, HttpResponse, client::Client, error, web, HttpRequest};
use std::env;

pub async fn upload_video(req: HttpRequest, payload: web::Payload) -> Result<HttpResponse, Error> {
    let muse_api_key = match env::var("MUSE_AI") {
        Ok(token) => token,
        Err(e) => {
            return Err(error::ErrorInternalServerError(e));
        }
    };
    
    let client = Client::default();
    let mut forward_req_resp = client
      .request_from("https://muse.ai/api/files/upload", req.head())
      .header("key", muse_api_key)
      .send_stream(payload)
      .await?;

    let mut client_resp = HttpResponse::build(forward_req_resp.status());

    Ok(client_resp.body(forward_req_resp.body().await?))
}

#[actix_web::main]
async fn main() -> std::io::Result<()> {
    HttpServer::new(move || {
        App::new()
        .service(
            web::scope("/video")
                .service(
                    web::resource("/upload").route(web::post().to(upload_video)),
                )
        )
    })
    .bind("127.0.0.1:8001")?
    .run()
    .await
}

Prior to wiring up my frontend, I'm testing with cURL:

curl -v -X POST -F "file=@/path/to/sintel_trailer-720p.mp4" 127.0.0.1:8001/video/upload

Resulting output:

* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8001 (#0)
> POST /video/upload HTTP/1.1
> Host: 127.0.0.1:8001
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Length: 7608419
> Content-Type: multipart/form-data; boundary=------------------------076c45bb618a62c2
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
< HTTP/1.1 400 Bad Request
< content-length: 40
< content-type: text/plain; charset=utf-8
< set-cookie: actix-session=3JVldgobuv9nv6zE9uV%2F1i0slQMHlV384rXzc9BWwRE%3D%7B%7D; HttpOnly; Path=/
< date: Mon, 01 Feb 2021 07:32:14 GMT
* HTTP error before end of send, stop sending
< 
* Closing connection 0
protocol error: not a result of an error% 

I've tried of a few variants, maybe one of which is notable: Adding the following to top of handler function:

    let mut bytes = web::BytesMut::new();
    while let Some(item) = payload.next().await {
        bytes.extend_from_slice(&item?);
    }

Then changing ClientRequestBuilding from .request_from to:

.post("https://muse.ai/api/files/upload")

This results in: { error: "missing_file"}.

Answer 1

I figured out a solution that technically works but it's not really what I originally intended, as I would prefer to forward the multipart/form-data stream along, instead of writing the file part of the stream to a local file, which could easily take up available memory at scale.

Although I'm answering my own question, I'd love additional input to fill in my missing gaps.

Solution:

use reqwest::blocking;
use actix_multipart::Multipart;

pub async fn upload_video(mut payload: Multipart) -> Result<HttpResponse, Error> {
    let muse_api_key = match env::var("MUSE_AI") {
        Ok(token) => token,
        Err(e) => {
            return Err(error::ErrorInternalServerError(e));
        },
    };

    let mut filepath_copy = String::from("");

    while let Ok(Some(mut field)) = payload.try_next().await {
        let content_type = field.content_disposition().unwrap();
        let filename = content_type.get_filename().unwrap();
        let filepath = format!("./{}", sanitize_filename::sanitize(&filename));
        filepath_copy = format!("./{}", sanitize_filename::sanitize(&filename));

        let mut f = web::block(|| std::fs::File::create(filepath))
            .await?;

        while let Some(chunk) = field.next().await {
            let data = chunk.unwrap();
            f = web::block(move || f.write_all(&data).map(|_| f)).await?;
        }
    }

    let client = blocking::Client::new();

    let form = blocking::multipart::Form::new()
      .file("file", filepath_copy.clone())?;

    let forward_req_resp = web::block(move ||
      client
        .post("https://muse.ai/api/files/upload")
        .header("key", muse_api_key)
        .multipart(form)
        .send()
    ).await?;
    
    web::block(|| fs::remove_file(filepath_copy)).await?;

    let status = forward_req_resp.status();

    Ok(HttpResponse::build(status).finish())
}

Dependences (used for this handler, not the whole app):

futures = "0.3.12"
sanitize-filename = "0.3.0"
reqwest = { version = "0.11.0", features = ["multipart", "blocking", "json"] }
actix-multipart = "0.3.0"