5

ps: the answer below helped but it's not the answer I need, I have a new problem and I edited the question

I'm trying to make a custom transporter for the hyper http crate, so I can transport http packets in my own way.

Hyper's http client can be passed a custom https://docs.rs/hyper/0.14.2/hyper/client/connect/trait.Connect.html here:

pub fn build<C, B>(&self, connector: C) -> Client<C, B> where C: Connect + Clone, B: HttpBody + Send, B::Data: Send,

If we look at

impl<S, T> Connect for S where    

S: Service<Uri, Response = T> + Send + 'static,    

S::Error: Into<Box<dyn StdError + Send + Sync>>,    

S::Future: Unpin + Send,    

T: AsyncRead + AsyncWrite + Connection + Unpin + Send + 'static, 

the type T, which is the type of the Response, must implement AsyncRead + AsyncWrite, so I've chosen type Response = Cursor<Vec<u8>>.

Here's my custom transporter with a Response of type std::io::Cursor wrapped in CustomResponse so I can implement AsyncWrite and AsyncRead to it:

use hyper::service::Service;
use core::task::{Context, Poll};
use core::future::Future;
use std::pin::Pin;
use std::io::Cursor;
use hyper::client::connect::{Connection, Connected};
use tokio::io::{AsyncRead, AsyncWrite};

#[derive(Clone)]
pub struct CustomTransporter;

unsafe impl Send for CustomTransporter {}

impl CustomTransporter {
    pub fn new() -> CustomTransporter {
        CustomTransporter{}
    }
}

impl Connection for CustomTransporter {
    fn connected(&self) -> Connected {
        Connected::new()
    }
}

pub struct CustomResponse {
    //w: Cursor<Vec<u8>>,
    v: Vec<u8>,
    i: i32
}

unsafe impl Send for CustomResponse {
    
}

impl Connection for CustomResponse {
    fn connected(&self) -> Connected {
        println!("connected");
        Connected::new()
    }
}

impl AsyncRead for CustomResponse {
    fn poll_read(
        mut self: Pin<&mut Self>,
        cx: &mut Context<'_>,
        buf: &mut tokio::io::ReadBuf<'_>
    ) -> Poll<std::io::Result<()>> {
        self.i+=1;
        if self.i >=3 {
            println!("poll_read for buf size {}", buf.capacity());
            buf.put_slice(self.v.as_slice());
            println!("did poll_read");
            Poll::Ready(Ok(()))
        } else {
            println!("poll read pending, i={}", self.i);
            Poll::Pending
        }
    }
}

impl AsyncWrite for CustomResponse {
    fn poll_write(
        mut self: Pin<&mut Self>,
        cx: &mut Context<'_>,
        buf: &[u8]
    ) -> Poll<Result<usize, std::io::Error>>{
        //let v = vec!();
        println!("poll_write____");

        let s = match std::str::from_utf8(buf) {
            Ok(v) => v,
            Err(e) => panic!("Invalid UTF-8 sequence: {}", e),
        };

        println!("result: {}, size: {}, i: {}", s, s.len(), self.i);
        if self.i>=0{
            //r
            Poll::Ready(Ok(s.len()))
        }else{
            println!("poll_write pending");
            Poll::Pending
        }
    }
    fn poll_flush(
        mut self: Pin<&mut Self>,
        cx: &mut Context<'_>
    ) -> Poll<Result<(), std::io::Error>> {
        println!("poll_flush");
        if self.i>=0{
            println!("DID poll_flush");
            Poll::Ready(Ok(()))
        }else{
            println!("poll_flush pending");
            Poll::Pending
        }
    }

    fn poll_shutdown(
        mut self: Pin<&mut Self>,
        cx: &mut Context<'_>
    ) -> Poll<Result<(), std::io::Error>>
    {
        println!("poll_shutdown");
        Poll::Ready(Ok(()))
    }
}


impl Service<hyper::Uri> for CustomTransporter {
    type Response = CustomResponse;
    type Error = hyper::http::Error;
    type Future = Pin<Box<dyn Future<Output = Result<Self::Response, Self::Error>> + Send>>;

    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        println!("poll_ready");
        Poll::Ready(Ok(()))
        //Poll::Pending
    }

    fn call(&mut self, req: hyper::Uri) -> Self::Future {
        println!("call");
        // create the body
        let body: Vec<u8> = "HTTP/1.1 200 OK\nDate: Mon, 27 Jul 2009 12:28:53 GMT\nServer: Apache/2.2.14 (Win32)\nLast-Modified: Wed, 22 Jul 2009 19:15:56 GMT\nContent-Length: 88\nContent-Type: text/html\nConnection: Closed<html><body><h1>Hello, World!</h1></body></html>".as_bytes()
            .to_owned();
        // Create the HTTP response
        let resp = CustomResponse{
            //w: Cursor::new(body),
            v: body,
            i: 0
        };
         
        // create a response in a future.
        let fut = async move{
            Ok(resp)
        };
        println!("gonna return from call");
        // Return the response as an immediate future
        Box::pin(fut)
    }
}

Then I use it like this:

let connector = CustomTransporter::new();
let client: Client<CustomTransporter, hyper::Body> = Client::builder().build(connector);
let mut res = client.get(url).await.unwrap();

However, it gets stuck and hyper never reads my response, but it writes the GET to it.

Here's a complete project for testing: https://github.com/lzunsec/rust_hyper_custom_transporter/blob/39cd036fc929057d975a71969ccbe97312543061/src/custom_req.rs

RUn like this:

cargo run http://google.com
Guerlando OCs
  • 1,886
  • 9
  • 61
  • 150
  • It looks to me like the ` Pin::new(&mut self.w)` does not survive long enough. – Unapiedra Feb 18 '21 at 03:08
  • @Unapiedra I don't get it. Rust, as far as I know, prevents this. If it didn't live enough then I'd have undefined behaviour, I guess? How should i try to fix this to see if this is the error? – Guerlando OCs Feb 19 '21 at 04:32
  • The `Send` is `unsafe`, which I guess might lead to this. Anyway, here's some link: https://github.com/hyperium/hyper/issues/1955 – Unapiedra Feb 19 '21 at 08:12
  • @Unapiedra do you have any idea on why this could be a problem? In theory, if I do not use `unsafe` in my code, there should be no undefined behaviour – Guerlando OCs Feb 25 '21 at 03:36
  • I don't think adding yet another bounty will solve the problem... – Unapiedra Mar 02 '21 at 18:35
  • There is no UB! The linked issue gives a hint on what's happening. Something goes out of scope, and the error that you get when running your code says `ChannelClosed`. There is no UB, just _unwanted_ behavior. Try to keep your `let r = Pin::new(&mut self.w).poll_read(cx, buf);` longer in scope, maybe? (I tried and couldn't get it to work.) – Unapiedra Mar 02 '21 at 18:40
  • @Unapiedra unfortunately this is not the problem. I discovered that it closes the channel because I sent the response before it sent the `GET / google.com` HTTP request. I changed the code so it only returns the answer after the GET is sent, however hyper gets stuck and never reads my answer, even though I receive the GET request. Please take a look at the new code. – Guerlando OCs Mar 02 '21 at 22:44
  • 1
    @Unapiedra new code: https://github.com/lzunsec/rust_hyper_custom_transporter/blob/39cd036fc929057d975a71969ccbe97312543061/src/custom_req.rs. Seems like I'm close to the answer, please tell me if you know what's happening. No more Pin and Cursors being used now, so that error is not the cause – Guerlando OCs Mar 02 '21 at 22:45
  • @Unapiedra I commented a LOT of things on hyper and still couldn't figure out why it gets stuck and do not read from my messages – Guerlando OCs Mar 02 '21 at 22:46

1 Answers1

6

I cannot simply implement Send to Future, and I cannot change Future by a wrapper. What should I do here?

It looks like the problem is your Service::Future is missing the Send constraint. The future being returned in call is already Send so it will work with the simple change:

impl Service<hyper::Uri> for CustomTransporter {
    type Response = CustomResponse;
    type Error = hyper::http::Error;
    type Future = Pin<Box<dyn Future<Output = Result<Self::Response, Self::Error>> + Send>>;
                                                                                  // ^^^^
    ...

Your code has a few other errors: un-inferred vec!(), self: Pin<...> missing mut, CustomResponse should be pub...

You can specify the B of client by using inference:

let client: Client<CustomTransporter, hyper::Body> = Client::builder().build(connector);

Or by using the turbofish operator on build:

let client = Client::builder().build::<CustomTransporter, hyper::Body>(connector);

I don't know enough about creating custom hyper transports to know if its functional, but these fixes make it compile. Hopefully it helps you make progress.

kmdreko
  • 42,554
  • 6
  • 57
  • 106
  • `let client = Client::::builder().build(connector);` this does not work. What you meant by the defining the `B` parameter? – Guerlando OCs Feb 05 '21 at 13:06
  • https://github.com/lzunsec/rust_hyper_custom_transporter/tree/91adf9ade257ac929252d3ecd20111866cd32685 here are my new modifications. I only get the `B` error, even though I made `let client = Client::<(), Body>::builder().build(connector);` – Guerlando OCs Feb 05 '21 at 13:29
  • Thanks, I'm gonna wait to see if anyone can help me with making the fake request work. Currently it runs but I get an error. https://github.com/lzunsec/rust_hyper_custom_transporter/tree/334d0f160255082e40c76e7b96d11ef1d006bce1 – Guerlando OCs Feb 05 '21 at 16:42
  • new bounty for you even though I didn't find the problem lol – Gatonito Feb 19 '21 at 13:16