How do I configure a hyper 0.11.x client variable that is polymorphic over the connector?

Question

I can't seem to figure out how to get Rust to accept a client and proxied client in the same variable. While I am still new to Rust, I have a basic understanding of programming. So far I have tried structs (but no impl's though), type casting, uninitialized variables, but nothing is working.

extern crate futures;
extern crate hyper;
extern crate hyper_proxy;
extern crate stopwatch;
extern crate tokio_core;

use futures::{Future, Stream};
use hyper::client::HttpConnector;
use hyper::Client;
use hyper_proxy::{Intercept, Proxy, ProxyConnector};
use tokio_core::reactor::Core;

fn main() {
    let use_proxy = true;
    let proxy_uri: Option<String> = Some("http://localhost:8118".to_owned());

    let mut core = Core::new().unwrap();
    let handle = core.handle();
    let mut proxy = None;
    // looking for polymorphic variable that works with both proxyed and unproxyed hyper clients
    let mut client: hyper::Client<hyper::client::HttpConnector, hyper::Body>;

    if use_proxy && proxy_uri.is_some() {
        println!("Using proxy: {}", proxy_uri.unwrap().as_str());
        proxy = Some({
            let proxy_uri = proxy_uri.unwrap().parse().unwrap();
            let mut proxy = Proxy::new(Intercept::All, proxy_uri);
            let connector = HttpConnector::new(4, &handle);
            let proxy_connector = ProxyConnector::from_proxy(connector, proxy).unwrap();
            proxy_connector
        });
        client = Client::configure()
            .connector(proxy.clone().unwrap())
            .build(&handle);
    } else {
        client = Client::configure()
            .connector(HttpConnector::new(4, &handle))
            .build(&handle);
    }

    // use hyper client below
}

[dependencies]
futures = "0.1.21"
hyper = "0.11.27"
tokio-core = "0.1.17"
hyper-proxy = "0.4.1"
stopwatch = "0.0.7"

I have made a GitHub repo of all the files.

I get this error when trying to compile:

 error[E0308]: mismatched types
  --> src/main.rs:32:18
   |
32 |           client = Client::configure()
   |  __________________^
33 | |             .connector(proxy.clone().unwrap())
34 | |             .build(&handle);
   | |___________________________^ expected struct `hyper::client::HttpConnector`, found struct `hyper_proxy::ProxyConnector`
   |
   = note: expected type `hyper::Client<hyper::client::HttpConnector, _>`
              found type `hyper::Client<hyper_proxy::ProxyConnector<hyper::client::HttpConnector>, _>`

If there is a better approach to this, I would also like to know about it.

Shepmaster · Accepted Answer · 2018-06-11T14:38:31.430

This solution is not pretty, but it does work.

We start by creating an enum to handle the two cases:

enum ProxyOrNotConnector {
    Proxy(ProxyConnector<HttpConnector>),
    Not(HttpConnector),
}

This enum can be a single type representing both cases. Constructing it is straightforward with a match statement:

let http_connector = HttpConnector::new(4, &handle);

let connector = match (proxy_uri, use_proxy) {
    (Some(proxy_uri), true) => {
        println!("Using proxy: {}", proxy_uri);
        let proxy_uri = proxy_uri.parse().unwrap();
        let mut proxy = Proxy::new(Intercept::All, proxy_uri);
        let proxy_connector = ProxyConnector::from_proxy(http_connector, proxy).unwrap();
        ProxyOrNotConnector::Proxy(proxy_connector)
    }
    _ => ProxyOrNotConnector::Not(http_connector),
};

We can then create a Client using this connector:

let client = Config::default().connector(connector).build(&handle);

This won't work until we've implemented Connect for our enum. There's a blanket implementation of Connect for any type that implements Service in the correct manner, so we go that route:

impl Service for ProxyOrNotConnector {
    type Request = Uri;
    type Response = Box<AsyncRw>;
    type Error = io::Error;

    type Future = Box<Future<Item = Self::Response, Error = Self::Error>>;

    fn call(&self, req: Self::Request) -> Self::Future {
        match self {
            ProxyOrNotConnector::Proxy(p) => {
                let x = p.call(req);
                let y = x.map(|y| Box::new(y) as Box<AsyncRw>);
                Box::new(y)
            }
            ProxyOrNotConnector::Not(n) => {
                let x = n.call(req);
                let y = x.map(|y| Box::new(y) as Box<AsyncRw>);
                Box::new(y)
            }
        }
    }
}

We use multiple trait objects to perform runtime polymorphism: one for the future returned by connecting and another for each value yielded by that future.

Complete code:

extern crate futures;
extern crate hyper;
extern crate hyper_proxy;
extern crate tokio_core;
extern crate tokio_io;

use futures::Future;
use hyper::{
    client::{Config, HttpConnector, Service},
    Uri,
};
use hyper_proxy::{Intercept, Proxy, ProxyConnector};
use std::io;
use tokio_core::reactor::Core;
use tokio_io::{AsyncRead, AsyncWrite};

trait AsyncRw: AsyncWrite + AsyncRead {}
impl<T> AsyncRw for T where T: AsyncWrite + AsyncRead {}

enum ProxyOrNotConnector {
    Proxy(ProxyConnector<HttpConnector>),
    Not(HttpConnector),
}

impl Service for ProxyOrNotConnector {
    type Request = Uri;
    type Response = Box<AsyncRw>;
    type Error = io::Error;

    type Future = Box<Future<Item = Self::Response, Error = Self::Error>>;

    fn call(&self, req: Self::Request) -> Self::Future {
        match self {
            ProxyOrNotConnector::Proxy(p) => {
                let x = p.call(req);
                let y = x.map(|y| Box::new(y) as Box<AsyncRw>);
                Box::new(y)
            }
            ProxyOrNotConnector::Not(n) => {
                let x = n.call(req);
                let y = x.map(|y| Box::new(y) as Box<AsyncRw>);
                Box::new(y)
            }
        }
    }
}

fn main() {
    let mut core = Core::new().unwrap();
    let handle = core.handle();

    let proxy_uri = Some("http://127.0.0.1");
    let use_proxy = true;

    let http_connector = HttpConnector::new(4, &handle);

    let connector = match (proxy_uri, use_proxy) {
        (Some(proxy_uri), true) => {
            println!("Using proxy: {}", proxy_uri);
            let proxy_uri = proxy_uri.parse().unwrap();
            let mut proxy = Proxy::new(Intercept::All, proxy_uri);
            let proxy_connector = ProxyConnector::from_proxy(http_connector, proxy).unwrap();
            ProxyOrNotConnector::Proxy(proxy_connector)
        }
        _ => ProxyOrNotConnector::Not(http_connector),
    };

    let client = Config::default().connector(connector).build(&handle);
    let g = client.get("http://127.0.0.1/".parse().unwrap());

    let x = core.run(g).unwrap();
    println!("{:?}", x);
}

I don't actually have a proxy lying around to test with, but it does compile and report a reasonable error about not being able to connect.

score 0 · Answer 2 · answered Jun 11 '18 at 14:58

Building on @Shepmaster's answer, the code below has been tested using Privoxy.

We use the double if let clauses to first extract the type hidden by ProxyOrNotConnector and then modify the http request to use the proxy.

if let ProxyOrNotConnector::Proxy(x) = connector.clone() {
    if let Some(headers) = x.http_headers(&uri) {
        req.headers_mut().extend(headers.iter());
        req.set_proxy(true);
    }
}

Complete code:

extern crate futures;
extern crate hyper;
extern crate hyper_proxy;
extern crate tokio_core;
extern crate tokio_io;

use futures::{Future, Stream};
use hyper::{Chunk, Method, Request, Uri, client::{Config, HttpConnector, Service}};
use hyper_proxy::{Intercept, Proxy, ProxyConnector};
use std::io;
use tokio_core::reactor::Core;
use tokio_io::{AsyncRead, AsyncWrite};

trait AsyncRw: AsyncWrite + AsyncRead {}
impl<T> AsyncRw for T
where
    T: AsyncWrite + AsyncRead,
{
}

#[derive(Clone)]
enum ProxyOrNotConnector {
    Proxy(ProxyConnector<HttpConnector>),
    Not(HttpConnector),
}

impl Service for ProxyOrNotConnector {
    type Request = Uri;
    type Response = Box<AsyncRw>;
    type Error = io::Error;

    type Future = Box<Future<Item = Self::Response, Error = Self::Error>>;

    fn call(&self, req: Self::Request) -> Self::Future {
        match self {
            ProxyOrNotConnector::Proxy(p) => {
                let x = p.call(req);
                let y = x.map(|y| Box::new(y) as Box<AsyncRw>);
                Box::new(y)
            }
            ProxyOrNotConnector::Not(n) => {
                let x = n.call(req);
                let y = x.map(|y| Box::new(y) as Box<AsyncRw>);
                Box::new(y)
            }
        }
    }
}

fn main() {
    let proxy_uri = Some("http://localhost:8118");
    let use_proxy = true;
    let uri: Uri = "http://httpbin.org/ip".parse().unwrap();

    let mut core = Core::new().unwrap();
    let handle = core.handle();
    let http_connector = HttpConnector::new(4, &handle);

    let connector = match (proxy_uri, use_proxy) {
        (Some(proxy_uri), true) => {
            println!("Using proxy: {}", proxy_uri);
            let proxy_uri = proxy_uri.parse().unwrap();
            let proxy = Some(Proxy::new(Intercept::All, proxy_uri));
            let proxy_connector =
                ProxyConnector::from_proxy(http_connector, proxy.unwrap()).unwrap();
            ProxyOrNotConnector::Proxy(proxy_connector)
        }
        _ => ProxyOrNotConnector::Not(http_connector),
    };

    let client = Config::default().connector(connector.clone()).build(&handle);
    let mut req: hyper::Request;
    match use_proxy {
        true => {
            req = Request::new(Method::Get, uri.clone());
            if let ProxyOrNotConnector::Proxy(x) = connector.clone() {
                if let Some(headers) = x.http_headers(&uri) {
                    req.headers_mut().extend(headers.iter());
                    req.set_proxy(true);
                }
            }
        }
        false => req = Request::new(Method::Get, uri.clone()),
    }

    let future_http = client
        .request(req)
        .and_then(|res| res.body().concat2())
        .map(move |body: Chunk| ::std::str::from_utf8(&body).unwrap().to_string());

    let x = core.run(future_http).unwrap();
    println!("{:?}", x);
}

How do I configure a hyper 0.11.x client variable that is polymorphic over the connector?

2 Answers2