How to get the cookie from a GET response?

Question

I am writing a function that makes a GET request to a website and returns the response cookie:

extern crate futures;
extern crate hyper;
extern crate tokio_core;

use tokio_core::reactor::Core;
use hyper::Client;
use std::error::Error;
use hyper::header::Cookie;
use futures::future::Future;

fn get_new_cookie() -> Result<String, Box<Error>> {
    println!("Getting cookie...");
    let core = Core::new()?;
    let client = Client::new(&core.handle());
    println!("Created client");

    let uri = "http://www.cnn.com".parse().expect("Cannot parse url");
    println!("Parsed url");
    let response = client.get(uri).wait().expect("Cannot get url.");
    println!("Got response");
    let cookie = response
        .headers()
        .get::<Cookie>()
        .expect("Cannot get cookie");
    println!("Cookie: {}", cookie);

    Ok(cookie)
}

fn main() {
    println!("{:?}", get_new_cookie());
}

This doesn't work; it is stuck on the client.get(...) string. The output I'm getting is:

Getting cookie...
Created client
Parsed url

and after that nothing happens.

What am I doing wrong and how I can change it so it'd work?

Answer 1

See the documentation for the wait method:

Note: This method is not appropriate to call on event loops or similar I/O situations because it will prevent the event loop from making progress (this blocks the thread). This method should only be called when it's guaranteed that the blocking work associated with this future will be completed by another thread.

Future::wait is already deprecated in the tokio-reform branch.

I'd recommend to design the full application to deal with async concepts (i.e. get_new_cookie should take a Handle and return a Future, not allocating its own event loop).

You could run the request with Core::run like this:

let response = core.run(client.get(uri)).expect("Cannot get url.");

Answer 2

As Stefan points out, by calling wait, you are putting the thread to sleep until the future has completed. However, that thread needs to run the event loop, so you've just caused a deadlock. Using Core::run is more correct.

As Francis Gagné points out, the "Cookie" header is used to send a cookie to the server. SetCookie is used to send a cookie to the client. It also returns a vector of all the cookies together:

fn get_new_cookie() -> Result<String, Box<Error>> {
    println!("Getting cookie...");
    let mut core = Core::new()?;

    let client = Client::new(&core.handle());
    println!("Created client");

    let uri = "http://www.cnn.com".parse().expect("Cannot parse url");
    println!("Parsed url");

    let response = core.run(client.get(uri)).expect("Cannot get url.");
    println!("Got response");

    let cookie = response
        .headers()
        .get::<SetCookie>()
        .expect("Cannot get cookie");
    println!("Cookie: {:?}", cookie);

    Ok(cookie.join(","))
}

---

However, if you only want a synchronous API, use Reqwest instead. It is built on top of Hyper:

extern crate reqwest;

use std::error::Error;
use reqwest::header::SetCookie;

fn get_new_cookie() -> Result<String, Box<Error>> {
    let response = reqwest::get("http://www.cnn.com")?;

    let cookies = match response.headers().get::<SetCookie>() {
        Some(cookies) => cookies.join(","),
        None => String::new(),
    };

    Ok(cookies)
}

fn main() {
    println!("{:?}", get_new_cookie());
}

Answer 3

reqwest 0.11 (and perhaps earlier) update

In the get_new_cookie function, I believe the code snippet to retrieve the cookies from a reqwest::Response goes something like:

// returns Option<&HeaderValue>

response.headers().get(http::header::SET_COOKIE)