6

I'm trying to get cookies on the Spotify login page with C# and the HttpClient class. However, the CookieContainer is always empty when I know cookies are being set. I'm not sending any headers, but it should still give me the cookie(s) because when I send a GET request without any headers with python (requests module) I get the csrf token. Here's my code:

using System;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Collections;
using System.Web;

class Program
{
    static void Main()
    {
        Task t = new Task(MakeRequest);
        t.Start();
        Console.WriteLine("Getting cookies!");
        Console.ReadLine();
    }

    static async void MakeRequest()
    {
        CookieContainer cookies = new CookieContainer();
        HttpClientHandler handler = new HttpClientHandler();

        handler.CookieContainer = cookies;
        Uri uri = new Uri("https://accounts.spotify.com/en/login/?_locale=en-US&continue=https:%2F%2Fwww.spotify.com%2Fus%2Faccount%2Foverview%2F");
        HttpClient client = new HttpClient(handler);
        var response = await client.GetAsync(uri);
        string res = await response.Content.ReadAsStringAsync();
        Console.WriteLine(cookies.Count);
        foreach (var cookie in cookies.GetCookies(uri)) {
            Console.WriteLine(cookie.ToString());
        }
    }
}

It seems pretty simple to me, but the program always says there's 0 cookies. Anyone know what's going on?

Crisp Apples
  • 267
  • 1
  • 2
  • 13
  • 1
    What makes you think there are cookies being sent back? – DavidG Dec 23 '18 at 18:50
  • @DavidG I stated that I tested the same program in python and when I tried to get the cookies I got the csrf token. – Crisp Apples Dec 23 '18 at 19:18
  • @CrispApples did you enable cookies on the handler `handler.UseCookies`? – Nkosi Dec 23 '18 at 21:34
  • This may be a dupe https://stackoverflow.com/questions/14681144/httpclient-not-storing-cookies-in-cookiecontainer – Nkosi Dec 23 '18 at 22:48
  • I tested what you have and can reproduce the problem. How ever when I call other URLs withing the root domain you have listed, cookies are being returned in the container. – Nkosi Dec 23 '18 at 22:49
  • `UseCookies = true` is the default setting. I suggest using a static HttpClient. Possibly a `Lazy httpClient = new Lazy(() => { HttpClientHandler handler = new HttpClientHandler { (set a new CookieContainer here or use a static one -> to store the Cookies of all "sessions") } etc }`. IMO, it works much better in all situations. – Jimi Dec 23 '18 at 23:22
  • If using FW 4.7.2, the HttpClientHandler also supports the SslProtocols property, which allows to set the SSL protocols directly through the handler (besides the `ServerCertificateCustomValidationCallback`). Required, when a connection uses `Https`, which might also be the problem here. – Jimi Dec 23 '18 at 23:28
  • @Jimi I’ll be sure to try both of those! – Crisp Apples Dec 24 '18 at 01:58

3 Answers3

5

You need to enable the use of cookies using HttpClientHandler.UseCookies Property

public bool UseCookies { get; set; }

Gets or sets a value that indicates whether the handler uses the CookieContainer property to store server cookies and uses these cookies when sending requests.

//...

CookieContainer cookies = new CookieContainer();
HttpClientHandler handler = new HttpClientHandler();
handler.CookieContainer = cookies;
handler.UseCookies = true; //<-- Enable the use of cookies.

//...
Nkosi
  • 235,767
  • 35
  • 427
  • 472
  • Just tried this, same result... No cookies were returned. Do you think it’s because HttpClient doesn’t account for cookies in the Set-Cookie response header? Take a look at my answer. – Crisp Apples Dec 23 '18 at 21:54
  • @CrispApples I tested this going to `https://spotify.com` and got back cookies. same `Set-Cookie` header with multiple entries. container seems empty for the original URL you provided – Nkosi Dec 23 '18 at 22:29
  • 2
    @CrispApples I also tested it with multiple URL within the spotify domain, the cookie container is empty for any calls with `accounts.spotify.com` even with the `Set-Cookie` header present – Nkosi Dec 23 '18 at 22:42
  • That’s weird. I got cookies back on http://www.spotify.com too. Maybe accounts.spotify.com’s Set-Cookie header isn’t formatted right or something? Either way, I’d say this question is solved because I can just extract the csrf token from the Set-Cookie response header even if HttpClient doesn’t pick it up itself. – Crisp Apples Dec 23 '18 at 23:30
  • 1
    Even enabling UseCookies to true, did not receive cookies – M. Paul Jan 23 '20 at 08:07
1

I tried to write the response headers to the console with Console.WriteLine(response.Headers) and a Set-Cookie header with the csrf token was printed to the console. So it seems that HttpClient doesn’t count cookies in this header as actual cookies, thus not adding these said cookies to the CookieContainer.

Crisp Apples
  • 267
  • 1
  • 2
  • 13
  • I am encountering the same issue but with a different URL it does detect cookies being set via headers. – sirdank May 16 '19 at 15:07
0

Make sure you are decompression the response automatically. See my answer here: https://stackoverflow.com/a/74750572/1158313

An example of what I use:

var clientHandler = new HttpClientHandler {
    AllowAutoRedirect = true,
    UseCookies = true,
    CookieContainer = cookieContainer,
    AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate, // <--
};
Nathan Goings
  • 1,145
  • 1
  • 15
  • 33