1

I'm working on a project that scrapes content from a page on Amazon.co.uk. I can submit the login form successfully and I store the cookies in a CookieContainer, everything appears normal and I am logged into the site successfully. However, when I browse to certain pages it appears logged out, prompting the login screen again. I can never access the page programmatically.

I think I might be falling into the bug reported here which relates to Cookie Container with subdomains on .NET 4: https://connect.microsoft.com/VisualStudio/feedback/details/771651/cookiecontainer-subdomain-handling-issue-in-net-4-0#

Two workarounds exist on this link:

Workaround #1: after receiving response with Version=1 cookies, add them to new CookieContainer as >plain cookies, and then use this new container to make subsequent requests.

Workaround #2: after receiving response with Version=1 cookies for parent domain, add them to same >CookieContainer once again, now for subdomain.

I'm not sure I understand how to implement this though, has anyone experienced this before that could share a solution with me? I'm running .NET 4.0.

Thanks, Colin.

Community
  • 1
  • 1
Colin Brown
  • 589
  • 1
  • 8
  • 15

1 Answers1

2

I found the solution to this was to recreate the CookieContainer with each request and modify the Version 1 Cookies to 0:

CookieContainer newCookies = new CookieContainer();
newCookies.Add(new Uri("https://www.amazon.co.uk/"), new Cookie
            {
                Name = c.Name,
                Version = 0,
                Comment = c.Comment,
                CommentUri = c.CommentUri,
                Discard = c.Discard,
                Domain = c.Domain,
                Expired = c.Expired,
                Expires = c.Expires,
                HttpOnly = c.HttpOnly,
                Path = c.Path,
                Port = c.Port,
                Secure = c.Secure,
                Value = c.Value
            });

An annoying bug that MS is refusing to fix it seems. Hope this helps someone out!

Colin Brown
  • 589
  • 1
  • 8
  • 15