Chromedp sending request with cookies

Question

I was trying to open a checkout page with chromedp library but it doesn't receive the cookies I'm sending. I tried network.SetCookies()and network.SetCookie() in a loop but it doesn't work. It compiles and run without errors. Help is appreciated, here's the code:

opts := append(chromedp.DefaultExecAllocatorOptions[:], chromedp.Flag("headless", false))
actx, cancel := chromedp.NewExecAllocator(context.Background(), opts...)
ctx, cancel := chromedp.NewContext(actx)

// Call cancel() to close Chrome on some condition.
if false {
    cancel()
}

task := chromedp.Tasks{
    network.Enable(),
    chromedp.ActionFunc(func(ctx context.Context) error {
        cookieJar := getCookies(client)
        var cookiesParam []*network.CookieParam
        for _, v := range cookieJar {
            fmt.Println(v.Name, ":"+v.Value)
            cookiesParam = append(cookiesParam, &network.CookieParam{Name: v.Name, Value: v.Value})
        }
        network.SetCookies(cookiesParam)
        return nil
    }),
    chromedp.Navigate(res.Request.URL.String()),
}

// Run task.
err := chromedp.Run(ctx, task)
if err != nil {
    log.Fatal(err)
}

EDIT: I tried the example given by @zachyoung but when I try to send any kind of cookie, it doesn't work. Here's the code:

// Command cookie is a chromedp example demonstrating how to set a HTTP cookie
// on requests.
package main

import (
    "context"
    "log"
    "time"

    "github.com/chromedp/cdproto/cdp"
    "github.com/chromedp/cdproto/network"
    "github.com/chromedp/chromedp"
)

func main() {

    // create context
    ctx, cancel := chromedp.NewContext(context.Background())
    defer cancel()

    // run task list
    var res string
    err := chromedp.Run(ctx, setcookies("https://en.afew-store.com/", &res,
        "cookie1", "value1",
        "cookie2", "value2",
    ))
    if err != nil {
        log.Fatal(err)
    }
    log.Printf("chrome received cookies: %s", res)
}

// setcookies returns a task to navigate to a host with the passed cookies set
// on the network request.
func setcookies(host string, res *string, cookies ...string) chromedp.Tasks {
    if len(cookies)%2 != 0 {
        panic("length of cookies must be divisible by 2")
    }
    return chromedp.Tasks{
        chromedp.ActionFunc(func(ctx context.Context) error {
            // create cookie expiration
            expr := cdp.TimeSinceEpoch(time.Now().Add(180 * 24 * time.Hour))
            // add cookies to chrome
            for i := 0; i < len(cookies); i += 2 {
                err := network.SetCookie(cookies[i], cookies[i+1]).
                    WithExpires(&expr).
                    WithDomain("https://en.afew-store.com/").
                    WithHTTPOnly(true).
                    Do(ctx)
                if err != nil {
                    return err
                }
            }
            return nil
        }),
        // navigate to site
        chromedp.Navigate(host),
        // read the returned values
        chromedp.Text(`#result`, res, chromedp.ByID, chromedp.NodeVisible),
        // read network values
        chromedp.ActionFunc(func(ctx context.Context) error {
            cookies, err := network.GetAllCookies().Do(ctx)
            if err != nil {
                return err
            }

            for i, cookie := range cookies {
                log.Printf("chrome cookie %d: %+v", i, cookie)
            }

            return nil
        }),
    }
}

Do you need to set some fields on your cookiesParam, like domain, in this example: https://github.com/chromedp/examples/blob/master/cookie/main.go? — Zach Young, Oct 17 '22 at 20:51
Still doesn't work even adding domain to the cookie as in the example. — TOMMASO26175, Oct 18 '22 at 08:45
@TOMMASO26175 You should make a runable self-contained demo to show what does not work. A demo like the one linked by Zach Young is helpful. — Zeke Lu, Oct 20 '22 at 03:05
Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. — mohammad mobasher, Oct 20 '22 at 07:54

Zeke Lu · Accepted Answer · 2023-07-12T01:25:57.013

https://en.afew-store.com/ is not a domain, you should replace it with en.afew-store.com.

- WithDomain("https://en.afew-store.com/").
+ WithDomain("en.afew-store.com").

And there is not an #result element on the page, so chromedp.Text("#result", res, chromedp.ByID, chromedp.NodeVisible) never returns. Here is a modified demo that works:

package main

import (
    "context"
    "log"
    "time"

    "github.com/chromedp/cdproto/cdp"
    "github.com/chromedp/cdproto/network"
    "github.com/chromedp/cdproto/storage"
    "github.com/chromedp/chromedp"
)

func main() {
    ctx, cancel := chromedp.NewContext(context.Background())
    defer cancel()

    err := chromedp.Run(ctx,
        chromedp.ActionFunc(func(ctx context.Context) error {
            expr := cdp.TimeSinceEpoch(time.Now().Add(180 * 24 * time.Hour))
            cookies := []string{"cookie1", "value1", "cookie2", "value2"}
            for i := 0; i < len(cookies); i += 2 {
                err := network.SetCookie(cookies[i], cookies[i+1]).
                    WithExpires(&expr).
                    WithDomain("en.afew-store.com").
                    WithHTTPOnly(true).
                    Do(ctx)
                if err != nil {
                    return err
                }
            }
            return nil
        }),
        chromedp.Navigate("https://en.afew-store.com/"),
        chromedp.ActionFunc(func(ctx context.Context) error {
            cookies, err := storage.GetCookies().Do(ctx)
            if err != nil {
                return err
            }

            for i, cookie := range cookies {
                log.Printf("chrome cookie %d: %+v", i, cookie)
            }

            return nil
        }),
    )
    if err != nil {
        log.Fatal(err)
    }
}

Hi @AndrewArrow, I just checked `github.com/chromedp/cdproto@v0.0.0-20230625224106-7fafe342e117` and the `WithHTTPOnly` method is still there. But `network.GetAllCookies()` was replaced with `storage.GetCookies()` (see [chromedp/examples/pull/37](https://github.com/chromedp/examples/pull/37)). I have just updated the answer to use `storage.GetCookies()`. — Zeke Lu, Jul 12 '23 at 01:24

score 0 · Answer 2 · answered Oct 19 '22 at 14:52

I faced the same issue and I added the url.

You could add an error check to see what happens.

        cdp.ActionFunc(func(ctx context.Context) error {
        for _, cookie := range cookies {
            err := network.SetCookie(cookie.Name, cookie.Value).
                WithDomain(cookie.Domain).
                WithURL(`https://www.blablabla.com/blu`).
                Do(ctx)

            if err != nil {
                panic(err)
            }
        }
        return nil
    }),

nope, it doesn't work, 0 cookies sent even if i set the URL – TOMMASO26175 Oct 19 '22 at 15:10 — TOMMASO26175, Oct 19 '22 at 15:10

Chromedp sending request with cookies

2 Answers2