We are working on a project similar to CookieBot as part of a larger project to protect personal data and information of web users. The aim is to obtain comprehensive information about tracking and profiling techniques so that the user can decide for themselves whether to allow their use. But prior owner of website has to know it.
How can we access a website in such a way that we can intercept requests and cookies?
You could try with puppeteer which allows you to script Chrome (or Chromium) via JavaScript.
In this example:
// pptr.js
const puppeteer = require('puppeteer-core');
module.exports = async () => {
const browser = await puppeteer.connect({ browserWSEndpoint: 'ws://localhost:3001' });
const page = await browser.newPage();
page.on('request', req => {
console.log(`request: ${req.url()}`);
});
await page.goto('https://stackoverflow.com/q/64901406/1244884');
const cookies = await page.cookies();
console.log('Cookies:');
cookies.forEach(cookie => {
console.log(`${cookie.name}\t${cookie.value}`);
});
await browser.close();
};
Which you can run as follow:
node -p -e 'require("./pptr.js")()'
Output:
request: https://stackoverflow.com/q/64901406/1244884
request: https://stackoverflow.com/questions/64901406/how-to-capture-trackers-like-cookies-pixels-etc-used-on-a-remote-website
request: https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js
request: https://cdn.sstatic.net/Js/stub.en.js?v=e10313546043
request: https://cdn.sstatic.net/Shared/stacks.css?v=3b16a418cc4c
request: https://cdn.sstatic.net/Sites/stackoverflow/primary.css?v=8cf2d7217e38
request: https://www.gravatar.com/avatar/79435cc2571ffa23dfcef703f8d762e0?s=32&d=identicon&r=PG
request: https://www.gravatar.com/avatar/f7c922ac3a9edef38d31baf959088f77?s=32&d=identicon&r=PG&f=1
request: https://cdn.sstatic.net/Img/unified/sprites.svg?v=fcc0ea44ba27
request: https://cdn.sstatic.net/Img/favicons-sprite16.png?v=f4676f10d215
request: https://cdn.sstatic.net/clc/clc.min.js?v=d7c7e62bd2f5
request: https://www.googletagservices.com/tag/js/gpt.js
request: https://www.google-analytics.com/analytics.js
request: https://secure.quantserve.com/quant.js
request: https://sb.scorecardresearch.com/beacon.js
request: https://cdn.sstatic.net/Js/full-anon.en.js?v=2d9c6897045f
request: https://cdn.sstatic.net/clc/styles/clc.min.css?v=83419f27e8fa
request: https://cdn.sstatic.net/Js/post-validation.en.js?v=2589ce0a8248
request: https://stackoverflow.com/posts/64901406/ivc/6697?_=1605741791032
request: https://cdn.sstatic.net/Img/unified/wmd-buttons.svg?v=c26278fc22d9
request: https://rules.quantcount.com/rules-p-c1rF4kxgLUzNc.js
request: https://pixel.quantserve.com/pixel;r=1818481277;rf=0;uht=2;a=p-c1rF4kxgLUzNc;url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F64901406%2Fhow-to-capture-trackers-like-cookies-pixels-etc-used-on-a-remote-website;fpan=1;fpa=P0-221458664-1605741791402;ns=0;ce=1;qjs=1;qv=3364aec3-20201006003021;cm=;gdpr=0;ref=;d=stackoverflow.com;je=0;sr=800x600x24;dst=0;et=1605741791401;tzo=0;ogl=type.website%2Curl.https%3A%2F%2Fstackoverflow%252Ecom%2Fquestions%2F64901406%2Fhow-to-capture-trackers-like-cookie%2Csite_name.Stack%20Overflow%2Cimage.https%3A%2F%2Fcdn%252Esstatic%252Enet%2FSites%2Fstackoverflow%2FImg%2Fapple-touch-icon%402%252Epng%3Fv%3D73d79a8%2Ctitle.How%20to%20capture%20trackers%20like%20cookies%252C%20pixels%252C%20etc%252E%252C%20used%20on%20a%20remote%20website%3F%2Cdescription.We%20are%20working%20on%20a%20project%20similar%20to%20CookieBot%20as%20part%20of%20a%20larger%20project%20to%20
Cookies:
__qca P0-221458664-1605741791402
prov d91ccb9e-35d8-78ff-61b2-2c157d6d65c7