0

folks! What's the real difference between basic and --headless mode in chromedriver used with selenium? I ask because I tried to track some prices at adidas.com, and when I set up my driver like:

chrome_options = webdriver.ChromeOptions()
browser = webdriver.Chrome(ChromeDriverManager().install(), options=chrome_options)
browser.get(url)

browser opens on my computer and it's all fine.

But when I'm trying to execute code below:

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
browser = webdriver.Chrome(ChromeDriverManager().install(), options=chrome_options)
browser.get(url)

the response is:

'<html lang="en"><head>\n    <meta charset="utf-8">\n    <meta name="viewport"
content="width=device-width, initial-scale = 1.0, maximum-scale=1.0, user- 
scalable=no">\n    <title>adidas</title>\n\n    <link 
href="/WAFfailoverassets/style.css" rel="stylesheet">\n</head>\n<body>\n\n<main 
class="content-wrapper">\n    <header class="top">\n        <div 
class="logo">adidas</div>\n        <span 
id="ctl00_mainContentPlaceHolder_lblInvalidRequest">\n<br> Reference Error: 
18.6c85655f.1629459608.2c05a32c </span>\n        <h1 class="title">Unfortunately we 
are unable to give you access to our site at this time. </h1>\n        <p>A security 
issue was automatically identified, when you tried to access the website.</p>\n    
</header>\n\n    <div class="columns">\n\n        <section class="column column-1">\n            
<h3>What could have caused this?</h3>\n            <p>During high-traffic product 
releases we have extra security in place to prevent bots entering our site. We do 
this to protect customers and to give everyone a fair chance of getting the sneakers. 
Something in your setup must have triggered our security system, so we cannot allow 
you onto the site.</p>\n        </section>\n\n        <section class="column column- 
2">\n            <h3>How can I resolve this?</h3>\n            <p>Please try 
refreshing or accessing our website from a different browser or another device. You 
could also check if there are any scripts, like ad blockers, running in your browser 
and disable them. </p>\n            <p>If this does not help, the issue might be 
caused by malware. If you\'re on a personal connection you can run an anti-virus scan 
on your device to make sure it\'s not affected by malware. If you\'re on a shared 
network, you could ask the system administrator to run a scan looking for 
misconfigured or infected devices across the network.</p>\n        </section>\n\n    
</div>\n\n    <small class="error">HTTP 403 - 
Forbidden</small>\n\n</main>\n\n\n\n</body></html>' 

So the question is how can website detect if its code really renders and how can I deal with that if I want to run my code on server?

  • 1
    What print statement did you use ? Also in headless mode you won't see any UI, and code execution will be bit faster, also you do not need driver file as well – cruisepandey Aug 20 '21 at 12:08
  • They have bot detection technique installed in their system. So that their system can fight against spammers. – cruisepandey Aug 20 '21 at 12:12
  • @cruisepandey I see that, but why their bot detection doesn't work if I don't use --headless mode ? Seems strange to me. – Denis Polikarpov Aug 20 '21 at 13:17
  • I guess cause Selenium is not fast enough or may be you have sleep method a lot of time in your code – cruisepandey Aug 20 '21 at 13:35

0 Answers0