2

I am using the requests library to parse the website but it returns the mobile version of the site. How can I get the HTML page of the desktop version?

import requests

sess = requests.Session() 
sess.get("https://google.com/")
sasha
  • 135
  • 11

2 Answers2

1

Try this:

from requests_html import HTMLSession
session = HTMLSession()
session.headers['user-agent'] = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36'
response = session.get(url, headers=session.headers)
huvdev4
  • 61
  • 5
0

I can see that google gets a User-Agent header specifying the browser, from the inspect element of my Mozilla I can see that google gets

User-Agent Mozilla/5.0 etc..

Maybe you can try to send a header tricking Google to think it's being accessed from a browser.

Mario Alla
  • 1
  • I tried "headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'} " but it did not help – sasha Apr 19 '22 at 08:23