0

From terminal the command response.status returns 200.

From myfile.py the command requests.get(next_page, headers=headers) returns <Response [403]>.

I expected to get 200 from file. What can be wrong?

myfile.py:

import lxml.html as parser
import requests

headers = {'User-Agent': '"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36"'}
start_urls = [
#"https://www.vivareal.com.br/aluguel/?__vt=vt:b#banheiros=2&preco-ate=2500&preco-desde=1000&quartos=2"
"https://www.zapimoveis.com.br/aluguel/apartamentos/pe+recife/2-quartos/?__zt=ldt%3Ab&onde=,Pernambuco,Recife,,,,BR%3EPernambuco%3ENULL%3ERecife,-8.0522404,-34.9286096&banheiros=2&quartos=2&transacao=Aluguel&vagas=1&precoMaximo=2500&precoMinimo=1000&tipoUnidade=Residencial,Apartamento&tipo=Im%C3%B3ve^X^X^X^XsdsDSl%20usado&pagina=1"
]

next_page = start_urls[0]
print(next_page)
r = requests.get(next_page, headers=headers)
print(r)

h = parser.fromstring(r.text)
address = h.xpath('//p[contains(@class,"address")]/text()')
print(address)
Gallaecio
  • 3,620
  • 2
  • 25
  • 64
Cleber Marques
  • 426
  • 5
  • 20

1 Answers1

1

Looks like there is something isn't right with the User-agent request headers. Try this

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.76 Safari/537.36'}
Rohit Kumar
  • 669
  • 3
  • 9
  • 22