0

I'm using the cloudscraper library on the URL https://monroe.county-taxes.com/public/search/property_tax?search_query=1573108&redirect=1573108

The site redirects to https://monroe.county-taxes.com/public/real_estate/parcels/1573108/bills?parcel=ef360da9-e509-11eb-9467-005056818710&qid=27a7b62cf21f5b6bf80d9b106b6e286f on my browser, which is the target site that I want to extract the data from. However, the responses from network tab are all 200 OK and I could not find a trace of anything that hints to this URL from the one above. What might be happening here?

Minimal reproducible example:

import cloudscraper

scraper = cloudscraper.create_scraper()
r = scraper.get("https://monroe.county-taxes.com/public/search/property_tax?search_query=1573108&redirect=1573108")
print(r.text)

Edit: I have tried this solution but due to Selenium's subpar speed, I prefer to leave it as the very last resort and looking for a different way.

Mint
  • 428
  • 5
  • 18
  • 2
    It seems that javascript is performing the redirect. Which intern doesn't yield a 3xx code in the network tab – ShadowCrafter_01 Jun 09 '23 at 07:55
  • 1
    Does this answer your question? [How to get the redirected URL in Web Scraping?](https://stackoverflow.com/questions/60576039/how-to-get-the-redirected-url-in-web-scraping) – tevemadar Jun 09 '23 at 13:52
  • @tevemadarUnfortunately, no. That was one of the first things I tried but it didn't follow the redirect hence `.history` is empty. Selenium might work but I am constrained to not use it due to its speed and the sheer quantity of data to scrape. – Mint Jun 09 '23 at 14:01

0 Answers0