-1

i need help for extract the links in the page: https://www.remax.pt/comprar-empreendimentos?searchQueryState={%22page%22:1,%22sort%22:{%22fieldToSort%22:%22PublishDate%22,%22order%22:1}}

HTML code Results from shell

Eduardo Henrique
  • 9
  • 1

1 Answers1

0

You could shorten it, you don't have to target from the top element to your target. It's easier to debug then.

response.css('div.developments-search-details-component a::attr(href)').get()

You can change this to Xpath if you prefer that. But usually when you try to target an element and it returns null or empty list it's because of a typo or because that element is dynamically rendered after page load.

To debug I'll usually start at a higher element in the tree and see if that exists. In this case you could try:

response.css('div.developments-search-details-component').get()

first and see if that works.

Felix Eklöf
  • 3,253
  • 2
  • 10
  • 27
  • Thank you Felix. I discovered with the element is dynamically rendered in javascript. I dont know how get it – Eduardo Henrique Jan 12 '21 at 15:05
  • You can look at the last part of the answer i gave here. https://stackoverflow.com/questions/65623157/page-with-anti-scraping-protection-in-the-code/65625805#65625805 – Felix Eklöf Jan 12 '21 at 15:33