1

I have this web:

https://analisi.transparenciacatalunya.cat/es/Energia/Certificats-d-efici-ncia-energ-tica-d-edificis/j6ii-t3w2

and am trying to get the total number of rows from the table that is at the end of the web (1.341.412) . the selector is :

#app > div > div.container.landing-page-container > section.landing-page-section.dataset-preview > div.table-contents > div > div > div.socrata-visualization-container.loaded > div.socrata-pager > span.pager-label

The web loads elements via JS, so I´m trying to get this element via shell like this:

scrapy shell "http://localhost:8050/render.html?url="https://analisi.transparenciacatalunya.cat/es/Energia/Certificats-d-efici-ncia-energ-tica-d-edificis/j6ii-t3w2/data/""

inside the shell :

 fetch('https://analisi.transparenciacatalunya.cat/es/Energia/Certificats-d-efici-ncia-energ-tica-d-edificis/j6ii-t3w2')

response status is ok: 200

but I can´t find a way to get the text from span class=pager-label

could someone help me please...I can´t find the

response.css(' '::text).get()

I always get an empty response....

Maximiliano Vazquez
  • 196
  • 1
  • 2
  • 12
  • Did you use `view(response)` in the shell ? For instance, when I do `scrapy shell https://analisi.transparenciacatalunya.cat/es/Energia/Certificats-d-efici-ncia-energ-tica-d-edificis/j6ii-t3w2` when I launch `view(response)` the table you are interested in has trouble to be downloaded `Tenemos problemas para mostrar esta tabla.` maybe due to my python's environment, but I prefer to ask you if you checked by this way at first. – AvyWam Jul 01 '22 at 17:28
  • Yes. That´s not the problem. I used scrapy shell to check, but in the code I´n gettong an empty variable, so it´´s not pointing well the path – Maximiliano Vazquez Jul 05 '22 at 22:32

0 Answers0