Scrapy AttributeError: 'NoneType' object has no attribute 'css'

Question

I'm trying to scrape details from this page using Scrapy: link

This is the command I am using to pull the Title but it is throwing an error:

response.css('#intitule > div.nom_fugitif::text').extract_first()

This is the error:

AttributeError Traceback (most recent call last) in ()

----> 1 response.css('#intitule > div.nom_fugitif::text').extract_first()

AttributeError: 'NoneType' object has no attribute 'css'

It says the div has no CSS but I'm not sure what that means because it does have a CSS class.

@araraonline I'm really new to this. Can you explain it more please? — TinyTiger, Nov 01 '17 at 12:38
The error doesn't say that HTML element doesn't have CSS class, it says that `response` object is `None`, hence doesn't have a method `css` that you call in `response.css(...)`. Please show complete code so we can examine. — Tomáš Linhart, Nov 01 '17 at 12:39
Type `print(response)` and you will see it probably is `None`. — araraonline, Nov 01 '17 at 12:39
It just started working again. No idea why! But thanks for the help regardless :) — TinyTiger, Nov 01 '17 at 12:42
It could be that `ROBOTSTXT_OBEY` is set to `True` in your spider's settings.py: https://stackoverflow.com/a/43994382/1011724 — Dan, Jan 07 '18 at 16:53

score 0 · Answer 1 · answered Nov 01 '17 at 12:40

0

NoneType means you're actually getting None in your response. You can check with

if response is not None:

answered Nov 01 '17 at 12:40

user8836893

56
5

Scrapy AttributeError: 'NoneType' object has no attribute 'css'

1 Answers1