Scrapy Ignore values after
tag

Question

HTML:

<span class="number"> - Sep 15, 1991<br><strong>Some Number: </strong>123, 123, 145</span>

Scrapy:

 samples = response.css('ul li.somthing')
    for sample in samples:
        loader = ItemLoader(item=CatelogItem(), selector=sample)
        loader.add_css('some', 'span.number::text')
        yield loader.load_item()

Item.py

some = Field(
    input_processor=MapCompose(str.strip),
    output_processor=Join()
)

Result

- Sep 15, 1991

Expected

- Sep 15, 1991 Some Number: 123, 123, 145

Why is this behavior? how do i get the full value loaded in itemloader?

you mean , loader.add_css('some', 'span.number::innerHtml') result is: The pseudo-element ::innerhtml is unknown — Satscreate, Sep 23 '20 at 10:21
https://stackoverflow.com/questions/28658047/how-to-get-innerhtml-of-a-node-using-scrapy-selector — Arundeep Chohan, Sep 23 '20 at 10:36
Yep.. Thanks a lot. that works like charm . Fix: loader.add_css('some', 'span.number *::text') — Satscreate, Sep 23 '20 at 10:40
I am just going to put that down and upvote and check mark the answer — Arundeep Chohan, Sep 23 '20 at 10:41

score 0 · Accepted Answer · answered Sep 23 '20 at 10:42

0

You needed to grab all the innerhtml instead of text which includes all of it's nested components.

loader.add_css('some', 'span.number *::text')

answered Sep 23 '20 at 10:42

Arundeep Chohan

9,779
5
15
32

Scrapy Ignore values after tag

1 Answers1

Scrapy Ignore values after
tag