Symfony DomCrawler Equivalent in Python

Asked Apr 01 '16 at 20:43

Active Apr 01 '16 at 20:46

Viewed 292 times

Are there any python libraries out there that work like Symfony's DomCrawler? I would like to use CSS selectors in order to extract data from some HTML.

The functionality I'm most interested in would be something similar to this snippet of PHP:

$crawler = new Crawler($html);
$crawler = $crawler->filter('body > p');

If there is any default functionality in python that allows this to be accomplished easily, that would work as well.

edited Apr 01 '16 at 20:46

asked Apr 01 '16 at 20:43

RattleyCooper

4,997
5
27
43

1

Have you looked at scrapy? http://scrapy.org/ – Padraic Cunningham Apr 01 '16 at 20:44
1

@PadraicCunningham, I forgot about scrapy, but it looks like it does use CSS selectors or XPath expressions, so this will work perfectly. Thanks! – RattleyCooper Apr 01 '16 at 20:47
1

Yes, I have used it myself and I would highly recommend it. – Padraic Cunningham Apr 01 '16 at 20:50
@PadraicCunningham, Do you happen to know if it breaks with malformed HTML, or does it do pretty well at recovering? – RattleyCooper Apr 01 '16 at 20:51
1

It uses lxml to build the Selectors, lxml handles broken html, to how bad I don't really know. All I can say is I never had issues. http://doc.scrapy.org/en/latest/topics/selectors.html – Padraic Cunningham Apr 01 '16 at 20:59
1

@PadraicCunningham, ok cool. thanks for the additional info Padraic. – RattleyCooper Apr 01 '16 at 21:06

Symfony DomCrawler Equivalent in Python

0 Answers0