27

What's my best bet for parsing HTML if I can't use BeautifulSoup or lxml? I've got some code that uses SGMLlib but it's a bit low-level and it's now deprecated.

I would prefer if it could stomache a bit of malformed HTML although I'm pretty sure most of the input will be pretty clean.

Andy Baker
  • 21,158
  • 12
  • 58
  • 71
  • 1
    If I may ask, why can't you use lxml, or BS? – Nick Presta Apr 04 '09 at 18:26
  • 1
    I was trying to avoid answers getting completely sidetracked. My reasons for avoiding BeautifulSoup are hugely debatable but I was saving that for another day! (My reasons for avoiding lxml are simple - a complete failure to install it on either Mac OSX or Linux :( – Andy Baker Apr 05 '09 at 09:27
  • 2
    Here is how to install lxml on Linux: `sudo apt-get install libxml2-dev libxslt-dev python2.7-dev` (`python2.6-dev` if you use Python 2.6). Then `sudo pip install lxml`. – Jabba Aug 12 '11 at 20:32

6 Answers6

10

Python has a native HTML parser, however the Tidy wrapper Nick suggested would probably be a solid choice as well. Tidy is a very common library, (written in C is it?)

Andrei Taranchenko
  • 1,266
  • 1
  • 10
  • 22
  • 1
    Can someone please tell me as to why do people suggest BeautifulSoup or lxml over the native HTML parser? – Shatu Aug 25 '17 at 20:33
  • Link is broken… I gues this is [html.parser](https://docs.python.org/3/library/html.parser.html)? Or the [version for legacy Python](https://docs.python.org/2/library/htmlparser.html). – Brutus Dec 17 '19 at 17:36
  • The module is still there, the URL appears to have changed though. Fixed. – Andrei Taranchenko Dec 17 '19 at 21:48
2

You can install lxml and many other python modules easily and seamlessly on the Mac (OS X) using Pallet, which is the MacPorts official GUI

The module name is py27-lxml. Easy as 1,2,3.

Gussisaurio
  • 41
  • 2
2

Perhaps µTidylib will meet your needs?

Nick Presta
  • 28,134
  • 6
  • 57
  • 76
1

html5lib is good:
http://code.google.com/p/html5lib/

Update: The link above is broken. A third-party mirror of above, can be accessed from https://github.com/html5lib/gcode-import

Shatu
  • 1,819
  • 3
  • 15
  • 27
rudyryk
  • 3,695
  • 2
  • 26
  • 33
  • While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. – dgw Aug 22 '12 at 13:30
  • This isn't *quite* a link-only answer, @Dgw. It contains a complete sentence mentioning the name of the linked-to library, and in the case of this question, the name of a library *is* the essential part of the answer. Anyone can search for it in case the link is dead. – Rob Kennedy Oct 10 '12 at 15:08
1

htql is good at handling malformed html:

http://htql.net/

seagulf
  • 380
  • 3
  • 5
1

http://www.xmlhack.com/read.php?item=1392 http://sourceforge.net/projects/pirxx/

http://pyxml.sourceforge.net/topics/

I don't have much experience with python, but I have used Xerces (from the Apache foundation) in the past and found it to be very useful. The learning curve isn't bad either, though I'm not coming from a python perspective. I suggest you consider it though. (The first two links I've included discuss python interfaces to Xerces and the last one is the first google hit on "python xml").

Joe Bane
  • 1,556
  • 1
  • 15
  • 30