7

Is there any way to split a long string of HTML after N words? Obviously I could use:

' '.join(foo.split(' ')[:n])

to get the first n words of a plain text string, but that might split in the middle of an html tag, and won't produce valid html because it won't close the tags that have been opened.

I need to do this in a zope / plone site - if there is something as standard in those products that can do it, that would be ideal.

For example, say I have the text:

<p>This is some text with a 
  <a href="http://www.example.com/" title="Example link">
     bit of linked text in it
  </a>.
</p>

And I ask it to split after 5 words, it should return:

<p>This is some text with</p>

7 words:

<p>This is some text with a 
  <a href="http://www.example.com/" title="Example link">
     bit
  </a>
</p>
rjmunro
  • 27,203
  • 20
  • 110
  • 132
  • Do you want to ignore the tags, so that they won't be split? In other words only get and split text that isn't contained in a tag. – monkut Dec 11 '08 at 17:03
  • Are you looking to split document text encapsulated between tags (e.g., between

    and

    tags)?     – gotgenes Dec 11 '08 at 17:05

4 Answers4

6

Take a look at the truncate_html_words function in django.utils.text. Even if you aren't using Django, the code there does exactly what you want.

Carl Meyer
  • 122,012
  • 20
  • 106
  • 116
3

I've heard that Beautiful Soup is very good at parsing html. It will probably be able to help you get correct html out.

recursive
  • 83,943
  • 34
  • 151
  • 241
0

I was going to mention the base HTMLParser that's built in Python, since I'm not sure what the end-result your trying to get to is, it may or may not get you there, you'll work with the handlers primarily

curtisk
  • 19,950
  • 4
  • 55
  • 71
0

You can use a mix of regex, BeautifulSoup or Tidy (I prefer BeautifulSoup). The idea is simple - strip all the HTML tags first. Find the nth word (n=7 here), find the number of times the nth word appears in the string till n words - coz u are looking only for the last occurrence to be used for truncation.

Here is a piece of code, though a bit messy but works

import re
from BeautifulSoup import BeautifulSoup
import tidy

def remove_html_tags(data):
    p = re.compile(r'<.*?>')
    return p.sub('', data)

input_string='<p>This is some text with a <a href="http://www.example.com/" '\
    'title="Example link">bit of linked text in it</a></p>'

s=remove_html_tags(input_string).split(' ')[:7]

###required to ensure that only the last occurrence of the nth word is                                                                                      
#  taken into account for truncating.                                                                                                                       
#  coz if the nth word could be 'a'/'and'/'is'....etc                                                                                                       
#  which may occur multiple times within n words                                                                                                            
temp=input_string
k=s.count(s[-1])
i=1
j=0
while i<=k:
    j+=temp.find(s[-1])
    temp=temp[j+len(s[-1]):]
    i+=1
####                                                                                                                                                        
output_string=input_string[:j+len(s[-1])]

print "\nBeautifulSoup\n", BeautifulSoup(output_string)
print "\nTidy\n", tidy.parseString(output_string)

The output is what u want

BeautifulSoup
<p>This is some text with a <a href="http://www.example.com/" title="Example link">bit</a></p>

Tidy
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 3.2//EN">
<html>
<head>
<meta name="generator" content=
"HTML Tidy for Linux/x86 (vers 6 November 2007), see www.w3.org">
<title></title>
</head>
<body>
<p>This is some text with a <a href="http://www.example.com/"
title="Example link">bit</a></p>
</body>
</html>

Hope this helps

Edit: A better regex 

`p = re.compile(r'<[^<]*?>')`
JV.
  • 2,658
  • 4
  • 24
  • 36
  • Any reason why you are writing your own function to remove HTML tags and not using Beauitful Soup's [soup.get_text()](http://www.crummy.com/software/BeautifulSoup/bs4/doc/#get-text) method? – tatlar Jun 28 '14 at 22:48