Filter whitespace and newlines with Pygments

Question

I've been trying to add syntax highlighting to my django site. The problem is I'm getting the   and <br /> characters formatted as well. Is there a way to preserve these characterss? Here is the code I'm using:

from BeautifulSoup  import BeautifulSoup
from django import template
from django.template.defaultfilters import stringfilter
import pygments
import pygments.formatters
import pygments.lexers


register = template.Library()

@register.filter
@stringfilter
def pygmentized(html):
    soup = BeautifulSoup(html)
    codeblocks = soup.findAll('code')
    for block in codeblocks:
        if block.has_key('class'):
            try:
                code = ''.join([unicode(item) for item in block.contents])
                lexer = pygments.lexers.get_lexer_by_name(block['class'], stripall=True)
                formatter = pygments.formatters.HtmlFormatter()
                code_hl = pygments.highlight(code, lexer, formatter)
                block.contents = [BeautifulSoup(code_hl)]
                block.name = 'code'
            except:
                raise
    return unicode(soup)

is usually used for code blocks. This way you don't need `
` or ` `. — Petri Lehtinen, Mar 22 '11 at 06:32
Thanks Petri. I should have read about html tags before, shame on me. — Ruben Quinones, Mar 23 '11 at 06:40

score 1 · Accepted Answer · answered Mar 23 '11 at 06:48

Well, Petri is right, pre is for code blocks. Before he pointed that out I just wrote a function to clean the first output, it's messy but maybe someone who just needs to remove certain things from the final output might find it ok:

from BeautifulSoup  import BeautifulSoup
from django import template
from django.template.defaultfilters import stringfilter
import pygments
import pygments.formatters
import pygments.lexers


register = template.Library()
wanted = {'br': '<br />', 'BR': '<BR />', 'nbsp': '&nbsp;', 'NBSP': '&NBSP;', '/&gt;': ''}

def uglyfilter(html):
    content = BeautifulSoup(html)
    for node in content.findAll('span'):
        data = ''.join(node.findAll(text=True))
        if wanted.has_key(data):
            node.replaceWith(wanted.get(data))
    return unicode(content)     


@register.filter
@stringfilter
def pygmentized(html):
    soup = BeautifulSoup(html)
    codeblocks = soup.findAll('pre')
    for block in codeblocks:
        if block.has_key('class'):
            try:
                code = ''.join([unicode(item) for item in block.contents])
                lexer = pygments.lexers.get_lexer_by_name(block['class'], stripall=True)
                formatter = pygments.formatters.HtmlFormatter()
                code_hl = pygments.highlight(code, lexer, formatter)
                clean = uglyfilter(code_hl)
                block.contents = [BeautifulSoup(clean)]
                block.name = 'pre'
            except:
                raise
    return unicode(soup)

Filter whitespace and newlines with Pygments

1 Answers1