Markdown and XSS

Question

Ok, so I have been reading about markdown here on SO and elsewhere and the steps between user-input and the db are usually given as

convert markdown to html
sanitize html (w/whitelist)
insert into database

but to me it makes more sense to do the following:

sanitize markdown (remove all tags - no exceptions)
convert to html
insert into database

Am I missing something? This seems to me to be pretty nearly xss-proof

Note that both procedures are flawed. It's better to store the Markdown in the database and convert it to HTML on output. Among other things, this makes it easier for the user to edit the Markdown later. — Marnen Laibow-Koser, Dec 25 '13 at 16:55

score 24 · Answer 1 · answered Jul 25 '11 at 22:55

Please see this link:

http://michelf.com/weblog/2010/markdown-and-xss/

> hello <a name="n"
> href="javascript:alert('xss')">*you*</a>

Becomes

<blockquote>
 <p>hello <a name="n"
 href="javascript:alert('xss')"><em>you</em></a></p>
</blockquote>

∴ you must sanitize after converting to HTML.

score 6 · Answer 2 · edited Apr 08 '12 at 09:05

6

There are two issues with what you've proposed:

I don't see a way for your users to be able to format posts. You took advantage of Markdown to provide nice numbered lists, for example. In the proposed no-tags-no-exceptions world, I'm not seeing how the end user would be able to do such a thing.
Considerably more important: When using Markdown as the "native" formatting language, and whitelisting the other available tags,you are limiting not just the input side of the world, but the output as well. In other words, if your display engine expects Markdown and only allows whitelisted content out, even if (God forbid) somebody gets to the database and injects some nasty malware-laden code into a bunch of posts, the actual site and its users are protected because you are sanitizing it upon display, as well.

There are some good resources on the web about output sanitization:

Sanitizing user data: Where and how to do it
Output sanitization (One of my clients, who shall remain nameless and whose affected system was not developed by me, was hit with this exact worm. We have since secured those systems, of course.)
BizTech: Best Practices: Never heard of XSS?

edited Apr 08 '12 at 09:05

Gerard Roche

6,162
4
43
69

answered Nov 06 '09 at 21:40

John Rudy

37,282
14
64
100

2

As for point #1, I think you misunderstood OP. You would still use Markdown-style numbered lists, no problem, because the HTML tag removal would happen *before* Markdown converted `1. Foo` into `
Foo

Alan H.

Mar 10 '11 at 23:18

OP seems to suggest storing the HTML in the DB, which would make it impossible to edit since the HTML tags that were stored are considered invalid when supplied as input. – Stijn de Witt Nov 27 '18 at 23:12

... which reminds me of one of the 'rules' I like to use when developing: "accept your own output as input". Don't you just hate it when you copy paste the output (of say an account number) into the input field and it won't accept it before you remove or add something to it first? You see this often with phone nu,bers where they will output it like e.g. `+31 555 1234 5678`, then tell you the plus symbol is an illegal character or that it's too long or contains spaces when you try to input it. – Stijn de Witt Nov 27 '18 at 23:15

score 4 · Answer 3 · answered Nov 06 '09 at 21:43

4

Well certainly removing/escaping all tags would make a markup language more secure. However the whole point of Markdown is that it allows users to include arbitrary HTML tags as well as its own forms of markup(*). When you are allowing HTML, you have to clean/whitelist the output anyway, so you might as well do it after the markdown conversion to catch everything.

*: It's a design decision I don't agree with at all, and one that I think has not proven useful at SO, but it is a design decision and not a bug.

Incidentally, step 3 should be ‘output to page’; this normally takes place at the output stage, with the database containing the raw submitted text.

answered Nov 06 '09 at 21:43

bobince

528,062
107
651
834

Aside from XSS, why don't you agree with the design decision of including HTML in Markdown? In my experience, it's quite useful, and one of the things that saves Markdown from being yet another dead-end, overly limited markup language. – Marnen Laibow-Koser Dec 25 '13 at 16:53
1

For uses like discussion sites I find it gets in the way much more often than its helps: in general conversation you're unlikely to want to do something clever enough that MD can't do it, but you're quite likely to want to use a `<` or `&` sign without accidentally introducing markup that breaks your comment. On a civilian site this is totally unwanted but even on a tech site like this it causes problems when people try to talk about HTML itself. I've had to edit many questions/answers to make them understandable due to this problem. – bobince Dec 27 '13 at 23:41
1

I wouldn't be against HTML in MD if (a) it were off by default, or there were a standard way to disable it, and (b) it were kept separate from MD markup instead of producing really weird interactions when you used both. At present you have to be aware of what characters are special in MD *and* in HTML before you write anything. For an application like article comments or a forum this is far too deep for a regular user to cope with. This is only a criticism for applications like web comments/forums - it's clearly far more suitable for uses like technical documentation. – bobince Dec 27 '13 at 23:46
"but you're quite likely to want to use a `<` or `&` sign without accidentally introducing markup that breaks your comment"—Sure, and Markdown handles that use case just fine, smartly escaping `<`, `>`, and `&` when they're meant as plain text, and not when they're meant as HTML metacharacters. See http://jsfiddle.net/marnen/U6r29/ and the Markdown docs. I think you're postulating a problem that doesn't actually exist. – Marnen Laibow-Koser Dec 28 '13 at 04:17
"At present you have to be aware of what characters are special in MD and in HTML before you write anything." Not really. The average Markdown user (who doesn't know HTML) doesn't have to know that `<` and `&` are special, because he'll never use them in a context where Markdown won't escape them. – Marnen Laibow-Koser Dec 28 '13 at 04:20

score 2 · Answer 4 · answered Aug 03 '11 at 05:48

insert into database
convert markdown to html
sanitize html (w/whitelist)

perl

use Text::Markdown ();
use HTML::StripScripts::Parser ();

my $hss = HTML::StripScripts::Parser->new(
   {
       Context         => 'Document',
       AllowSrc        => 0,
       AllowHref       => 1,
       AllowRelURL     => 1,
       AllowMailto     => 1,
       EscapeFiltered  => 1,
   },
   strict_comment => 1,
   strict_names   => 1,
);

$hss->filter_html(Text::Markdown::markdown(shift))

Mike Samuel · Answer 5 · 2012-02-10T03:27:42.627

convert markdown to html

sanitize html (w/whitelist)

insert into database

Here, the assumptions are

Given dangerous HTML, the sanitizer can produce safe HTML.
The definition of safe HTML will not change, so if it is safe when I insert it into the DB, it is safe when I extract it.

sanitize markdown (remove all tags - no exceptions)

convert to html

insert into database

Here the assumptions are

Given dangerous markdown, the sanitizer can produce markdown that when converted to HTML by a different program will be safe.
The definition of safe HTML will not change, so if it is safe when I insert it into the DB, it is safe when I extract it.

The markdown sanitizer has to know not just about dangerous HTML and dangerous markdown, but how the markdown->HTML converter does its job. That makes it more complex, and more likely to be wrong than the simpler unsafeHTML->safeHTML function above.

As a concrete example, "remove all tags" assumes you can identify tags, and would not work against UTF-7 attacks. There might be other encoding attacks out there that render this assumption moot, or there might be a bug that causes the markdown->HTML program to convert (full-width '<', exotic white-space characters stripped by markdown, SCRIPT) into a <script> tag.

The most secure would be:

sanitize markdown (remove all tags - no exceptions)
convert markdown to HTML
sanitize HTML
insert into a DB column marked risky
re-sanitize HTML every time you fetch that column from the DB

That way, when you update your HTML sanitizer you get protection against any newly discovered attacks. This is often inefficient, but you can get pretty good security by storing a timestamp with HTML inserted so that you can tell which might have been inserted during the time when someone knew about an attack that gets past your sanitizer.

Markdown and XSS

5 Answers5

perl

Linked