Best practice in filtering non-printable characters

Question

While building a ticketing system frontend I stumbled upon an issue found by our QA team.

The problem is that we are not filtering any non-printable characters, which could lead to strange issues:

copy-paste example code does not work
clients abusing the form by pasting multiple non-printable characters, which actually got no meaning

If we require the client to enter any non-printable characters, we can request him to upload a text file to the ticket.

So, what should be considered while removing the non-printables?

I would like to thank in advance to all the participans in this discussion!

score 1 · Accepted Answer · edited May 08 '15 at 11:56

Apparently there are 2 sets of UTF-8 non-printable control characters based on this resource:

http://www.utf8-chartable.de/

With that in mind the array in the function would look like that:

array(
    '/\x00/', '/\x01/', '/\x02/', '/\x03/', '/\x04/',
    '/\x05/', '/\x06/', '/\x07/', '/\x08/', '/\x09/',
    '/\x0A/', '/\x0B/', '/\x0C/', '/\x0D/', '/\x0E/', '/\x0F/', '/\x10/',
    '/\x11/', '/\x12/', '/\x13/', '/\x14/', '/\x15/', '/\x16/', '/\x17/',
    '/\x18/', '/\x19/', '/\x1A/', '/\x1B/', '/\x1C/', '/\x1D/', '/\x1E/',
    '/\x1F/', '/\x7F/', '/\xC2 \x80/', '/\xC2 \x81/', '/\xC2 \x82/',
    '/\xC2 \x83/', '/\xC2 \x84/', '/\xC2 \x85/', '/\xC2 \x86/', '/\xC2 \x87/',
    '/\xC2 \x88/', '/\xC2 \x89/', '/\xC2 \x8A/', '/\xC2 \x8B/', '/\xC2 \x8C/',
    '/\xC2 \x8D/', '/\xC2 \x8E/', '/\xC2 \x8F/', '/\xC2 \x90/', '/\xC2 \x91/',
    '/\xC2 \x92/', '/\xC2 \x93/', '/\xC2 \x94/', '/\xC2 \x95/', '/\xC2 \x96/',
    '/\xC2 \x97/', '/\xC2 \x98/', '/\xC2 \x99/', '/\xC2 \x9A/', '/\xC2 \x9B/',
    '/\xC2 \x9C/', '/\xC2 \x9D/', '/\xC2 \x9E/', '/\xC2 \x8F/'
);

Nikola Petkanski · Answer 2 · 2013-02-24T20:10:57.740

We're currently using the following PHP method:

function filterNonPrintableCharacters($text)
{
    $text = preg_replace(
        array(
            '/\x00/', '/\x01/', '/\x02/', '/\x03/', '/\x04/',
            '/\x05/', '/\x06/', '/\x07/', '/\x08/', '/\x09/',
            '/\x0B/','/\x0C/','/\x0D/', '/\x0E/', '/\x0F/', '/\x10/', '/\x11/',
            '/\x12/','/\x13/','/\x14/','/\x15/', '/\x16/', '/\x17/', '/\x18/',
            '/\x19/','/\x1A/','/\x1B/','/\x1C/','/\x1D/', '/\x1E/', '/\x1F/'
        ),
        '',
        $text
    );

    return $text;
}

I may be missing something and so I'll be grateful for any comments.

Best practice in filtering non-printable characters

2 Answers2