13

While I was parsing the Snort regex set I found a very odd character class syntax, like [\x80-t] or [\x01-t\x0B\x0C\x0E-t\x80-t], and I can't figure out (really no clue) what -t means. I don't even know if it's standard PCRE or a sort of Snort extension.

Here are some regular expression that contains these character classes:

/\x3d\x00\x12\x00..........(.[\x80-t]|...[\x80-t])/smiR
/^To\x3A[^\r\n]+[\x01-t\x0B\x0C\x0E-t\x80-t]/smi

PS: please note that \x80-t is not even a valid range in the standard way because character t is \x74.

Simone-Cu
  • 1,109
  • 8
  • 21
  • I'm intrigued. Can I ask exactly where you found this? –  Dec 12 '13 at 23:20
  • 1
    @polkadotcadaver Of course. I was investigating some projects, one is [netbench](http://merlin.fit.vutbr.cz/ant/netbench/index.html). It contains several regular expression from L7, Bro and Snort under pattern_match/rules directory. There are some of these character classes in Snort/voip.rules.pcre, others in Snort/exploit.rules.pcre. – Simone-Cu Dec 12 '13 at 23:31
  • @Simone-Cu It's definitely a range. I searched the [pcre manual](http://www.pcre.org/pcre.txt) for `-t\b` but there was not match. Which means there's nothing special about `-t` in pcre. Now there are a few possibilities: **1)** The range is just an error from the author **2)** `0x80` is `128` in decimal, if you try `€` in a browser you get the euro symbol `€`. So maybe the program is using some kind of other encoding/character table ? – HamZa Dec 17 '13 at 09:16
  • 3
    Does the code where that comes from compile? – Angelo Fuchs Dec 17 '13 at 09:38
  • 1
    Also, did you copy/paste the regex? (Just to be sure it is `t` and not `τ` or other letters that look close to `t`) – Angelo Fuchs Dec 17 '13 at 09:46
  • 2
    @Hamza I've also looked into the pcre manual and I thougth about an error, too. But there are more than two regexes, it's strange to find so many similar errors. About the encoding/character table I have found nothing, but this doesn't mean it can't be. – Simone-Cu Dec 17 '13 at 10:22
  • 1
    @AngeloNeuschitzer I have done a copy/paste of the regexes from the files mentioned above. – Simone-Cu Dec 17 '13 at 10:28
  • @Simone-Cu Did you ever come around to solve this? – Angelo Fuchs Apr 15 '14 at 09:01

2 Answers2

4

This could reference a different character encoding where t is larger than x80 and x80 can't be addressed normally.

Take EBCDIC Scan codes for example (see here for a reference).

(But I too have no clue why somebody would want to write it that way)

For ASCII I have a wild guess: If -t means "until the next token -1" or if placed last in line "until the end of allowed characters" the second query would state this:

To:(not a newline, more than one character)(not a newline)

So basically the expression [\x01-t\x0B\x0C\x0E-t\x80-t] would mean [^\r\n].

If one applies that to (.Ç-t]|...[Ç-t]) that would address any character larger than 7bit ASCII which also could address all of unicode (besides the first 127 characters).

(That being said, I still have no clue why somebody should write it like this, but at least thats a coherent explanation besides "Its a bug")

Maybe helpful: What does the rexexes you posted mean if one writes out the \xYY? ASCII:

/=\NULL\DEVICE_CONTROL_2\NULL\.{10}\(.Ç-t]|...[Ç-t])/smiR
/^To\:[^\r\n]+[\START_OF_HEADING-t\VERTICALTAB\FORMFEED\SHIFTOUT\Ç-t]/smi

Looking after the \0x12 aka Device control 2 could help, because that won't show up in text, but maybe in net traffic.

Angelo Fuchs
  • 9,825
  • 1
  • 35
  • 72
  • This is an interesting point, but in this case I can't undestand the class `[\x01-t\x0B\x0C\x0E-t\x80-t]` that would have overlapping ranges. – Simone-Cu Dec 17 '13 at 10:17
  • 1
    @Simone-Cu Could you post some code that uses that regex? - Also, overlapping ranges may not be "good" but they should work, so as it may not "make sense" to the observer, it can be explained by an author that meddles with his regexes "until they work" but does not check them throughly afterwards. – Angelo Fuchs Dec 17 '13 at 10:21
  • This is true, but really weird. About the code, these regexes should come from Snort regex set (if netbench team has collected them without mistakes, see one of my first comment in the question for netbench). Up to now I'm trying to convert it into java regex and into a parse tree for other purposes, so I would say that the regex are the data. – Simone-Cu Dec 17 '13 at 10:34
  • 1
    You have found a strange artifact, no question. I don't know netbench but if it can be "run" did you try to run it and trigger that expression? I mean, it SHOULD break, maybe it does? – Angelo Fuchs Dec 17 '13 at 10:43
  • 1
    This is a good piece of advice. I will investigate in this direction for sure. – Simone-Cu Dec 17 '13 at 10:46
  • @Simone-Cu I added a wild guess on what that could mean into my answer and some further thoughts on the queries. – Angelo Fuchs Dec 21 '13 at 17:13
3

The second regex matches lines that begin with To: (case-insensitive) followed by at least one character that isn't a line feed or carriage return. Since this is a greedy match, I'd expect \r or \n to be the only possible terminating matches in the [\x01-t\x0B\x0C\x0E-t\x80-t] character class. Note: \r is equivalent to \x0D and \n is equivalent to \x0A. Not sure what -t means but let's pretend it was - instead. Then the character class would be [\x01-\x0B\x0C\x0E-\x80-], which is still a bit convoluted but would make a little bit more sense - i.e. allowing a \n as a terminating character but not \r.

This is a very long shot but is there any chance this could be some kind of search-and-replace gone wrong?! (Guess this can probably be quickly discounted if there are other regexes that have normal ranges without the t.)

Steve Chambers
  • 37,270
  • 24
  • 156
  • 208