Case and diacritic insensitive matching of regex with metacharacter in Swift

Question

I am trying to match rude words in user inputs, for example "I Hate You!" or "i.håté.Yoù" will match with "hate you" in an array of words parsed from JSON.

So I need it to be case and diacritic insensitive and to treat whitespaces in the rude words as any non-letter character: regex metacharacter \P{L} should work for that, or at least \W

Now I know [cd] works with NSPredicate, like this:

 func matches(text: String) -> [String]? {
        if  let rudeWords = JSON?["words"] as? [String]{
            return rudeWords.filter {
                let pattern = $0.stringByReplacingOccurrencesOfString(" ", withString: "\\P{L}", options: .CaseInsensitiveSearch)
                    return NSPredicate(format: "SELF MATCHES[cd] %@", pattern).evaluateWithObject(text)
            }
        } else {
            log.debug("error fetching rude words")
            return nil
        }
    }

That doesn't work with either metacharacters, I guess they are not parsed by NSpredicate, so I tried using NSRegularExpression like this:

func matches(text: String) -> [String]? {
        if  let rudeWords = JSON?["words"] as? [String]{
            return rudeWords.filter {
                do {
                    let pattern = $0.stringByReplacingOccurrencesOfString(" ", withString: "\\P{L}", options: .CaseInsensitiveSearch)
                    let regex = try NSRegularExpression(pattern: pattern, options: .CaseInsensitive)
                    return regex.matchesInString(text, options: [], range: NSMakeRange(0, text.characters.count)).count > 0
                }
                catch _ {
                    log.debug("error parsing rude word regex")
                    return false
                }
            }
        } else {
            log.debug("error fetching rude words")
            return nil
        }
    }

This seem to work OK however there is no way that I know to make regex diacritic insensitive, so I tried this (and other solutions like re-encoding)

let text = text.stringByFoldingWithOptions(.DiacriticInsensitiveSearch, locale: NSLocale.currentLocale())

However, this does not work for me since I check user input every time a character is typed so all the solutions I tried to strip accents made the app extremely slow.

Does someone know if there any other solutions or if I am using this the wrong way ?

Thanks

EDIT

I was actually mistaken, what was making the app slow was trying to match with \P{L}, I tried the second soluton with \W and with the accent-stripping line, now it works OK even if it matches with less strings than I initially wanted.

Links

These might help some people dealing with regex and predicates:

@WiktorStribiżew Thanks a lot but that's not what I need, I actually used `P{L}` (so, the opposite of `p{L}`), I'll correct that in my question. Also, I know metacharacters inside sets (like this `[\w\d]`) won't work: [see here](http://userguide.icu-project.org/strings/regexp) — CyrilFind, Jan 26 '16 at 15:40
Yes it was just an example but I think I saw somewhere that putting metacharacters in a set did not work (that's not the subject anyway ^^) EDIT : I remember now, it was [here](https://developer.apple.com/library/ios/documentation/Cocoa/Conceptual/Predicates/Articles/pUsing.html), but maybe it's just with some metacharacters — CyrilFind, Jan 26 '16 at 15:47
Sorry to be offtopic, but [this link](http://swiftstub.com/160262119) shows it is ok to use shorthand character class in a character class. I will remove my comments. There must be some other restriction somewhere else. — Wiktor Stribiżew, Jan 26 '16 at 15:53
@WiktorStribiżew No problem, leave your comments, they could be useful to someone but maybe not with `NSPredicate` — CyrilFind, Jan 26 '16 at 16:05
@zaph sorry if that's ambiguous, I'll correct my question, it's just a JSON with an array of banned words associated to the key `"words"`, but that doesn't matter the parsing part is OK — CyrilFind, Jan 26 '16 at 16:05
JSON is a text format, not an array or dictionary, see [Introducing JSON](http://www.json.org). Are you actually using a JSOM string or an array of words? Correct terminology does matter. — zaph, Jan 26 '16 at 16:08
I know what JSON is, thanks, I meant it **contains** an array of words, if you prefer... anyway it's just not the matter here. — CyrilFind, Jan 26 '16 at 16:16

Laurel · Accepted Answer · 2020-04-22T14:02:16.200

2

It might be worthwhile to go in a different direction. Instead of flattening the input, what if you changed the regex?

Instead of matching against hate.you, could match against [h][åæaàâä][t][ëèêeé].[y][o0][ùu], for example (it's not a comprehensive list, in any case). It would make most sense to do this transformation on the fly (not storing it) because it might be easier if you need to change what the characters expand to later.

This will give you some more control over what characters will match. If you look, I have 0 as a character matching o. No amount of Unicode coercion could let you do that.

edited Apr 22 '20 at 14:02

answered Apr 19 '16 at 00:09

Laurel

5,965
14
31
57

As my edit suggests, I handled this differently, since we wanted a quick and simple solution, but I might use your answer if (/ when) we need to refactor because it indeed allows to have more control. – CyrilFind Apr 21 '16 at 07:27

score 2 · Answer 2 · answered May 19 '20 at 14:10

I ended up using the solution suggested by Laurel. It works well for me.

I post it here for anybody who might need it.

extension String {
    func getCaseAndDiacriticInsensitiveRegex() throws -> NSRegularExpression {
        var pattern = self.folding(options: [.caseInsensitive, .diacriticInsensitive], locale: .current)
        pattern = pattern.replacingOccurrences(of: "a", with: "[aàáâäæãåā]")
        pattern = pattern.replacingOccurrences(of: "c", with: "[cçćč]")
        pattern = pattern.replacingOccurrences(of: "e", with: "[eèéêëēėę]")
        pattern = pattern.replacingOccurrences(of: "l", with: "[lł]")
        pattern = pattern.replacingOccurrences(of: "i", with: "[iîïíīįì]")
        pattern = pattern.replacingOccurrences(of: "n", with: "[nñń]")
        pattern = pattern.replacingOccurrences(of: "o", with: "[oôöòóœøōõ]")
        pattern = pattern.replacingOccurrences(of: "s", with: "[sßśš]")
        pattern = pattern.replacingOccurrences(of: "u", with: "[uûüùúū]")
        pattern = pattern.replacingOccurrences(of: "y", with: "[yýÿ]")
        pattern = pattern.replacingOccurrences(of: "z", with: "[zžźż]")
        return try NSRegularExpression(pattern: pattern, options: [.caseInsensitive])
    }
}

Case and diacritic insensitive matching of regex with metacharacter in Swift

EDIT

Links

2 Answers2