How can I add custom signs to spaCy's punctuation functionality?

Question

Is there any option to add custom punctuation marks, which aren't included in the default punctuation rules? (https://github.com/explosion/spaCy/blob/develop/spacy/lang/de/punctuation.py)

I am using spaCy's Matcher class (https://spacy.io/usage/rule-based-matching) and the attribute "IS_PUNCT" to remove punctuation from my text.

from spacy.matcher import Matcher

# instantiate Matcher
matcher = Matcher(nlp.vocab)

# define pattern
pattern = [{"IS_PUNCT": False}]

# add pattern to matcher
matcher.add("Cleaning", None, pattern)

I would like to customize the punctuation rules to be able to remove "|" from my texts with the Matcher.

Answer 1

You can do this by replacing the lex_attr_getters[IS_PUNCT] function by a custom one which holds a list of symbols describing the additional characters.

import spacy
from spacy.symbols import IS_PUNCT
from spacy.lang.en import EnglishDefaults

def is_punct_custom(text):
    extra_punct = ["|"]
    if text in extra_punct:
        return True
    return is_punct_original(text)

# Keep a reference to the original is_punct function
is_punct_original = EnglishDefaults.lex_attr_getters[IS_PUNCT]
# Assign a new function for IS_PUNCT
EnglishDefaults.lex_attr_getters[IS_PUNCT] = is_punct_custom