1

I keep receiving the error message in the title, and I'm at my wits end.

Officially. Google shows no matches for this phrase: "Cannot add pattern for zero tokens to matcher". When I searched for help on Explosion's support page, I could not find any reference to it either.

I left a request for info. there. It contains some additional information that references a previous post that did not seem relevant here.

But, here is the link if you wish to review it.

Could you help explain the error message to me?

And explain how I might possibly fix it?

The problem? If I run my code (shown below) using their food_patterns.jsonl file, it works as it should. If I run it with my food_patterns_rp.jsonl file, I receive the error message in the title. despite no visual differences that I can detect in the formatting of those files.

Here is the relevant code:

nlp = spacy.load("en_core_web_lg")
    
ruler = nlp.add_pipe("entity_ruler")
    
ruler.from_disk("food_patterns_rp.jsonl")
    
print("This is the decoded recipe:")
print("")
    
doc = nlp(clean_recipe)

for ent in doc.ents:
print(ent.text, ent.label_)

In short, my app is designed to identify ingredients in a culinary recipe and return a nutritional profile of the recipes. But I cannot create a "patterns" file (the JSONL file that contains a large (10,000+) dictionary of ingredients) that works.

This screen shot below shows the entire Traceback message, though the most relevant part reads as follows:

File "spacy\matcher\matcher.pyx", line 118, in spacy.matcher.matcher.Matcher.add ValueError: [E012] Cannot add pattern for zero tokens to matcher. Key: INGRED

I will assume that no one has time to review my JSONL patterns file.

But here a sample from the working patterns file:

{"label":"INGRED","pattern":[{"lower":"olive"},{"lower":"oil"}]}

Versus a corresponding sample from mine:

{"label": "INGRED", "pattern": [{"lower": "olive"}, {"lower": "oil"}]}

With the exception of some extra spaces, there really is no difference in the formatting of these file that I can find.

Thank you in advance. I welcome all sincere feedback

Robert

Screenshot of the complete Traceback message from Python

  • It sounds like there was an issue in whatever code you used to generate the list of patterns, and somewhere you created something where the pattern was like `"pattern":[]` - it was empty. The Matcher can't match an empty pattern, that's what your error means. – polm23 Oct 11 '21 at 04:24

2 Answers2

0

I still don't know what I did wrong, or why the file would not work, but I was able to solve the problem by running it through Prodigy's NER manual recipe, accepting it all the data as valid one more time. Then, I exported it to a new patterns file, using Prodigy's terms.to-patterns module to export a new patterns file.

It is now working great so far.

  • Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Oct 10 '21 at 19:14
0

The error is:

File "spacy\matcher\matcher.pyx", line 118, in spacy.matcher.matcher.Matcher.add ValueError: [E012] Cannot add pattern for zero tokens to matcher. Key: INGRED

A "pattern for zero tokens" is an empty pattern, like []. The Matcher can't match on an empty pattern so it gives an error.

Based on your file at the linked forum thread, you somehow generated empty patterns like that. If you remove them your file should work, since it looks like there is nothing otherwise wrong with your patterns.

polm23
  • 14,456
  • 7
  • 35
  • 59