0

I have "N" product reviews and "K" concepts such as "Color", "Audio", "User Experience", "Durability", etc. I want to get back an "N x K" boolean numpy array where 1 indicates if the review talks about the said concept, and 0 indicates it does not. For example:

The review : "Super durable and I get compliments on it daily. I found one like it at charming Charlie's for 25.00 way to high. I'm happy with this case" clearly should return {"Color":0, "Audio":0, "User Experience":1, "Durability":1}

Zero-shot Generative models such as Flan-T5 aren't good enough. I am okay with annotating a few hundred reviews if this can be converted into a supervised downstream task.

I use Python, Pytorch and this is an NLP problem. Any suggestion

Edit: Here are a few reviews with 5 concepts (I have about 100 concepts for my use-case)

[{"Review": "Looks even better in person. Be careful to not drop your phone so often because the rhinestones will fall off (duh). More of a decorative case than it is protective, but I will say that it fits perfectly and securely on my phone. Overall, very pleased with this purchase.", "Product Quality":1, "Color":0, "Positive":1, "Safety/Protection":1, "Price":0},  
{"Review": "When you don't want to spend a whole lot of cash but want a great deal...this is the shop to buy from!", "Product Quality":0, "Color":0, "Positive":1, "Safety/Protection":0, "Price":1},  
{"Review": "DO NOT BUY! this item is seriously cheap as heck. not worth buying it at all. I didn't even get to use it and it was already losing all of its gems. I wish I got my money back on this item!!", "Product Quality":1, "Color":0, "Positive":0, "Safety/Protection":0, "Price":1},
{"Review": "Beautiful quality and outstanding product! Everyone compliments me on the case and thinks I spent wayy wayy more than I really did :)", "Product Quality":1, "Color":0, "Positive":1, "Safety/Protection":0, "Price":0},
{"Review": "Comfortable shoe for both workout sessions and walking.  Neither too bulky nor too light.  This is my second pair because they fit and have no neon or pink colors, just nice basic B&W.", "Product Quality":1, "Color":1, "Positive":1, "Safety/Protection":0, "Price":0}]

Let me know if more is needed

Shrayani Mondal
  • 160
  • 1
  • 7
  • Have you tried multishot prompting? This is something that frequently performs much better than zeroshot. Typically that needs much less training data to work than a new model. Could you provide 3-5 labelled examples of each category? – Nick ODell May 17 '23 at 04:17
  • @NickODell I have added a few examples in the edited version of the question. And multishot setting made the model generate inconsistent answer instead of a simple "yes/no" answer – Shrayani Mondal May 17 '23 at 05:02

0 Answers0