0

I have a dataset containing list of sentences that have both proper nouns and common nouns in them. Example -

  1. Google is a website
  2. the universe is expanding constantly
  3. I wish I had bagels for bReakfasT
  4. the GUITAR sounded a bit off-key
  5. The rumors of Moira Rose's death are greatly exaggerated online
  6. Mahatma Gandhi was a national treasure
  7. i strongly believe that beyonce is overrated

The casing of the dataset can also be mixed.

I want to extract all the proper nouns AND the corresponding sentences where they appear in two separate columns -

Output example

Is there any way to do this in Python? I am quite new to concepts of NLP and Python overall. Thanks!

Prat1
  • 13
  • 4

1 Answers1

0

you can try with any languauge model like , spacy or nltk as mention by @ivanp

I have just used spacy model ,

 import spacy
 import string
 nlp = spacy.load("en_core_web_sm") # load pretrained model 

 def proper_noun_extraction(x):
     prop_noun = []
     doc = nlp(string.capwords(x))
     for tok in doc:
         if tok.pos_ == 'PROPN':
            prop_noun.append(str(tok))
     if len(prop_noun) !=0:
        return (' '.join(prop_noun), x)
     else:
        return ('no proper noun found', None)

tuple_noun_sent = df['sentence'].apply(lambda x:proper_noun_extraction(x))

resultant_df = pd.DataFrame(tuple_noun_sent.tolist(), columns = ['proper_noun', 'sentence'])

enter image description here

qaiser
  • 2,770
  • 2
  • 17
  • 29
  • Hi @qaiser thank you for your answer! However, "bReakfast" is not a proper noun and it is not capturing "beyonce" as a proper noun. Is there any way out for that? – Prat1 Sep 08 '22 at 13:29
  • 100% accuracy will be difficult to achieve , just try out with different available trained model and check which one is working best for your dataset – qaiser Sep 08 '22 at 15:28
  • Got it! Is there a way I can include a capitalization code in this function so that it capitalizes the first letter of each word in the sentence before passing it to get better accuracy? @qaiser – Prat1 Sep 09 '22 at 05:31
  • @Prat1 updated the code .... – qaiser Sep 09 '22 at 05:39