Extracting multiple sentences surrounding a keyword from cell of text

Question

I am trying to search a large text in R for keywords. Once I find one, I want to extract the 1 sentence before and after that keyword (including the sentence with the keyword in it). Ideally, I would like to be able to change this code to extract up to 3 sentences around the keyword. Sample data below.

text <- "This is an article about random things. Usually, there are a few sentences that are irrelevant to what I am interested in. Then in the middle, there is a sentence that I want to extract. Water quality is a serious concern in Akron, Ohio. It can impact ecological systems and human health. Jon Doe is a key player in this realm. Then the article goes on talking about something else that I don't care about."

keywords <- c("water quality", "health")

So with the text above, I want to search the text for "water quality" and "health" and when there is a match, I want to extract from "Then in the middle there is..." to "Jon Doe is a key player in this realm."

Finally, I want to repeat this over a number of rows with each row having its own text.

I've looked into using stringr/regex but it's not giving me what I want- I can't pull the full sentences. Any ideas?

Code I've tried:

str_extract_all(text,paste0("([^\\s+\\s){5}",keywords,"(\\s[^\\s]+){5}"))

-> that gets me a few words on either side

gsub(".*?([^\\.]*('water quality'|health)[^\\.]*).*","\\1", text, ignore.case = TRUE)

-> close also

Answer 1

Use keywords to create a pattern to look for, put the data in a tibble, separate them out in sentences (splitting on period) and select n-1, n and n+1 row for every n row where the pattern is found.

library(dplyr)
library(tidyr)

keywords <- c("water quality", "health")
pat <- paste0(keywords, collapse = '|')
pat
#[1] "water quality|health"

tibble(text) %>%
  separate_rows(text, sep = '\\.\\s*') %>%
  slice({
    tmp <- grep(pat, text, ignore.case = TRUE)
    sort(unique(c(tmp-1, tmp, tmp + 1)))
  })

#  text                                                          
#  <chr>                                                         
#1 Then in the middle, there is a sentence that I want to extract
#2 Water quality is a serious concern in Akron, Ohio             
#3 It can impact ecological systems and human health             
#4 Jon Doe is a key player in this realm       

Answer 2

This can be done with a regular expression.

for(kw in keywords) {
    Pat <- paste(".*?(([^.]+\\.){0,1}[^.]+", kw, ".*?\\.(.*?\\.){0,1}).*", sep="")
    print(sub(Pat, "\\1", text, ignore.case=T))
}
[1] " Then in the middle, there is a sentence that I want to extract. Water quality is a serious concern in Akron, Ohio. It can impact ecological systems and human health."
[1] " Water quality is a serious concern in Akron, Ohio. It can impact ecological systems and human health. Jon Doe is a key player in this realm."

Some details about the regex. This works the same for each keyword. I will use the second one "health" as my example. If you print out the pattern Pat, you get

".*?(([^.]+\\.){0,1}[^.]+health.*?\\.(.*?\\.){0,1}).*"

What is this doing? The sub statement will replace whatever is matched with the contents of \1 - the first capture group, the stuff inside the first set of parentheses. Let's look at pieces of this.

To get the sentence containing the keyword "health" we have [^.]+health.*?\\. this matches any number of characters other than period followed by health followed by any number of characters up to the next period. To get the sentence after the sentence with health, we add (.*?\\.){0,1} That means any character up to and including the next period. But what if there is no sentence after the health sentence? That is why I wrote {0,1} to make the next sentence optional. Similarly, we include (.*?\\.){0,1} in front of the part that captures the "health" sentence to get an optional sentence before the health sentence. All of this is in parentheses to make it a capture group - the first capture group, the one that gets stored in \1. That matches the part that we want, but what about the rest? We want to get rid of everything else, so we put .*? in front and .* at the end so that the rest of the text will be matched. Now the pattern matches the entire string, but sub replaces it with the part that we want. If you want two sentences before and after the keyword sentence, just replace {0,1} with {0,2}.