Replace rules(String pattern matching) in R

Question

I know similar question might have asked in this forum but I feel my requirement is peculiar. I have a data frame with a column with the following values. Below is the just sample and it contains more than 1000 observations

Reported Terms

"2 Left Axillary Lymph Nodes Resection"             
"cardyoohyper"                                      
"Ablation Breast"                                    
"Hypercarido"                                       
"chordiohyper"                                       
"Adenocarcinoma Of Colon (Radical Resection And Cr)"
"myocasta"
"hypermyopa"

I have another data frame with the below rules:

Data frame

I am expecting the below output:

"2 Left Axillary Lymph Nodes Resection"             
"carddiohiper"                                      
"Ablation Breast"                                    
"hipercardio"                                       
"cardiohyper"                                       
"Adenocarcinoma Of Colon (Radical Resection And Cr)"
"miocasta"
"hipermiopa"

I am trying with hot encoding with gsub function but I understand that it will take a lot time.

pattern <- c("kardio, "carido", "cardyo", "cordio", "chordio")
replacement <- "cardio"
gusub(pattern,replacement,df$reportedterms)

with the above approach I need to encode every time for every rule and I need to create different variables each time for pattern and replacement in gsub function.

Is there a simple approach to solve this problem?

JBGruber · Answer 1 · 2019-11-20T10:01:56.463

First let's set this up as described by you:

library(tibble)

df <- tibble(text = c("2 Left Axillary Lymph Nodes Resection",
                      "cardyoohyper",
                      "Ablation Breast",
                      "Hypercarido",
                      "chordiohyper",
                      "Adenocarcinoma Of Colon (Radical Resection And Cr)",
                      "myocasta",
                      "hypermyopa"))

replace_dict <- tibble(pattern = list(c("kardio", "carido", "cardyo", "cordio", "chordio"), 
                                      "myoca",
                                      "myopa",
                                      "hyper"),
                       replacement = c("cardio", 
                                       "mioca",
                                       "miopa",
                                       "hiper"))

I would simply use stringi for the task as it has an extremely efficient version of gsub which is stri_replace_all_fixed (note that you could also use the regex version, which is a bit slower but works the same). It can handle several patterns and replacements at the same time, so all we need to do is unnest the pattern column first and then run stringi:

batch_replace <- function(text, replace_dict) {

  replace_dict <- tidyr::unnest(replace_dict, pattern) 

  stringi::stri_replace_all_fixed(str = text, 
                                  pattern = replace_dict$pattern, 
                                  replacement = replace_dict$replacement, 
                                  vectorize_all = FALSE)
}

Let's put this function to a test:

df$text_new <- batch_replace(df$text, replace_dict)
df
#> # A tibble: 8 x 2
#>   text                                 text_new                            
#>   <chr>                                <chr>                               
#> 1 2 Left Axillary Lymph Nodes Resecti~ 2 Left Axillary Lymph Nodes Resecti~
#> 2 cardyoohyper                         cardioohiper                        
#> 3 Ablation Breast                      Ablation Breast                     
#> 4 Hypercarido                          Hypercardio                         
#> 5 chordiohyper                         cardiohiper                         
#> 6 Adenocarcinoma Of Colon (Radical Re~ Adenocarcinoma Of Colon (Radical Re~
#> 7 myocasta                             miocasta                            
#> 8 hypermyopa                           hipermiopa

I think that is what you wanted. Note that the function isn't very flexible as you have to provide stri_replace_all_fixed exactly in the way shown. Since you haven't shared the file, I can't help you with wrangling into that form, so you have to figure that out or ask a new question.

update

If you want replacement to be case insensitive and also want to lowercase the text, the function could look like this:

batch_replace <- function(text, replace_dict, to_lower = TRUE, case_insensitive = TRUE) {

  replace_dict <- tidyr::unnest(replace_dict, pattern) 

  if (to_lower) {
    text <- tolower(text)
  }

  stringi::stri_replace_all_fixed(str = text, 
                                  pattern = replace_dict$pattern, 
                                  replacement = replace_dict$replacement, 
                                  vectorize_all = FALSE,
                                  opts_fixed = stringi::stri_opts_fixed(case_insensitive = case_insensitive))
}

You can turn on/off lower casing and case-insensitive replacement as you need it.

Thanks, But in one scenario , I want to replace the wrong word only start with hyper of the string . For example "hypercardio" should be replaced with "hipercardio" but in another case "cardiohyper" should not replaced with "cardiohiper" it should remain as "cardiohyper" ..Is there anyway to do this — Pavan kumar, Oct 24 '19 at 07:11
You can use `stri_replace_all_regex` instead and then use the regular expression `"\\bhyper"` as pattern. This means "hyper" must be at the beginning of a word in order to be replaced. `\\b` is a word boundary, which could also be used to indicate the end of a word. This would replace "hyper" only if it appears at the end of a word `"hyper\\b"`. — JBGruber, Oct 24 '19 at 09:56
Did that clear up your questions? Did my answer work for you? — JBGruber, Oct 31 '19 at 12:55
Sorry for delay response, I was in sick condition for the last 3 weeks. I tried as follow replace_dict <- tibble(pattern = list(c("\\bNuetrop","\\bnautrop"), c("ektomy\\b","ektomia\\b","ektomie\\b","ectomi\\b"),"thiroid"), replacement = c("neutrop" ",ectomy","thyroid" )) and its working now but I am using to_lower() function when I am applying the function. con_df1$verbatim1 <- batch_replace(tolower(con_df1$Verbatim),replace_dict) Is there any chance where we can incorporate to_lower() function in batch_replace function — Pavan kumar, Nov 20 '19 at 06:23

Replace rules(String pattern matching) in R

1 Answers1

update