Count number of times a word-wildcard appears in text (in R)

Question

I have a vector of either regular words ("activated") or wildcard words ("activat*"). I want to:

1) Count the number of times each word appears in a given text (i.e., if "activated" appears in text, "activated" frequency would be 1).

2) Count the number of times each word wildcard appears in a text (i.e., if "activated" and "activation" appear in text, "activat*" frequency would be 2).

I'm able to achieve (1), but not (2). Can anyone please help? thanks.

library(tm)
library(qdap)
text <- "activation has begun. system activated"
text <- Corpus(VectorSource(text))
words <- c("activation", "activated", "activat*")

# Using termco to search for the words in the text
apply_as_df(text, termco, match.list=words)

# Result:
#      docs    word.count    activation    activated    activat*
# 1   doc 1             5     1(20.00%)    1(20.00%)           0

score 1 · Accepted Answer · answered Jan 28 '16 at 18:29

1

Is it possible that this might have to do something with the versions? I ran the exact same code (see below) and got what you expected

    > text <- "activation has begunm system activated"
    > text <- Corpus(VectorSource(text))
    > words <- c("activation", "activated", "activat")
    > apply_as_df(text, termco, match.list=words)
       docs word.count activation activated   activat
    1 doc 1          5  1(20.00%) 1(20.00%) 2(40.00%)

Below is the output when I run R.version(). I am running this in RStudio Version 0.99.491 on Windows 10.

    > R.Version()

    $platform
    [1] "x86_64-w64-mingw32"

    $arch
    [1] "x86_64"

    $os
    [1] "mingw32"

    $system
    [1] "x86_64, mingw32"

    $status
    [1] ""

    $major
    [1] "3"

    $minor
    [1] "2.3"

    $year
    [1] "2015"

    $month
    [1] "12"

    $day
    [1] "10"

    $`svn rev`
    [1] "69752"

    $language
    [1] "R"

    $version.string
    [1] "R version 3.2.3 (2015-12-10)"

    $nickname
    [1] "Wooden Christmas-Tree"

Hope this helps

answered Jan 28 '16 at 18:29

vmachan

1,672
1
10
10

Thanks, this worked. I used asterisks to mark wildcards - removing them solved the problem. – David Spivak Jan 28 '16 at 21:33
Do you know how to make `qdap` distinguish between wildcards and exact words? For example, one of my target words is "led", but I want `R` to find this exact word in the text, not "mailed", "ledge", etc. Is that possible? – David Spivak Jan 29 '16 at 15:52
Does this `apply_as_df(text, termco_d, match.string="led")` help?.. it uses the `termco_d` instead of `termco` – vmachan Jan 29 '16 at 16:08
Thanks, but no, it still handles "led" as a wildcard word (counts "mailed" as a match too, for example). – David Spivak Jan 29 '16 at 19:13
Got it, you have to use `match.string=" led "` (with spaces). Also works with `termco` if you store `" led "` in a vector. – David Spivak Jan 29 '16 at 19:35

score 1 · Answer 2 · answered Jan 28 '16 at 19:31

Maybe consider different approach using library stringi?

text <- "activation has begun. system activated"
words <- c("activation", "activated", "activat*")

library(stringi)
counts <- unlist(lapply(words,function(word)
{
  newWord <- stri_replace_all_fixed(word,"*", "\\p{L}")
  stri_count_regex(text, newWord)
}))

ratios <- counts/stri_count_words(text)
names(ratios) <- words
ratios

Result is:

activation  activated   activat* 
0.2         0.2        0.4

In the code I convert * into \p{L} which means any letter in regex pattern. After that I count found regex occurences.

Count number of times a word-wildcard appears in text (in R)

2 Answers2

Linked