R's tm_map is creating non-existing words

Asked Dec 03 '16 at 16:14

Active Dec 03 '16 at 16:14

Viewed 35 times

I'm using tm package to find associations between words in a text.

This is what I did (I'm also using tidytext package)

book <- Corpus(VectorSource(c(part1,part2,part3,part4,part5)))
book <- tm_map(book, content_transformer(tolower))
book <- tm_map(book, removeNumbers)
book <- tm_map(book, removePunctuation)
book <- tm_map(book, stripWhitespace)
book <- tm_map(book, removeWords, stopwords("english"))

TDM_book <- TermDocumentMatrix(book)

book_tidy <- tidy(TDM_book)

When I check my final table there are words like informationare but in the text there's noting like information are in the text but lots of information this and information that.

How can I get rid of that "magic pasting"?

Best regards

asked Dec 03 '16 at 16:14

pachadotdev

3,345
6
33
60

2

can you share your sample data? – Sandipan Dey Dec 03 '16 at 16:22
1

the only problem is that I'm using copyrighted material... let me try to recreate that with something obtained from Gutenberg – pachadotdev Dec 03 '16 at 16:26
1

Did you check for "information + are"? – David. Dec 05 '16 at 07:14
yes, this is weird but after some weird tests I found that ctrl+f and to replace " " by " " (blank spaces) worked :S – pachadotdev Dec 06 '16 at 04:04

R's tm_map is creating non-existing words

0 Answers0