tm package in R hangs with small dataset

Question

I have a data.frame of 30k records (company name and other attributes). dba_nm is the company name field with longest element < 60 characters.

The R session's memory usage goes up from 100MB to 3GB and hangs when I try the code in ?tm::VectorSource:

ds <- VectorSource(dat$dba_nm)
inspect(Corpus(ds))

can you post a sample of your data so we can try to replicate your problem? — Ben, Aug 30 '12 at 05:33
Why are you trying to print all 30k records to screen? Do something sensible like `inspect(head(Corpus(ds)))` instead. — Andrie, Aug 30 '12 at 07:17
I've found that the `tm` package doesn't like factors, so my guess is that your `dat` object contains factors instead of character vectors. — Joshua Ulrich, Aug 30 '12 at 12:38

score 0 · Answer 1 · answered Aug 30 '12 at 13:32

Well, I was getting a dataframe (dat) off a database and trying to read one of the columns (dba_nm) into a vectorsource. It turns out you have to convert it into a character vector. the following code works:

> cs <- as.character(dat$dba_nm)
> ds <- VectorSource(cs)
> Corpus(ds)
A corpus with 30453 text documents

tm package in R hangs with small dataset

1 Answers1