Stemmer function in R Slow

Question

I am trying to run stemmer function on a dataset(uploaded through data.table package) in R of around 40000 rows,but its taking forever to run. My code looks like this:

data[, Description := map(Description, function(k) stemmer(k))]

If manually stop the process, it shows more than 50 warnings as: Image Link

Is there an alternative to run it faster. My computer has 8Gb ram.

stemmer in general takes long time, did you check the time difference for running say 10 records? — Hardik Gupta, Oct 10 '17 at 07:18
Yeah running it on 10 rows is faster but what's the alternative, how do I do it for the entire data?? — Hartej Singh Kathuria, Oct 10 '17 at 08:38
try running it in batch, stemmer does take lot of time wen you use large dataset. have you tried lapply? — Hardik Gupta, Oct 10 '17 at 08:48

score 0 · Accepted Answer · edited Oct 10 '17 at 21:12

0

Use lapply, your function will run faster

edited Oct 10 '17 at 21:12

John Petrone

26,943
6
63
68

answered Oct 10 '17 at 16:03

Hardik Gupta

4,700
9
41
83

Stemmer function in R Slow

1 Answers1