4

I currently have a dataframe with 260,000 rows and 50 columns where 3 columns are numeric and the rest are categorical. I wanted to one hot encode the categorical columns in order to perform PCA and use regression to predict the class. How can I go about accomplishing the below example in R?

Example:
V1 V2 V3 V4 V5 .... VN-1 VN

to

V1_a V1_b V2_a V2_b V2_c V3_a V3_b and so on
Community
  • 1
  • 1
Nick
  • 81
  • 5

3 Answers3

3

You can use model.matrix or sparse.model.matrix. Something like this:

sparse.model.matrix(~. -1, data = your_data)

The ~. tells R that your entire table (the .) is the right hand side of some hypothetical model, and the -1 says to leave out the intercept. Without the -1 your first column will be a vector of 1s.

ericgtaylor
  • 311
  • 2
  • 6
  • How would I turn that back into a dataframe to look at its contents? I get that it can't be coerced from a dgCMatrix. – Nick Apr 24 '17 at 02:53
  • 1
    Yea, would be nice if you could just say `as.data.frame()` but you have to go to matrix first. Try `as.data.frame(as.matrix(sparse.model.matrix(~. -1, data = your_data)))`. – ericgtaylor Apr 24 '17 at 03:10
  • 1
    Unless you opted for `model.matrix`, in which case it's just `as.data.frame(model.matrix(~., ex_dat %>% select(starts_with("cat"))))`. – ericgtaylor Apr 24 '17 at 03:11
  • How would I exclude the first column only in this situation? – Nick Apr 24 '17 at 20:17
  • what do you mean by "first column"? in case this helps, here is another SO post that better explains the use of `-1` https://stats.stackexchange.com/questions/174976/why-does-the-intercept-column-in-model-matrix-replace-the-first-factor – ericgtaylor Apr 25 '17 at 02:40
0

Don't really what you mean by "hot encode".

Here's an example of using dplyr to spread out the catagorical variable iris$Species into three separate columns:

df <- iris %>% 
        mutate(id = rownames(.) %>%  # unique identified to prevent duplicate rows when spreading
        mutate(val=1) %>% # give the categorical variable a value of 1
       spread(Species, val) # spread out each level of iris$Species as columns

 df[76:80,]

   Sepal.Length Sepal.Width Petal.Length Petal.Width  id setosa versicolor virginica
76          5.8         2.7          4.1         1.0  68     NA          1        NA
77          5.8         2.7          5.1         1.9 102     NA         NA         1
78          5.8         2.7          5.1         1.9 143     NA         NA         1
79          5.8         2.8          5.1         2.4 115     NA         NA         1
80          5.8         4.0          1.2         0.2  15      1         NA        NA
Adam Quek
  • 6,973
  • 1
  • 17
  • 23
  • If you do add `fill = 0` to `spread`, so that there are 0s instead of NAs, this will be what OP is asking for, but it's not clear how this would handle multiple categorical variables. – Gregor Thomas Oct 26 '17 at 20:29
0

Basically a oneliner with data.table and mltools:

# data.table with 125 variables:
dt_1h <- one_hot(dt)

# MD5 for checking reproducibility:
> digest::digest(dt_1h, algo = "md5")
[1] "f1eb1c1e2d5d94b709101557c9ed8d0d"

Data

library(data.table)
library(mltools)
set.seed(1701)
df <- data.frame(matrix(sample(c(LETTERS[1:26]),
                               260000*3, replace = TRUE), ncol = 3),
                 matrix(rnorm(260000*47), ncol = 47))
dt <- as.data.table(df)
Roman
  • 4,744
  • 2
  • 16
  • 58