perform LDA with 3 classes in R

Question

I have three classes with mean

mu1 <- matrix(c(3, 1), nrow=2)
mu2 <- matrix(c(4, 3), nrow=2)
mu3 <- matrix(c(8, 2), nrow=2)

and covariance

cov <- matrix(c(.5, .3, .3, .5), nrow=2, ncol=2)

I would like to simulate about 100 observations from each class and perform LDA. first, I made three matrix with 100 observations.

x1 <- matrix(c(rmvnorm(100, mean=mu1, sigma=cov), matrix("x1", ncol=1, nrow=100)), ncol=3)
x2 <-matrix(c(rmvnorm(100, mean=mu2, sigma=cov), matrix("x2", ncol=1, nrow=100)), ncol=3)
x3 <- matrix(c(rmvnorm(100, mean=mu3, sigma=cov), matrix("x3", ncol=1, nrow=100)), ncol=3)

and made those to data frame and bind it together.

d1 <- data.frame(x1)
d2 <- data.frame(x2)
d3 <- data.frame(x3)
alld <- rbind(d1, d2, d3)

now I would like to perform lda with code of

lda.x1 <- lda(alld[,3]~alld[,1]+alld[,2], data=alld)

here... I got warning message and weird result. please help me out Thank you

Warning message: In lda.default(x, grouping, ...) : variables are collinear — user3358686, Oct 07 '14 at 06:31

score 0 · Answer 1 · answered Oct 07 '14 at 10:13

Your groups are on a line, which is tripping off lda (see plot(alld[, 1], alld[, 2], col = alld[, 3]). I've modified your code a bit and added some noise to means.

set.seed(357)
mu1 <- sample(1:10, 2)
mu2 <- sample(1:10, 2)
mu3 <- sample(1:10, 2)

cov <- matrix(c(.5, .3, .3, .5), nrow=2, ncol=2)

require(mvtnorm)
x1 <- rmvnorm(100, mean= mu1, sigma=cov)
x2 <- rmvnorm(100, mean= mu2, sigma=cov)
x3 <- rmvnorm(100, mean= mu3, sigma=cov)

alld <- data.frame(rbind(x1, x2, x3))
alld$col <- rep(1:3, each = 100)
names(alld) <- c("a", "b", "col")
plot(b ~ a, data = alld, col = alld$col)

mdl <- lda(col ~ a + b, data = alld)
plot(mdl)
points(predict(mdl)$x, cex = 0.5, pch = "+")

perform LDA with 3 classes in R

1 Answers1