linear regression by gradient desent in R

Question

I am very new to machine learning and currently is trying to do a linear regression using R, my code is below:

x <- runif(1000, -5, 5)
y <- runif(1000, -2, 2)
z <- x + y
res <- lm(z ~ x + y)
alpha <- 0.01
num_iters <- 10000
theta <- matrix(c(0,0,0), nrow = 3)
X <- cbind(1, matrix(x), matrix(y))

for (i in 1:num_iters){
    cost <- (X %*% theta - y)
    delta <- t(X) %*% cost / length(y)
    theta <- theta - alpha * delta
}

You can see after gradient for 10000 times, the result theta is different with res which resulted by function lm of R, can somebody advise where I missed?

Please indent your code and add spaces after commas and around operators. That would make your code more readable. — Roland, Aug 19 '16 at 09:15

score 1 · Accepted Answer · answered Aug 19 '16 at 10:43

You just need one small change, the variable that you're trying to model is z not y:

set.seed(0)
x <- runif(1000,-5,5)
y <- runif(1000,-2,2)
z <- x+y
res <- lm(z~x+y)
alpha <- 0.01
num_iters <- 10000
theta <- matrix(c(0,0,0), nrow=3)
X <- cbind(1, matrix(x), matrix(y))
for (i in 1:num_iters){
  cost <- (X%*%theta-z)  ##What was y is now z
  delta <- t(X)%*%cost/length(y)
  theta <- theta-alpha*delta
}

theta
#             [,1]
#[1,] -1.56669e-16
#[2,]  1.00000e+00
#[3,]  1.00000e+00

Thank you Miff, what a stupid mistake I made – albert_wzh Aug 22 '16 at 05:05 — albert_wzh, Aug 22 '16 at 05:05

linear regression by gradient desent in R

1 Answers1