3

Example:

df <- data.frame(A=1:5, B=2:6, C=3:7,D=4:8,E=5:9,F=6:10)

I want make a regression loop lm(y,x) using like y the col 1 and 2 and like x the rest of the cols.

my idea:

lmf <- function (y,x) {
                         f <- lm(y ~ x, data=df)
                         cbind(summary(f)$r.squared,summary(f)$coefficients)                  
                        }
 for(y in 1:3)
  {
    R<- apply(df[,3:6], 2, lmf(y,x)); R
  }

error: Error in model.frame.default(formula = y ~ x, data = df, drop.unused.levels = TRUE) : variable lengths differ (found for 'x')

I give this example very small but my data are 50 cols for the y and 300 cols for the x.

What I want is the same to do: lm(df$1~df$3, data=df); lm(df$1~df$4, data=df),[...], lm(df$2~df$3, data=df)... but in automatic way. Moreover I want to extract the results $coefficients and $r.squared.

Giffredo
  • 79
  • 2
  • 8
  • You are really far off. You are using apply in correctly, not using the correct function arguments, and not using the for loop iterator correctly. **Start simpler** – Señor O Aug 26 '15 at 16:22
  • Try this `lapply(2:6, function(x) lm(df[ , 1] ~ df[, x], data = df ) )`, it is just a little bit tricky to subset (it is a nested list) but it does the job. – SabDeM Aug 26 '15 at 16:22
  • Thanks for the replies, the code lapply(2:6, function(x) lm(df[ , 1] ~ df[, x], data = df ) ) give me the regression function using a fix y (in this case the col df[,1] and different x. I already reach this goal.. I want something in more adding the possibility to use different y too. – Giffredo Aug 27 '15 at 07:25

2 Answers2

7

I have an alternative version using dplyr, tidyr and broom packages. The idea is to specify the variables you want to treat as Y and X. Create 2 different datasets based on those Y and X sets. Then reshape datasets in order to be able to combine each Y with one X. Finally, for each combination run a linear regression and save the model output as a dataset.

# Check whether package name is installed...
check_package <- function(package_name) {
  if (!(package_name %in% rownames(installed.packages()))) {
    install.packages(package_name, dependencies = TRUE)
  }
}

check_package("broom")
check_package("dplyr")
check_package("tidyr")

library(dplyr)
library(broom)
library(tidyr)

# example dataset (picking 4 columns)
dt <- data.frame(mtcars) %>% select(mpg, disp, cyl, wt)

# specify which columns we want as y (dependent) and x (independent)
ynames <- c("disp","mpg")
xnames <- c("cyl","wt")

# create and reshape datasets
dt1 <- dt[,ynames]
dt1 <- gather(dt1,y,yvalue)

dt2 <- dt[,xnames]
dt2 <- gather(dt2, x, xvalue)



dt1 %>% 
  group_by(y) %>%                       # group by dependent variable
  do(data.frame(.,dt2)) %>%             # combine each y with all x
  group_by(y,x)%>%                      # get combinations of y and x to regress
  do(tidy(lm(yvalue~xvalue, data=.)))   # return lm output as dataframe


#      y   x        term    estimate  std.error statistic      p.value
# 1 disp cyl (Intercept) -156.608976 35.1805064 -4.451584 1.090157e-04
# 2 disp cyl      xvalue   62.598925  5.4693168 11.445474 1.802838e-12
# 3 disp  wt (Intercept) -131.148416 35.7165961 -3.671918 9.325668e-04
# 4 disp  wt      xvalue  112.478138 10.6353299 10.575896 1.222320e-11
# 5  mpg cyl (Intercept)   37.884576  2.0738436 18.267808 8.369155e-18
# 6  mpg cyl      xvalue   -2.875790  0.3224089 -8.919699 6.112687e-10
# 7  mpg  wt (Intercept)   37.285126  1.8776273 19.857575 8.241799e-19
# 8  mpg  wt      xvalue   -5.344472  0.5591010 -9.559044 1.293959e-10
Stereo
  • 1,148
  • 13
  • 36
AntoniosK
  • 15,991
  • 2
  • 19
  • 32
  • it looks like very good!!!!.. unfortunately my R doesn't recognize the gather function.. maybe for this problem: "> library(dplyr) Attaching package: ‘dplyr’ The following object is masked from ‘package:stats’: filter The following objects are masked from ‘package:base’: intersect, setdiff, setequal, union " – Giffredo Aug 27 '15 at 07:27
  • Gather is from the tidyr package. You shouldn't have a problem. Have you installed that package? There are other ways to do that step. You can use other reshaping commands. Let me know if you want an alternative. – AntoniosK Aug 27 '15 at 08:20
  • @Giffredo , I updated the solution so that it installs the packages + dependencies. – Stereo Aug 27 '15 at 19:56
  • I have almost the exact same issue, except that I want to do this for a bunch of different groups in my original dt. For example, imagine a regression per `gear` in `mtcars`. I should prob ask a new question, but in case its easy to add.. `group_by(gear)` won't work. @Stereo any ideas? – LauraR Apr 30 '20 at 23:56
  • 1
    @LauraR it would be much easier via a new question, where you specify what kind of combinations you're trying to get. – AntoniosK May 01 '20 at 12:50
3

I just give an example with the numerical variables in iris since but you can change that to whatever data set you want to use.

I construct the formula based on the names I prefer that to using numbers to index the columns you are interested in.

I would suggest,

 result <- sapply(names(iris)[1 : 4], 
   function(x) { 
     lapply(names(iris)[1 : 4], 
            function(y) {
              if (x != y) {
                model <- lm(as.formula(paste0(y, "~", x)), iris) 
                return(list(x = x, 
                            y = y, 
                            r.squared = summary(model)$r.squared, 
                            coefficients =  summary(model)$coefficients))
              }
              })
            })


 result
 ## Sepal.Length Sepal.Width Petal.Length Petal.Width
 ## [1,] NULL         List,4      List,4       List,4     
 ## [2,] List,4       NULL        List,4       List,4     
 ## [3,] List,4       List,4      NULL         List,4     
 ## [4,] List,4       List,4      List,4       NULL       

 result[1, 2]
 ## $Sepal.Width
 ## $Sepal.Width$x
 ## [1] "Sepal.Width"
 ## 
 ## $Sepal.Width$y
 ## [1] "Sepal.Length"
 ## 
 ## $Sepal.Width$r.squared
 ## [1] 0.01382265
 ## 
 ## $Sepal.Width$coefficients
 ## Estimate Std. Error   t value     Pr(>|t|)
 ## (Intercept)  6.5262226  0.4788963 13.627631 6.469702e-28
 ## Sepal.Width -0.2233611  0.1550809 -1.440287 1.518983e-01

Alternatively, you could store the results in a list and write a separate function that traverses that list to create a matrix with just the information you are interested in.

Stereo
  • 1,148
  • 13
  • 36