How to save temporary object within dplyr summarize so that multiple summary functions can refer to it?

Question

I am using dplyr to summarize a dataframe that contains many groups. The original data is density data over time for populations of organisms. For instance (where dots denote many more rows of data):

group	time	density
A	0	0
A	10	1
A	20	2
A	30	4
A	...	...
B	0	0
B	10	2
B	...	...
C	0	0
...	...	...

I have various other metrics I've used to summarize each group's timeseries already within a summarize call. However, I'd now like to use optim to fit mathematical models of population growth to the density data. For instance, the logistic model of the form:

density = K/(1+((K-initial_density)/initial_density)*e^(r*time))

I can do so normally (outside of summarize) by writing a function for the error (d_0 is initial density):

logis_fit_err <- function(params, t_vals, dens_vals) {
  #params <- c("k" = ..., "d_0" = ..., "r" = ...)
  pred_vals <- with(as.list(params),
                    k/(1+(k*d_0/d_0)*exp(r*t_vals)))
  return(sum((log10(pred_vals) - log10(dens_vals))**2))
}

Then calling optim using that function:

my_fit <- optim(par = c("k" = 10**9, "d_0" = 0, 
                         "r" = 0.04),
                 fn = logis_fit_err, 
                 dens_vals = density,
                 t_vals = time,
                 method = "BFGS")

The results will be stored as a named vector in my_fit$par. However, from each fit I need to extract both the r and k parameters into different columns of my summarize output

summarize(my_grouped_data,
          fit_r = optim(par = c("k" = 10**9, "d_0" = 0, 
                         "r" = 0.04),
                 fn = logis_fit_err, 
                 dens_vals = density,
                 t_vals = time,
                 method = "BFGS")$par["r"],
          fit_k = optim(par = c("k" = pseudo_K, "d_0" = first_min, 
                         "r" = max_percap_gr_rate),
                 fn = logis_fit_err, 
                 dens_vals = density,
                 t_vals = time,
                 method = "BFGS")$par["k"])

Naturally, though, this will run the optim algorithm once for each parameter, which is both not ideal for speed and, depending on the stochasticity of the optim solution, may give me wrong results. How can I save the r and k fit parameters from the same run of optim into different columns of the output summary dataframe?

For future readers: in the end, I decided to loop through my groups outside of the summarize statement.

May I suggest that you ask one question at a time? The question meanders and needs to be made much clearer and simpler. Nail down the problem to one line of code and make a [reproducible example](https://stackoverflow.com/help/how-to-ask). — mccurcio, Jan 13 '21 at 22:38
Hi there, I only have one question in my post: how can I save multiple values from optim when optim is run within dplyr::summarize without running optim multiple times? What would I need to add to make my example more reproducible? — mikeblazanin, Jan 14 '21 at 00:35
A reproducible example is an example where we can copy your data and code directly into our R session and test the code that you have shared. For example we don't have `my_grouped_data` in our R. If the data is too big you can share only part of it. All these details are included in this link http://stackoverflow.com/questions/5963269 — Ronak Shah, Jan 14 '21 at 05:11

score 0 · Answer 1 · answered Jan 13 '21 at 22:29

Difficult to test, but if you do something like the following, you will get a column that goes 'k', 'r', 'k', 'r' etc. for each value. From there, you can create a column which is equal to 'k', 'r', 'k', 'r', and then use the 'pivot_wider' function to spread these figures into the wide format you're after.

summarize(my_grouped_data,
          fit = optim(par = c("k" = 10**9, "d_0" = 0, 
                         "r" = 0.04),
                 fn = logis_fit_err, 
                 dens_vals = density,
                 t_vals = time,
                 method = "BFGS")$par[, c("k'", "r")])

This is a good idea I hadn't thought of, but the problem is that I'd also like to save the $value of optim. Following your idea I looked around and found this: https://stackoverflow.com/questions/56957399/summarize-to-vector-output I'm going to try nesting a list inside the output df and see if that works — mikeblazanin, Jan 14 '21 at 00:37

How to save temporary object within dplyr summarize so that multiple summary functions can refer to it?

1 Answers1