I am using the R package MARSS to run a dynamic factor analysis. I have 8 timeseries and all of the time series have at least 1 NA value (range 1-20 of 50 years/timeseries).
When I ran my model with just 23 years of data (the years where all timeseries had no NA values), it had both Abstol and log-log convergence after 293,368 iterations (maxit was set to 1,000,000). However, after trying it again with the full time series, I only have Abstol convergence after 1,000,000 iterations and this took 2 days to run.
I can't seem to find any guidance on how many NA values a DFA can handle nor what is typically used for maxit. Are there any tools to determine if there are too many NA values in a timeseries for a DFA?
Here is how I have specified model. Note: I haven't provided any data because I don't think anyone wants to run this model given how long it presently takes.
library(MARSS)
listMod = list(m = mm, R = "diagonal and unequal")
listInit = list(x0 = matrix(rep(0, mm), mm, 1))
listCont = list(maxit = 1000000, allow.degen = TRUE)
dfa1 <- MARSS(y = data, # matrix with 50 columns (years) & 8 rows (each timeseries); 84 NA values
form = "dfa",
z.score = FALSE, # timeseries were individually centred and scaled while preparing the dataset (mean = 0, sd = 1)
model = listMod,
inits = listInit,
control = listCont)
Results:
Warning! Abstol convergence only. Maxit (=1e+06) reached before log-log convergence.
Alert: Numerical warnings were generated. Print the $errors element of output to see the warnings.
MARSS fit is
Estimation method: kem
Convergence test: conv.test.slope.tol = 0.5, abstol = 0.001
WARNING: Abstol convergence only no log-log convergence.
maxit (=1e+06) reached before log-log convergence.
The likelihood and params might not be at the ML values.
Try setting control$maxit higher.
Convergence warnings
2998019 warnings. First 10 shown. Type cat(object$errors) to see the full list.
Warning: the R.(Y1,Y1) parameter value has not converged.
Warning: the R.(Y2,Y2) parameter value has not converged.
Warning: the R.(Y7,Y7) parameter value has not converged.
Warning: the logLik parameter value has not converged.
Type MARSSinfo("convergence") for more info on this warning.
MARSSkem warnings. Type MARSSinfo() for help.
iter=412 Setting element of R to 0, blocked. See MARSSinfo("R0blocked"). The error is due to the following MARSSkemcheck errors.
MARSSkemcheck error: t=1: For method=kem (EM), if an element of the diagonal of R is 0, the corresponding row of Z must be fixed. See MARSSinfo('AZR0').
iter=413 Setting element of R to 0, blocked. See MARSSinfo("R0blocked"). The error is due to the following MARSSkemcheck errors.
MARSSkemcheck error: t=1: For method=kem (EM), if an element of the diagonal of R is 0, the corresponding row of Z must be fixed. See MARSSinfo('AZR0').
iter=414 Setting element of R to 0, blocked. See MARSSinfo("R0blocked"). The error is due to the following MARSSkemcheck errors.
MARSSkemcheck error: t=1: For method=kem (EM), if an element of the diagonal of R is 0, the corresponding row of Z must be fixed. See MARSSinfo('AZR0').
iter=415 Setting element of R to 0, blocked. See MARSSinfo("R0blocked"). The error is due to the following MARSSkemcheck errors.
MARSSkemcheck error: t=1: For method=kem (EM), if an element of the diagonal of R is 0, the corresponding row of Z must be fixed. See MARSSinfo('AZR0').
