Understanding the reason for Missingness in Cox model

Question

I ran the following Cox model and got 1526679 deleted observations, which is a large portion of my data.

Call: coxph(formula = Surv(time1sec, time2sec, event) ~ gain + 
Buy + Lev + TP + frailty(ID), data)

n= 73322, number of events= 73322 (1526679 observations deleted due to missingness)

I am not sure why these observations were deleted. I am certain that these values are there, and are not empty. This started happening when I added the ID as a frailty term.

Any ideas what might be going on here?

Try `dim(data[complete.cases(data),])` to see if you don't have missing values. — zx8754, Apr 13 '15 at 19:03
I did that, the dimensions are 1600001 and 21 which are the number of rows and columns in my data frame. — finstats, Apr 13 '15 at 19:12
@roland i currently have a similar problem, did you find a solution? — Amy, Dec 19 '19 at 16:34

score 0 · Answer 1 · answered Apr 13 '15 at 19:19

0

In standard regression (and regression-esque as in the case of cox regression) types of problems, the default method to handling missing values is simply to ignore them. This occurs for both the coefficients and the values you're trying to predict.

I would start out by explicitly verifying that everything you want is there. This can be done via the following code:

apply(data, 2, function(x) length(which(is.na(x))))

I would also verify that my Surv object does not have any NA's, with the following code:

length(which(is.na(Surv(time1sec, time2sec, event))))

answered Apr 13 '15 at 19:19

kblansit

29
2

op has done this with `dim(data[complete.cases(data),])` in the comments suggesting that there is no missing data at all – rawr Apr 13 '15 at 19:22
Got 0 NAs. This issue started to happen when I added the frailty(ID) variable. The model was working fine before adding that term. – finstats Apr 13 '15 at 19:22

score 0 · Answer 2 · answered Mar 22 '16 at 22:36

Also, make sure that your data source only contains relevant information. For instance, I imported my data file into R which included many entries which were not useful for the analysis (e.g. legends, keywords) and R obviously excludes those results, and they show as "missingness."

Understanding the reason for Missingness in Cox model

2 Answers2