0

I am building a Cox Proportional hazards model with the lifelines package to predict the time a borrower potentially prepays its mortgage. I fit a model by means of the cph.coxphfitter() within the liflines package. Despite the fact that the model performs decent in terms of a Concordance index, when I make a calibration plot, see screenshot, I receive the following two error messages:

C:Anaconda3\lib\site-packages\lifelines\calibration.py:45: RuntimeWarning: divide by zero encountered in log
  return np.log(-np.log(1 - p))
C:Anaconda3\lib\site-packages\lifelines\fitters\mixins.py:19: RuntimeWarning: invalid value encountered in subtract
  return self.relu(x - knot) ** 3 - (lambda_ * self.relu(x - min_knot) ** 3 + (1 - lambda_) * self.relu(x - max_knot) ** 3)

In which column of the dataset could these problems occur? Is it the duration column, the event column, or one of the covariates? And if so, how could I solve them? Thank you in advance!

Calibration plot

T=140 calibration plot: calibration plot at t=140

Michiel Voortman
  • 11
  • 2
  • How are you fitting the Cox model? Which baseline method are you using? – Cam.Davidson.Pilon Jun 11 '21 at 15:02
  • I am using the default, i.e., breslow with a small penalizer: cph_total = CoxPHFitter(penalizer = 0.001,alpha=0.05). – Michiel Voortman Jun 14 '21 at 11:49
  • Hm, I think the problem is that all your predictions are equal (or near equal) to 1. For example, what does ` model.predict_survival_function(df, times=[360])` return? Do the results there make sense? – Cam.Davidson.Pilon Jun 14 '21 at 13:51
  • @Cam.Davidson.Pilon they do make sense (somewhat). At t=360, the mean probability of survival of the test set is 0. From t=120 to t=150, there is a strong drop in the probability of survival from 0.9 at t=120 to approximately 0 at t=150. Do you have other clues what could be the reason behind the errors? – Michiel Voortman Jun 14 '21 at 14:52
  • Can you try `t0=140` in the call to `survival_probability_calibration`? – Cam.Davidson.Pilon Jun 14 '21 at 15:24
  • I have included it as a link in the original post. Would you mind sharing the interpretation of the plot at t=140? Thank you in advance! – Michiel Voortman Jun 14 '21 at 16:09

0 Answers0