0

I am trying to run my custom logistic regression model as below

from sklearn import linear_model
import scipy.stats as stat

class LogisticRegression_with_p_values:

    def __init__(self,*args,**kwargs):#,**kwargs):
        self.model = linear_model.LogisticRegression(*args,**kwargs)#,**args)

    def fit(self,X,y):
        self.model.fit(X,y)

        #### Get p-values for the fitted model ####
        denom = (2.0 * (1.0 + np.cosh(self.model.decision_function(X))))
        denom = np.tile(denom,(X.shape[1],1)).T
        F_ij = np.dot((X / denom).T,X) ## Fisher Information Matrix
        Cramer_Rao = np.linalg.inv(F_ij) ## Inverse Information Matrix
        sigma_estimates = np.sqrt(np.diagonal(Cramer_Rao))
        z_scores = self.model.coef_[0] / sigma_estimates # z-score for eaach model coefficient
        p_values = [stat.norm.sf(abs(x)) * 2 for x in z_scores] ### two tailed test for p-values

        self.coef_ = self.model.coef_
        self.intercept_ = self.mode

and when I call the fit method as below

reg = LogisticRegression_with_p_values()
reg.fit(inputs_train,loan_data_targets_train)

I get the error

---> 97     raise LinAlgError("Singular matrix")
     98 
     99 def _raise_linalgerror_nonposdef(err, flag):

LinAlgError: Singular matrix

I have coded all numeric variables as dummy variables of 1 and 0

However when I run the normal scikit learn logistic regression there is no error

from sklearn.linear_model import LogisticRegression
reg = LogisticRegression()
reg.fit(inputs_train,loan_data_targets_train)
Billeh Sarkozy
  • 53
  • 1
  • 3
  • 11
  • Error is not caused by `logistic-regression`. It occurs in `inverse` function, as it identifies `F_ij` as the `singular-matrix`. Please check it. – Ankish Bansal Oct 12 '19 at 19:15
  • Possible duplicate of [Why am I getting "LinAlgError: Singular matrix" from grangercausalitytests?](https://stackoverflow.com/questions/44305456/why-am-i-getting-linalgerror-singular-matrix-from-grangercausalitytests) – Ankish Bansal Oct 12 '19 at 19:16

0 Answers0