1

I am looking for help about the implementation of a logit model with statsmodel for binary variables.

Here is my code : (I am using feature selection methods : MinimumRedundancyMaximumRelevance and RecursiveFeatureElimination available on python)

for i_mrmr in range(4,20):

    for i_rfe in range(3,i_mrmr):

        regressors_step1 = I am selecting the MRMR features
        
        regressors_step2 = I am selecting features from the previous list with RFE method
        
        for method in ['newton', 'nm', 'bfgs', 'lbfgs', 'powell', 'cg', 'ncg']:

            logit_model = Logit(y,X.loc[:,regressors_step2])

            try:
                result = logit_model.fit(method=method, cov_type='HC1')
                print(result.summary)
            
            except:
                result = "error"

I am using Logit from statsmodels.discrete.discrete_model.Logit.

The y variable, the target, is a binary.

All explanatory variables, the X, are binary too.

The logit model is "functionning" for the different optimization methods. That is to say, I end up with some summary to print. Nonetheless, various warnings print such as : "Maximum Likelihood optimization failed to converge."

The optimization methods presented in the statsmodel algorithm are the ones from scipy :

  • ‘newton’ for Newton-Raphson, ‘nm’ for Nelder-Mead
  • ‘bfgs’ for Broyden-Fletcher-Goldfarb-Shanno (BFGS)
  • ‘lbfgs’ for limited-memory BFGS with optional box constraints
  • ‘powell’ for modified Powell’s method
  • ‘cg’ for conjugate gradient
  • ‘ncg’ for Newton-conjugate gradient We can find these methods on scipy.optimize.

Here are my questions :

I did not find anywhere any argument against the use of these optimization methods for a binary set of variables. But, because of the warnings, I am asking myself if it is correct to do so. And then, what is the best method, the one which is the more appropriate in this case ?

Here : Scipy minimize: how to restrict x only to 0 and 1? it is implicitely said that a model of the kind Python MIP (Mixed-Integer Linear Programming) could be better in the binary set of variables case. In the documentation of the MIP package of python it appears that to implement this kind of model I should explicitly give a function to minimize or maximize and also I should express the constraints... (see : https://docs.python-mip.com/en/latest/quickstart.html#creating-models)

Therefore I am wondering if i need to define a logit function as the objective function ? What are the constraints I should express ? Is there any easier way to do ?

user16241544
  • 11
  • 1
  • Logit regression does not require binary variables. Note that x and y are data in an estimation exercise. I may have misunderstood your approach. – Erwin Kalvelagen Jun 16 '21 at 14:25
  • I agree with you, the problem is i only have dummies as explanatory variables so that i am wondering if the scipy.optimize methods used in the implementation of the logit model from statsmodel works well with this kind of variable or require continuous variables... – user16241544 Jun 17 '21 at 15:15

0 Answers0