sklearn (scikit-learn) logistic regression package -- set trained coefficients for classification.

Question

So I read the scikit-learn package webpate:

http://scikit-learn.sourceforge.net/dev/modules/generated/sklearn.linear_model.LogisticRegression.html

I can use logistic regression to fit the data, and after I obtain an instance of LogisticRegression, I can use it to classify new data points. So far so good.

Is there a way to set the coefficients of LogisticRegression() instance though? Because after I obtain the trained coefficients, I want to use the same API to classify new data points.

Or perhaps someone else recommends another python machine learning package that have better APIs?

Thanks

doug · Answer 1 · 2011-12-16T21:04:09.290

The coefficients are attributes of the estimator object--that you created when you instantiated the Logistic Regression class--so you can access them in the normal python way:

>>> import numpy as NP
>>> from sklearn import datasets
>>> from sklearn import datasets as DS
>>> digits = DS.load_digits()
>>> D = digits.data
>>> T = digits.target

>>> # instantiate an estimator instance (classifier) of the Logistic Reg class
>>> clf = LR()
>>> # train the classifier
>>> clf.fit( D[:-1], T[:-1] )
    LogisticRegression(C=1.0, dual=False, fit_intercept=True, 
      intercept_scaling=1, penalty='l2', tol=0.0001)

>>> # attributes are accessed in the normal python way
>>> dx = clf.__dict__
>>> dx.keys()
    ['loss', 'C', 'dual', 'fit_intercept', 'class_weight_label', 'label_', 
     'penalty', 'multi_class', 'raw_coef_', 'tol', 'class_weight', 
     'intercept_scaling']

So that's how to get the coefficients, but if you are going to just use those for prediction, a more direct way is to use the estimator's predict method:

>>> # instantiate the L/R classifier, passing in norm used for penalty term 
>>> # and regularization strength
>>> clf = LR(C=.2, penalty='l1')
>>> clf
    LogisticRegression(C=0.2, dual=False, fit_intercept=True, 
      intercept_scaling=1, penalty='l1', tol=0.0001)

>>> # select some "training" instances from the original data
>>> # [of course the model should not have been trained on these instances]
>>> test = NP.random.randint(0, 151, 5)
>>> d = D[test,:]     # random selected data points w/o class labels
>>> t = T[test,:]     # the class labels that correspond to the points in d

>>> # generate model predictions for these 5 data points
>>> v = clf.predict(d)
>>> v
    array([0, 0, 2, 0, 2], dtype=int32)
>>> # how well did the model do?
>>> percent_correct = 100*NP.sum(t==v)/t.shape[0]
>>> percent_correct
    100

Hi doug. I understand how to fit the data and get the coefficients, but after I fit the data, I want to *set* coefficients use the predict() API to classify new data over and over, without having to fit the data again. Is there a way to set the coefficients of clf instance in your example? like clf.set_coefficients(), clf.set_interception() ? Thanks. — CuriousMind, Dec 17 '11 at 05:25
Hi--i'm not sure i follow. To predict (without re-training the classifier) you just need to persist the classifier after you train it--i.e., which we can do w/ my snippet above. You can call the 'predict' method on the clf object over and over again, w/o retraining. That's what you want, no? If you are concerned about this, just check the values in clf.__dict__ and you'll see it doesn't change. Now if you want to re-set the coefficients, well you can do it because clf.__dict__ is just a python dictionary and you can set the value associated w/ any key. — doug, Dec 17 '11 at 06:17
doug, Thanks for your reply. Yes my question is precisely about how to persist the classifier. But ogrisel answered my question in his reply. Thank you so much for your time! — CuriousMind, Dec 19 '11 at 01:47

ogrisel · Accepted Answer · 2011-12-18T11:21:31.823

Indeed, the estimator.coef_ and estimator.intercept_ attributes are read-only python properties instead of usual python attributes. Their values come from the estimator.raw_coef_ array whose memory layout directly maps the expected memory layout of the underlying liblinear C++ implementation of logistic regression so as to avoid any memory copy of the parameters when calling estimator.predict or estimator.predict_proba.

I agree that having read-only properties is a limitation and we should find a way to get rid of those properties but if we refactor this implementation we should also take care of not introducing any unnecessary memory copy which is not trivial to do after having a quick look at the source code.

I have opened an issue on the tracker not to forget about this limitation.

In the mean time you can read the @property annotated estimator.coef_ method to understand how estimator.coef_ and estimator.raw_coef_ are related and change the value in estimator.raw_coef_ directly.

sklearn (scikit-learn) logistic regression package -- set trained coefficients for classification.

2 Answers2