asymmetric regularization in machine learning libraries (e.g. scikit ) in python

Question

The problem requires me to regularize weights of selected features while training a linear classifier. I am using python SKlearn.

Having googled a lot about incorporating asymmetric regularization for classifiers in SKlearn, I could not find any solution. The core library function that performs this task is provided as a DLL for windows hence modifying the existing library is not possible.

Is there any machine learning library for python with this kind of flexibility? Any kind of help will be appreciated.

Answer 1

To modify the library you have to download the source (e.g. from the github repository of the project: https://github.com/scikit-learn/scikit-learn) and then install the build requirements (e.g. a C/C++ compiler for your platform).

Here are building instructions for Windows:

• http://scikit-learn.org/dev/install.html#building-on-windows

Answer 2

Depending on the amount of data you have and the classifier you would like to use, it might be easier to implement the loss and then use a standard solver like lbfgs or newton, or do stochastic gradient descent, if you have a lot of data.

Using a simple custom solver will most likely be much slower than using scikit-learn code, but it will also be much easier to write. In particular if you are after logistic regression, for example, you would need to dig into LibLinear C code. On the other hand, I'm pretty certain that you can implement it in ~10 lines of python using lbfgs in an unoptimized way.