error when using keras' sk-learn API

Question

  i'm learning keras these days, and i met an error when using scikit-learn API.Here are something maybe useful:

ENVIRONMENT:

python:3.5.2  
keras:1.0.5  
scikit-learn:0.17.1

CODE

import pandas as pd
from keras.layers import Input, Dense
from keras.models import Model
from keras.models import Sequential
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.cross_validation import train_test_split
from sklearn.cross_validation import cross_val_score
from sqlalchemy import create_engine
from sklearn.cross_validation import KFold


def read_db():
    "get prepared data from mysql."
    con_str = "mysql+mysqldb://root:0000@localhost/nbse?charset=utf8"
    engine = create_engine(con_str)
    data = pd.read_sql_table('data_ml', engine)
    return data

def nn_model():
    "create a model."
    model = Sequential()
    model.add(Dense(output_dim=100, input_dim=105, activation='softplus'))
    model.add(Dense(output_dim=1, input_dim=100, activation='softplus'))
    model.compile(loss='mean_squared_error', optimizer='adam')
    return model

data = read_db()
y = data.pop('PRICE').as_matrix()
x = data.as_matrix()
model = nn_model()
model = KerasRegressor(build_fn=model, nb_epoch=2)
model.fit(x,y)  #something wrong here!

ERROR

Traceback (most recent call last):
  File "C:/Users/Administrator/PycharmProjects/forecast/gridsearch.py", line 43, in <module>
    model.fit(x,y)
  File "D:\Program Files\Python35\lib\site-packages\keras\wrappers\scikit_learn.py", line 135, in fit
    **self.filter_sk_params(self.build_fn.__call__))
TypeError: __call__() missing 1 required positional argument: 'x'

Process finished with exit code 1

  the model works well without packaging with kerasRegressor, but i wanna using sk_learn's gridSearch after this, so i'm here for help. I tried but still have no idea.

something maybe helpful:

keras.warappers.scikit_learn.py  

class BaseWrapper(object):  


    def __init__(self, build_fn=None, **sk_params):
        self.build_fn = build_fn
        self.sk_params = sk_params
        self.check_params(sk_params)  


    def fit(self, X, y, **kwargs):
        '''Construct a new model with build_fn and fit the model according
        to the given training data.
    # Arguments
        X : array-like, shape `(n_samples, n_features)`
            Training samples where n_samples in the number of samples
            and n_features is the number of features.
        y : array-like, shape `(n_samples,)` or `(n_samples, n_outputs)`
            True labels for X.
        kwargs: dictionary arguments
            Legal arguments are the arguments of `Sequential.fit`

    # Returns
        history : object
            details about the training history at each epoch.
    '''

    if self.build_fn is None:
        self.model = self.__call__(**self.filter_sk_params(self.__call__))
    elif not isinstance(self.build_fn, types.FunctionType):
        self.model = self.build_fn(
            **self.filter_sk_params(self.build_fn.__call__))
    else:
        self.model = self.build_fn(**self.filter_sk_params(self.build_fn))

    loss_name = self.model.loss
    if hasattr(loss_name, '__name__'):
        loss_name = loss_name.__name__
    if loss_name == 'categorical_crossentropy' and len(y.shape) != 2:
        y = to_categorical(y)

    fit_args = copy.deepcopy(self.filter_sk_params(Sequential.fit))
    fit_args.update(kwargs)

    history = self.model.fit(X, y, **fit_args)

    return history

  error occored in this line:

    self.model = self.build_fn(
        **self.filter_sk_params(self.build_fn.__call__))

self.build_fn here is keras.models.Sequential

models.py  

class Sequential(Model):

    def call(self, x, mask=None):
        if not self.built:
            self.build()
        return self.model.call(x, mask)

So, what's that x mean and how to fix this error?
Thanks!

Answer 1

xiao, I ran into the same issue! Hopefully this helps:

Background and The Issue

The documentation for Keras states that, when implementing Wrappers for scikit-learn, there are two arguments. The first is the build function, which is a "callable function or class instance". Specifically, it states that:

build_fn should construct, compile and return a Keras model, which will then be used to fit/predict. One of the following three values could be passed to build_fn:

1. A function
2. An instance of a class that implements the call method
3. None. This means you implement a class that inherits from either KerasClassifier or KerasRegressor. The call method of the present class will then be treated as the default build_fn.

In your code, you create the model, and then pass the model as the value for the argument build_fn when creating the KerasRegressor wrapper:

model = nn_model()
model = KerasRegressor(build_fn=model, nb_epoch=2)

Herein lies the issue. Rather than passing your nn_model function as the build_fn, you pass an actual instance of the Keras Sequential model. For this reason, when fit() is called, it cannot find the call method, because it is not implemented in the class you returned.

Proposed Solution

What I did to make things work is pass the function as build_fn, rather than an actual model:

data = read_db()
y = data.pop('PRICE').as_matrix()
x = data.as_matrix()
# model = nn_model() # Don't do this!
# set build_fn equal to the nn_model function
model = KerasRegressor(build_fn=nn_model, nb_epoch=2) # note that you do not call the function!
model.fit(x,y)  # fixed!

This is not the only solution (you could set build_fn to a class that implements the call method appropriately), but the one that worked for me. I hope it helps you!

Answer 2

User-defined keyword arguments passed to __init__() that is to say, all keyword arguments that were given to __init__() will be passed to model_build_fn directly. For example, calling KerasClassifier(myparam=10) will result in a model_build_fn(my_param=10)

here's an example:

class MyMultiOutputKerasRegressor(KerasRegressor):
    
    # initializing
    def __init__(self, **kwargs):
        KerasRegressor.__init__(self, **kwargs)
        
    # simpler fit method
    def fit(self, X, y, **kwargs):
        KerasRegressor.fit(self, X, [y]*3, **kwargs)

(...)

def get_quantile_reg_rpf_nn(layers_shape=[50,100,200,100,50], inDim= 4, outDim=1, act='relu'):
          # do model stuff...

(...) initialize the Keras regressor:

base_model = MyMultiOutputKerasRegressor(build_fn=get_quantile_reg_rpf_nn,
                                         layers_shape=[50,100,200,100,50], inDim= 4, 
                                         outDim=1, act='relu', epochs=numEpochs, 
                                         batch_size=batch_size, verbose=0)