1

I have a multi-label problem and with some research, I was able to use Label powerset in conjunction with ML algorithms. Now I want to use the Label powerset with neural network and as per the official website I can use Label powerset. But I am not able to understand how to modify my existing code to be able to use Label Powerset.

I want to know how can we pass epoch or batch_size or any other parameter passed in the fit function of the model.

Since I have a multi-label problem I have used MultiLabelBinarizer of sklearn so my each target row looks like this [1,0,0,1,0,0,0,0,0,0,0,0].

and lastly, if someone could explain to me what is KERAS_PARAMS and Keras() in the below line:

def create_model_multiclass(input_dim, output_dim):
    # create model
    model = Sequential()
    model.add(Dense(8, input_dim=input_dim, activation='relu'))
    model.add(Dense(output_dim, activation='softmax'))
    # Compile model
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model
clf = LabelPowerset(classifier=Keras(create_model_multiclass, True, KERAS_PARAMS), require_dense=[True,True])
clf.fit(X_train,y_train)
y_pred = clf.predict(X_test)

Below is my existing neural network code

cnn_model = Sequential()
cnn_model.add(Dropout(0.5))
cnn_model.add(Conv1D(25,7,activation='relu'))
cnn_model.add(MaxPool1D(2))
cnn_model.add(Dropout(0.2))
cnn_model.add(Conv1D(25,7,activation='relu'))
cnn_model.add(MaxPool1D(2))
cnn_model.add(Flatten())
cnn_model.add(Dense(25,activation='relu'))
cnn_model.add(Dense(12,activation='softmax'))
cnn_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc'])
history = cnn_model.fit(X_train, y_train, validation_data=(X_test,y_test), batch_size=32, epochs=180,verbose=1)
plot_history(history)
predictions = cnn_model.predict(X_test)

I want my output row to look like this only [1,0,0,1,0,0,0,0,0,0,0,0] as later I will use my MultiLabelBinarizer for the inverse transform of this.

Shubham Trivedi
  • 130
  • 8

1 Answers1

1

KERAS_PARAMS are parameters to the Keras scikit wrapper. The documentation for it is rather sparse.

Basically it seems to be the params that you would pass, for instance, to keras.fit.

KERAS_PARAMS = dict(epochs=10, batch_size=100, verbose=0)

From reading the docs, it seems to me that LabelPowerset transforms a multi-label problem into a multi-class problem by creating class permutations. You may consider just using a native Keras solution for a multi-label problem rather than using a wrapper.

The following tutorial seems reasonable: https://medium.com/@vijayabhaskar96/multi-label-image-classification-tutorial-with-keras-imagedatagenerator-cd541f8eaf24

The key differences are that your output layer should have a sigmoid activation rather than softmax and the loss should be binary_crossentrophy rather than categorical.

Pedro Marques
  • 2,642
  • 1
  • 10
  • 10
  • The thing is LabelPowerset makes the combinations of all the different classes and those combinations become one class making the problem from Multi-Label to Multi-Class so, wouldn't it be significant to use softmax for the multi-class problem with loss as categorical_crossentropy? – Shubham Trivedi Jun 25 '19 at 08:34
  • Suppose you have three classes A, B, C. And the end goal is to be able to have a multi-label classification where you get a probability of these classes being present. This can be achieve with Keras directly using 'sigmoid' activation / binary_crossentrophy loss. Or you can have LabelPowerset create artificial classes with each set permutation e.g. Empty, A, B, C, AB, AC, BC, ABC and use a Keras multi-class model (softmax / categorical_crossentrophy). To me it would seem that the first approach is preferable. – Pedro Marques Jun 25 '19 at 09:17
  • In case anyone wonders what is Keras(). It is a MEKA wrapper and if you get are not sure how to use is just use this line ``` from skmultilearn.ext import Keras ``` – Shubham Trivedi Jun 25 '19 at 10:24