3

I want to get feature importance in Keras neural network by applying:

import eli5
from eli5.sklearn import PermutationImportance
perm = PermutationImportance(model, scoring='accuracy',
 random_state=1).fit(X_train,y_train)  


eli5.show_weights(perm, feature_names = X_train.columns.tolist())

and see the error:

Classification metrics can't handle a mix of binary and continuous targets

I've unsuccessfully tried to setup to PermutationImportance, scoring all the metrics from the sklearn list (sorted(sklearn.metrics.SCORERS.keys())):

    ['accuracy', 'adjusted_mutual_info_score', 'adjusted_rand_score', 'average_precision', 'balanced_accuracy', 'brier_score_loss', 

'completeness_score', 'explained_variance', 'f1', 'f1_macro', 'f1_micro',

'f1_samples', 'f1_weighted', 'fowlkes_mallows_score', 'homogeneity_score', 

'jaccard', 'jaccard_macro', 'jaccard_micro', 'jaccard_samples', 

'jaccard_weighted', 'max_error', 'mutual_info_score', 'neg_log_loss', 

'neg_mean_absolute_error', 'neg_mean_squared_error', 

'neg_mean_squared_log_error', 'neg_median_absolute_error', 

'normalized_mutual_info_score', 'precision', 'precision_macro',

'precision_micro', 'precision_samples', 'precision_weighted', 'r2', 'recall', 

'recall_macro', 'recall_micro', 'recall_samples', 'recall_weighted', 'roc_auc', 

'v_measure_score']

On the forums I read: this can be caused by the conflict of the purpose (binary classification) and the method (linear regression). But I don't use in linear regression in network (see code below).

Would you advise, please, how to solve the problem. Thank you.

model = Sequential()
model.add(Dense(1200, input_dim=len(X_train.columns), activation='relu'))  
model.add(Dense(150, activation='relu'))
model.add(Dense(1, activation='sigmoid')) 

loss="binary_crossentropy"  
optimizer=adam(lr=0.01)
metrics=['accuracy']
epochs = 2
batch_size = 32
verbose = 0

model.compile(loss=loss,  
              optimizer=optimizer, 
              metrics=metrics)

model.fit(X_train, y_train, 
          epochs = epochs, 
          batch_size=batch_size, 
          verbose = verbose)

# feature importance
import eli5
from eli5.sklearn import PermutationImportance
perm = PermutationImportance(model, scoring='accuracy', random_state=1).fit(X_train,y_train)      
eli5.show_weights(perm, feature_names = X_train.columns.tolist())
Alex Ivanov
  • 657
  • 1
  • 8
  • 17

0 Answers0