I am trying to extract feature importance from my 1D CNN. Most of the online documentation refers to 2D, 3D, image data and classification problems. I have a multivariate time series that outputs time series sequences. I have tried Shaply and keras_vis, but nothing addresses my issue. One issue is that my input data has 229 features and the first layer of the 1DConv maps 64 filters. Extracting the weights from the first layer, I can determine what filter is contributing to the learning of the layer. However, I cannot translate this to the original output.
My question is two-fold (if I can do both):
- Given the weights in the first layer, how can I visualize the saliency, of a particular filter?
- How can I extract the important features; given my specific problem and arch.?
Here is the summary and code used to extract the weights....
Model: "model_3"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
CNN1-Input-Historical (InputLay [(None, 10, 229)] 0
__________________________________________________________________________________________________
CNN1-Conv1D (Conv1D) (None, 7, 64) 58688 CNN1-Input-Historical[0][0]
__________________________________________________________________________________________________
conv1d_3 (Conv1D) (None, 3, 128) 24704 CNN1-Conv1D[0][0]
__________________________________________________________________________________________________
CNN1-MaxPooling (MaxPooling1D) (None, 2, 128) 0 conv1d_3[0][0]
__________________________________________________________________________________________________
CNN1-Flatten (Flatten) (None, 256) 0 CNN1-MaxPooling[0][0]
__________________________________________________________________________________________________
dense_9 (Dense) (None, 50) 12850 CNN1-Flatten[0][0]
__________________________________________________________________________________________________
dense_10 (Dense) (None, 50) 2550 dense_9[0][0]
__________________________________________________________________________________________________
dropout_6 (Dropout) (None, 50) 0 dense_10[0][0]
__________________________________________________________________________________________________
dense_11 (Dense) (None, 100) 5100 dropout_6[0][0]
__________________________________________________________________________________________________
dropout_7 (Dropout) (None, 100) 0 dense_11[0][0]
__________________________________________________________________________________________________
Output-1 (Dense) (None, 3) 303 dropout_7[0][0]
__________________________________________________________________________________________________
Output-2 (Dense) (None, 3) 303 dropout_7[0][0]
__________________________________________________________________________________________________
Output-3 (Dense) (None, 3) 303 dropout_7[0][0]
==================================================================================================
Total params: 104,801
Trainable params: 104,801
Non-trainable params: 0
model.summary()
outputs = [layer.output for layer in model.layers[1:]]
activation_model = Model(inputs=model.input, outputs=outputs)
activations = activation_model.predict(X.reshape(1,10,229))#3-d 10 timesteps and 229 featrues
first_layer_activation = activations[0]
second_layer_activation = activations[1]
