Training sparse model with Keras

Question

I'm trying to train a sparse model, that is some of the model parameters have to remain zero during optimization.

Is this possible in Keras to define a mask for the parameters so that the optimizer would not update masked ones?

Unfortunately, freezing one layer would not work as I need to mask parameters in a more fine-grained fashion.

Answer 1

You can use tf.where to select elementwise between the parameters and tf.stop_gradient(parameters).