4

I have keras pretrained model(model.h5). And I want to prune that model with tensorflow Magnitude-based weight pruning with Keras. One curious things is that my pretrained model is built with original keras model > I mean that is not from tensorflow.keras. Inside tensorflow Magnitude-based weight pruning with Keras example, they show how to do with tensorflow.keras model. I want to ask is that can I use their tool to prune my original keras pretrained model?

inside their weight pruning toolkit ,there is two way. one is pruned the model layer by layer while training and second is pruned the whole model. I tried the second way to prune the whole pretrained model. below is my code. inside their weight pruning toolkit ,there is two way. one is pruned the model layer by layer while training and second is pruned the whole model. I tried the second way to prune the whole pretrained model. below is my code. For my original pretrained model, I load the weight from model.h5 and can call model.summary() after I apply prune_low_magnitude() none of the method from model cannot call including model.summary() method. And show the error like AttributeError: 'NoneType' object has no attribute 'summary'

model = get_training_model(weight_decay)
model.load_weights('model/keras/model.h5')
model.summary()


epochs = 1
end_step = np.ceil(1.0 * 100 / 2).astype(np.int32) * epochs
print(end_step)

new_pruning_params = {
      'pruning_schedule': tfm.sparsity.keras.PolynomialDecay(initial_sparsity=0.1,
                                                   final_sparsity=0.90,
                                                   begin_step=40,
                                                   end_step=end_step,
                                                   frequency=30)
}

new_pruned_model = tfm.sparsity.keras.prune_low_magnitude(model, **new_pruning_params)
print(new_pruned_model.summary())

Inside their weight pruning toolkit enter link description here ,there is two way. one is pruned the model layer by layer while training and second is pruned the whole model. I tried the second way to prune the whole pretrained model. below is my code. For my original pretrained model, I load the weight from model.h5 and can call model.summary() after I apply prune_low_magnitude() none of the method from model cannot call including model.summary() method. And show the error like

AttributeError: 'NoneType' object has no attribute 'summary'

kaythi swetun
  • 53
  • 10

1 Answers1

1

I hope this answer still helps, but I recently had the same issue that prune_low_magnitude() returns an object of type 'None'. Also new_pruned_model.compile() would not work.

The model I had been using was a pretrained model that could be imported from tensorflow.python.keras.applications.

For me this worked:

(0) Import the libraries:

from tensorflow_model_optimization.python.core.api.sparsity import keras as sparsity
from tensorflow.python.keras.applications.<network_type> import <network_type>

(1) Define the pretrained model architecture

# define model architecture
loaded_model = <model_type>()
loaded_model.summary()

(2) Compile the model architecture and load the pretrained weights

# compile model
opt = SGD(lr=learn_rate, momentum=momentum)
loaded_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
loaded_model.load_weights('weight_file.h5')

(3) set pruning parameters and assign pruning schedule

# set pruning parameters
pruning_params = {
  'pruning_schedule': sparsity.PolynomialDecay(...)
}

# assign pruning schedule
model_pruned = sparsity.prune_low_magnitude(loaded_model, **pruning_params)

(4) compile model and show summary

# compile model
model_pruned.compile(
loss=tf.keras.losses.categorical_crossentropy,
optimizer='SGD',
metrics=['accuracy'])

model_pruned.summary()

It was important to import the libraries specifically from tensorflow.python.keras and use this keras model from the TensorFlow library.

Also, it was important to use the TensorFlow Beta Release (pip install tensorflow==2.0.0b1), otherwise still an object with type 'None' would be returned by prune_low_magnitude.

I am using PyCharm 2019.1.3 (x64) as IDE. Here is the link that led me to this solution: https://github.com/tensorflow/model-optimization/issues/12#issuecomment-526338458

asti205
  • 154
  • 7