How to customize Keras layer names and also have it automatically increment layer.name

Question

I am currently trying to have multiple layers with a customized activation with the name cust_sig. But when I try to compile the model, I get a ValueError raised as multiple layers have the same name cust_sig. I am aware that I can manually change the name for every layer but wanted to know if there is something that can be done to automatically have _1, _2, ... added to the name as it does for in-built layers. The model definition can be found below.

# Creating a model
from tensorflow.python.keras import keras
from tensorflow.python.keras.models import Model
from tensorflow.python.keras.layers import Dense

# Custom activation function
from tensorflow.python.keras.layers import Activation
from tensorflow.python.keras import backend as K
from keras.utils.generic_utils import get_custom_objects

def custom_activation(x):
    return (K.sigmoid(x) * 5) - 1

get_custom_objects().update({'custom_activation': Activation(custom_activation)})

data_format = 'channels_first'

spec_input = keras.layers.Input(shape=(1, 3, 256), name='spec')
x = keras.layers.Flatten(data_format)(spec_input)

for layer in range(3):
  x = Dense(512)(x)
  x = Activation('custom_activation', name='cust_sig')(x)

out = Dense(256, activation="sigmoid", name='out')(x)
model = Model(inputs=spec_input, outputs=out)

The error message is shown below

Traceback (most recent call last):
  File "/home/xyz/anaconda3/envs/ctf/lib/python3.7/site-packages/tensorflow/python/training/tracking/base.py", line 457, in _method_wrapper
    result = method(self, *args, **kwargs)
  File "/home/xyz/anaconda3/envs/ctf/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 315, in _init_graph_network
    self.inputs, self.outputs)
  File "/home/xyz/anaconda3/envs/ctf/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 1861, in _map_graph_network
    str(all_names.count(name)) + ' times in the model. '
ValueError: The name "cust_sig" is used 3 times in the model. All layer names should be unique.

The answer from https://stackoverflow.com/questions/53050448/custom-activation-with-parameter can also be used for reference — DVK, Oct 14 '19 at 17:55

OverLordGoldDragon · Accepted Answer · 2019-10-11T23:54:39.477

7

Below should do:

def custom_activation(x):
    return (K.sigmoid(x) * 5) - 1

class CustSig(Layer):
    def __init__(self, my_activation, **kwargs):
        super(CustSig, self).__init__(**kwargs)
        self.supports_masking = True
        self.activation = my_activation

    def call(self, inputs):
        return self.activation(inputs)

    def get_config(self):
        config = {'activation': activations.serialize(self.activation)}
        base_config = super(Activation, self).get_config()
        return dict(list(base_config.items()) + list(config.items()))

    def compute_output_shape(self, input_shape):
        return input_shape

Explanation:

From source code, automatic naming works as follows:

if not name:
  self._name = backend.unique_object_name(
      generic_utils.to_snake_case(self.__class__.__name__),
      zero_based=zero_based)
else:
  self._name = name

The Keras graph is checked for existing objects with the same name as the object you're defining - if any exist, continues to increment by 1 until none match. The catch is, you cannot specify name=, as that eliminates automatic naming per above conditional.

The only workaround is likely defining your own custom activation layer using desired name as class name as above, which manifests itself as follows:

ipt = Input(shape=(1, 3, 256), name='spec')
x   = Flatten('channels_last')(ipt)
for _ in range(3):
    x   = Dense(512)(x)
    x   = CustSig(custom_activation)(x)
out = Dense(256, activation='sigmoid', name='out')(x)

model = Model(ipt, out)

print(model.layers[3].name)
print(model.layers[5].name)
print(model.layers[7].name)

cust_sig
cust_sig_1
cust_sig_2

edited Oct 11 '19 at 23:54

answered Oct 11 '19 at 22:50

OverLordGoldDragon

1
9
53
101

Thank you, but as I mentioned I am aware I can change the name manually but wanted to know how to get Keras to add the increment as it does for defined layers such as ReLU, Dense, etc. – DVK Oct 11 '19 at 23:00
@DeepakKadetotad It'd help if you shared the full model code, as the code as-is throws errors for the custom layer, or is ignored after first call under `CustomObjectScope` – OverLordGoldDragon Oct 11 '19 at 23:17
I have included the entire model definition and the error that is thrown, hope it helps – DVK Oct 11 '19 at 23:26
@DeepakKadetotad Are you sure you copied your code correctly? The very first line is an invalid Python syntax, the for-loop is incorrectly indented, `data_format` is unspecified (though I guessed it), and `ValueError: Unknown activation function:custom_activation` is thrown – OverLordGoldDragon Oct 11 '19 at 23:28
The code I am using much larger but I have corrected the issues you pointed out. However I do not get any `Unknown activation function:custom_activation` errors – DVK Oct 11 '19 at 23:36
@DeepakKadetotad If I cannot reproduce the issue as you're having it, I can't properly debug it; I suggest running the code you post, as the current edit doesn't compile either (first line's still faulty). Regardless, I do have a suggested code I'm working on now, whether or not it directly modifies yours – OverLordGoldDragon Oct 11 '19 at 23:38
@OverLordGoldDragon Use `import keras` style instead of `from tensorflow.python.keras` style will fix the issue. – zihaozhihao Oct 11 '19 at 23:41
@zihaozhihao I'm aware, the idea was to have the user provide valid code, for their own sake. And it appears your answer beat me to it - good work. – OverLordGoldDragon Oct 11 '19 at 23:51

zihaozhihao · Answer 2 · 2019-10-11T23:47:32.497

If you check the source code of Layer class, you can find these lines that decide the name of layer.

if not name:
    prefix = self.__class__.__name__
    name = _to_snake_case(prefix) + '_' + str(K.get_uid(prefix))
self.name = name

K.get_uid(prefix) will get the unique id from the graph, that's why you see activation_1, activation_2.

If you want to have the same effect on your customized activation function, a better way is to define your own class which inherits from Layer.

class MyAct(Layer):
    def __init__(self):
        super().__init__()

    def call(self, inputs):
        return (K.sigmoid(inputs) * 5) - 1 

spec_input = Input(shape=(10,10))
x = Flatten()(spec_input)
for layer in range(3):
    x = Dense(512)(x)
    x = MyAct()(x)

model = Model(spec_input, x)
model.summary()

Output

# Layer (type)                 Output Shape              Param #   
# =================================================================
# input_1 (InputLayer)         (None, 10, 10)            0         
# _________________________________________________________________
# flatten_1 (Flatten)          (None, 100)               0         
# _________________________________________________________________
# dense_1 (Dense)              (None, 512)               51712     
# _________________________________________________________________
# my_act_1 (MyAct)             (None, 512)               0         
# _________________________________________________________________
# dense_2 (Dense)              (None, 512)               262656    
# _________________________________________________________________
# my_act_2 (MyAct)             (None, 512)               0         
# _________________________________________________________________
# dense_3 (Dense)              (None, 512)               262656    
# _________________________________________________________________
# my_act_3 (MyAct)             (None, 512)               0         
# =================================================================
# Total params: 577,024
# Trainable params: 577,024
# Non-trainable params: 0

score 3 · Answer 3 · answered Nov 09 '19 at 08:22

If you want to use a specific_name multiple times with number suffix, use this:

tf.get_default_graph().unique_name("specific_name")

or

tf.compat.v1.get_default_graph().unique_name("specific_name")

In your case:

...
for layer in range(3):
  x = Dense(512)(x)
  x = Activation('custom_activation', name=tf.get_default_graph().unique_name("cust_sig"))(x)
...

How to customize Keras layer names and also have it automatically increment layer.name

3 Answers3