2

I am just getting into Keras and Tensor flow. Im having a lot of problems adding an input normalization layer in a sequential model. Now my model is ;

 model = tf.keras.models.Sequential()
 model.add(keras.layers.Dense(256, input_shape=(13, ), activation='relu'))
 model.add(tf.keras.layers.LayerNormalization(axis=-1 , center=True , scale=True))
 model.add(keras.layers.Dense(128, activation='relu'))
 model.add(keras.layers.Dense(64, activation='relu'))
 model.add(keras.layers.Dense(64, activation='relu'))
 model.add(keras.layers.Dense(1))
 model.summary()

My doubts are whether I should first perform an adapt function and how to use it in the sequential model. Thanks to all!!

Eduardo Perona Jiménez
  • 23
  • 1
  • 3

1 Answers1

0

I'm trying to figure this out as well. According to this example, adapt is not necessary.

model = tf.keras.models.Sequential([
  # Reshape into "channels last" setup.
  tf.keras.layers.Reshape((28,28,1), input_shape=(28,28)),
  tf.keras.layers.Conv2D(filters=10, kernel_size=(3,3),data_format="channels_last"),
  # LayerNorm Layer
  tf.keras.layers.LayerNormalization(axis=3 , center=True , scale=True),
  tf.keras.layers.Flatten(),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])
model.fit(x_test, y_test)

Also, make sure you want a LayerNormalization. If I understand correctly, that normalizes every input on its own. Batch normalization may be more appropriate. See this for more info.

galactikuh
  • 765
  • 8
  • 18
  • 1
    The idea was to normalize the inputs, finally I could do it like this in a previous step to the model; norm = tf.keras.layers.experimental.preprocessing.Normalization(axis=-1, dtype=None, mean=None, variance=None) norm.adapt(x_train) x_train = norm(x_train). Thank you very much for your help! – Eduardo Perona Jiménez May 19 '21 at 20:11
  • 3
    So you had to do this prior to the model layers, directly on the actual data? I ask since it could be better to have normalization incoroporated in the model if the model is going to be exported to some other environment where normalization is not so easy to do. – Oortone Nov 16 '21 at 08:50