1

I am learning how to create sequential models. I have a model:

*model  =  Sequential()*

I then went on to add pooling layers and convolution layers (which were fine). But when creating the dense layer:

*model.add(Dense(num_classes, activation = 'softmax'))*

the line returned: 

*tf.nn.softmax(x, axis=axis)*

which caused an error as axis was not defined. Both Keras and TensorFlow documentation show that the default axis for softmax is None or -1.

Is this a bug with keras? And is there an easy workaround (If I were to set the axis I am not sure what the input tensor would be)?

-I can add the rest of the code if necessary, but it simply consists of the other layers and I don't think it will help much.

J Houseman
  • 91
  • 2
  • 10
  • Normally this works without any problem, and `axis=-1` indeed. But I'm not sure how you got to `tf.nn.softmax`. How did you get that? – Daniel Möller May 13 '18 at 23:33
  • I saw it in the traceback when trying to find the root of the error: line 2963, in softmax return tf.nn.softmax(x, axis=axis) TypeError: softmax() got an unexpected keyword argument 'axis' – J Houseman May 14 '18 at 00:06
  • So, actually, that softmax is "not" expecting an axis.... this might be an old version of tensorflow, no? – Daniel Möller May 14 '18 at 00:56
  • Yup that it! Was using tensorflow 1.4.0 – J Houseman May 14 '18 at 03:27

1 Answers1

2

I believe your Keras and/or TensorFlow is not up to date, you should update it/them.

This was a known problem in Keras in the Summer of 2017 and was fixed in this commit. See more on this comment on the bug report.

Also axis was introduced as an argument on November 22, 2017 in TensorFlow's softmax() so if the TensorFlow version is 1.4.0 or less, that would also cause this error.

Which one exactly causes the error depends on the rank of the tensor processed if you review the source of Keras at the linked commit.

This code is fine with the current versions (tested on https://colab.research.google.com):

import keras
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.optimizers import SGD

print( keras.__version__ )

model = Sequential()
model.add( Dense(6, input_shape=(6,), activation = 'softmax' ) )

sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy',
              optimizer=sgd,
              metrics=['accuracy'])

outputs

2.1.6

but more importantly, compiles the model with no error.

Peter Szoldan
  • 4,792
  • 1
  • 14
  • 24
  • 1
    I think, by the comment on the question, that it might be a problem in the tensorflow version instead. Saying that "softmax" got an unexpected argument "axis". – Daniel Möller May 14 '18 at 00:57
  • No, actually it refers to the Keras implementation of `softmax()` in the `Activations` package. If you follow the [link to the commit](https://github.com/keras-team/keras/commit/31d821d8785979f071c578ffaa20dda79914f49c) you see that Keras implements its own softmax based on `K.exp()`, `K.sum()`, etc. – Peter Szoldan May 14 '18 at 01:12
  • @DanielMöller Or maybe you're right, based on the error message, just re-read it. Would be nice to see the whole thing... – Peter Szoldan May 14 '18 at 01:16
  • 1
    @DanielMöller `axis` was [introduced as an argument on November 22, 2017](https://github.com/tensorflow/tensorflow/commit/b1d8c59e9b014b527fb2fbef9ce9afc14dbc4938#diff-7e066e8d1b9eff5c683b33517b35ae13) in TensorFlow's `softmax()` so yes, if the TensorFlow version is 1.4.0 or less, that would cause this error. – Peter Szoldan May 14 '18 at 01:28