50

I am working on a simple cnn classifier using keras with tensorflow background.

def cnnKeras(training_data, training_labels, test_data, test_labels, n_dim):
  print("Initiating CNN")
  seed = 8
  numpy.random.seed(seed)
  model = Sequential()
  model.add(Convolution2D(64, 1, 1, init='glorot_uniform', 
   border_mode='valid',input_shape=(16, 1, 1), activation='relu'))
  model.add(MaxPooling2D(pool_size=(1, 1)))
  model.add(Convolution2D(32, 1, 1, init='glorot_uniform', 
   activation='relu'))
  model.add(MaxPooling2D(pool_size=(1, 1)))
  model.add(Dropout(0.25))
  model.add(Flatten())
  model.add(Dense(128, activation='relu'))
  model.add(Dropout(0.5))
  model.add(Dense(64, activation='relu'))
  model.add(Dense(1, activation='softmax'))
  # Compile model
  model.compile(loss='sparse_categorical_crossentropy',
              optimizer='adam', metrics=['accuracy'])
  model.fit(training_data, training_labels, validation_data=(
    test_data, test_labels), nb_epoch=30, batch_size=8, verbose=2)

  scores = model.evaluate(test_data, test_labels, verbose=1)
  print("Baseline Error: %.2f%%" % (100 - scores[1] * 100))
  # model.save('trained_CNN.h5')
  return None

It is a binary classification problem, but I keep getting the message Received a label value of 1 which is outside the valid range of [0, 1) which does not make any sense to me. Any suggesstions?

Tahjid Ashfaque
  • 781
  • 2
  • 8
  • 16

8 Answers8

72

Range [0, 1) means every number between 0 and 1, excluding 1. So 1 is not a value in the range [0, 1).

I am not 100% sure, but the issue could be due to your choice of loss function. For a binary classification, binary_crossentropy should be a better choice.

Michele Tonutti
  • 4,298
  • 1
  • 21
  • 22
30

In the last Dense layer you used model.add(Dense(1, activation='softmax')). Here 1 restricts its value from [0, 1) change its shape to the maximum output label. For eg your output is from label [0,7) then use model.add(Dense(7, activation='softmax'))

Cray
  • 2,774
  • 7
  • 22
  • 32
Shaili
  • 359
  • 5
  • 9
  • 4
    According to documentation (https://keras.io/layers/core/), here 1 is the output dimensionality (rank) rather than the output range. – Luis Kabongo Nov 05 '19 at 20:48
  • Here the output will be 7 layered, what does it mean? Would it be like each layer gives probability for it to be in the corresponding class? – fahd Mar 14 '23 at 15:35
23

Peculiarities of sparse categorical crossentropy

The loss function sparse_categorical_crossentropy interprets the final layer in the context of classifiers as a set of probabilities for each possible class, and the output value as the number of the class. (The Tensorflow/Keras documentation goes into a bit more detail.) So x neurons in output layer are compared against output values in the range from 0 to x-1; and having just one neuron in the output layer is an 'unary' classifier that doesn't make sense.

If it's a classification task where you want to have output data in the form from 0 to x-1, then you can keep sparse categorical crossentropy, but you need to set the number of neurons in the output layer to the number of classes you have. Alternatively, you might encode the output in a one-hot vector and use categorical crossentropy loss function instead of sparse categorical crossentropy.

If it's not a classification task and you want to predict arbitrary real-valued numbers as in a regression, then categorical crossentropy is not a suitable loss function at all.

Peteris
  • 3,281
  • 2
  • 25
  • 40
  • So, if one wanted to predict the 6th number, where the training data is [1,2,3,4,5] and there is one category: [6] , what would be your classification approach? I would assume keep the categorical crossentropy, and in this specific case, a 'unary' classifier might make sense. – spencer741 Apr 07 '20 at 22:50
  • If it's a classification task, then you're learning (i.e. tuning parameters of) a function where the set of possible outputs (i.e. the range of a function, in the technical mathematical definition of what a function is) is the list of categories. If there's only one category, this is the degenerate form where the range consists of a single value and you you have a function that by definition can't output anything other than that single category, no matter what inputs it's given, no matter if it's trained or not, it always outputs the only technically possible answer. – Peteris Apr 08 '20 at 00:09
  • So no, a unary classifier does not make sense ever, only as an example of a degenerate edge case that might technically count as a "learnable function". This comment seems to illustrate a fundamental misconception of what classification means - the classifier will try to predict one of the (predefined) possible answers. If there's only one category, there's nothing to predict, it's not possible to give a wrong answer because no other answer is possible - "there is one category: [6]" literally means that you're making an explicit assumption that [6] is the only possible output, no matter what. – Peteris Apr 08 '20 at 00:15
  • And "if one wanted to predict the 6th number, where the training data is [1,2,3,4,5] and there is one category: [6]" - if I'm understanding it as a task for predicting the next number in a sequence (so it would have to also work for [11,12,13,14,15]->[16] ) then *no* classification approach would be suitable, because it (just as any other task of predicting arbitrary real or natural numbers) is very much not a classification problem, it's violating core assumptions of classification problems. – Peteris Apr 08 '20 at 00:23
  • This site helped me pick my loss function https://machinelearningmastery.com/how-to-choose-loss-functions-when-training-deep-learning-neural-networks/ – Stevo Nov 25 '21 at 08:44
3

the error is in range [0,4) ,you can just add one to the number of classes(lables) . for example change this :

layers.Dense(4)

to :

layers.Dense(5)

**same for [0,1)

Abdellah Azzouzi
  • 31
  • 1
2

Cray and Shaili's answer was correct! I had a range of outcomes from 1 to 6, and the line:

tf.keras.layers.Dense(6, activation = 'softmax')

Produced that error message, saying that things were outside of the range [0,6). I had thought that it was a labels problem (were all values present in both the training and validation label sets?), and was flogging them.

)

Barry DeCicco
  • 251
  • 1
  • 7
1

I had this problem when I had labels of type "float", cast them it "int" and the problem was solved...

Romeo Kienzler
  • 3,373
  • 3
  • 36
  • 58
0

Another potential answer to this question is regarding the workspace. If it's not a logic/sparseness/entropy error as other answers suggest, keep reading:

If you created a workspace to hold the data as the model trained, the old workspace data can cause this error if you re-train the data with new samples, especially with a different number of folders and are using the folders as the labels for classification.

Example:

I trained my original set on:

Original Sample Set

and when I tried to retrain on the new Sample Set:

New Sample Set

I received the error: Received a label value of 3 which is outside the valid range of [0, 3)

This is likely because the old sample set's cached values of 4 folders versus the new sample set's 3 folders caused some kind of issue. All I know for sure is once I cleared the old information out of my workspace, and ran it again, it ran to completion. This was an isolated change after multiple failures, so I am certain it is what solved the issue.

Disclaimer: I am using C# and ML.NET, but it is still utilizing TensorFlow, which is where both of our errors were produced, so it should absolutely apply to the question.

Hashgrammer
  • 155
  • 1
  • 5
0

For me issue was that the number of class passed to model was less than the actual number of class in the data. Hence model predicted -1 for most case and thus giving error as out of range.

sakeesh
  • 919
  • 1
  • 10
  • 24