1

New to Keras/Machine Learning. I figure I am making a dumb mistake but I don't know what.

I have 3 labels. The training data for each sequence of timesteps is labeled as [1, 0, 0] or [0, 1, 0], or [0, 0, 1].

I always get a distribution that looks something like this. You can't tell in the photo, but the numbers aren't the same when you zoom in or look at the actual data results.

https://i.stack.imgur.com/4LLdc.jpg

The actual results is just color coding that spot based on the category above, so the values are all 1 but the labels are always one of the above.

model = Sequential()
model.add(LSTM(units=50, return_sequences=False, input_shape=(num_timesteps, num_features))
model.add(Dense(3, activation="softmax"))
model.compile(optimizer='adam', loss="categorical_crossentropy", metrics=["accuracy"])
model.fit(x_train, y_train, epochs=100, validation_data=(x_test, y_test))

results = model.predict(x_train)

I can change the number of sequences, timesteps, features, epochs, add other lstm layers. The distribution will change but always be like that. I'm expecting based on the data (and based on even just making things random), that the probabilities would be varied and not always discretely layered.

I originally did this with just a regular Dense layer and then Dense(3) layer to categorize and I was getting results that went with that expectation. Switching to LSTM due to the type of data and no longer getting expected results but same data

Michael M.
  • 10,486
  • 9
  • 18
  • 34
Pickles
  • 11
  • 1
  • Its not clear to me what you are plotting, are these the probabilities of the correct class, or what? – Dr. Snoopy Dec 02 '22 at 16:57
  • Sorry about the confusion. They are the probabilities in a 3 category system. So, my labels are (1,0,0) (0,1,0) (0,0,1). The actual is the color coding of those training labels. The prediction is color coded to those as well. So, the prediction is 33% almost evenly split. – Pickles Dec 03 '22 at 20:52
  • I think I may have discovered the issue though. Once I added a lot more data, the probabilities widened and stopped layering so discreetly. I think maybe my model wasn't finding anything specific so the probability was split evenly between every category? – Pickles Dec 03 '22 at 20:53

0 Answers0