0

I'm trying to implement a simple keras autoencoder in R using the MNIST sample dataset. I got my example from a blog but it doesn't work. I get almost a 0 % accuracy.

The objective is to compress each 28 x 28 image (784 entries) into a vector of 32 entries:

Here's my code:

library(keras)
mnist <- dataset_mnist()
x_train <- mnist$train$x

# reshape
x_train <- array_reshape(x_train, c(nrow(x_train), 784))
x_train <- x_train / 255

model <- keras_model_sequential() 
model %>% 
 layer_dense(units = 32, activation = 'relu', input_shape = c(784)) %>% 
 layer_dense(units=784, activation='sigmoid')

model %>% compile(
  loss = 'categorical_crossentropy',
  optimizer = 'adam',
  metrics = c('accuracy')
)

history <- model %>% fit(
  x_train, x_train, 
  epochs = 15, batch_size = 128, 
  validation_split = 0.2
)
Random Cotija
  • 613
  • 1
  • 10
  • 26
animalcroc
  • 283
  • 4
  • 13

1 Answers1

2

You want to use binary_crossentropy as a loss function here. categorical_crossentropy is intended for multi-class classification problems (only one output is 1), binary_crossentropy is suitable for multi-label classification.

sdcbr
  • 7,021
  • 3
  • 27
  • 44
  • Thanks, I got it to run, but there is one more problem. I changed the input data from images to a random matrix that has positive and negative values. The decoded output only has positive values. Any idea why this is so? It seems like the autoenconder only wants and image (i.e. a matrix with positive values) – animalcroc Jul 30 '18 at 14:04
  • The output of a sigmoid function is always between 0 and 1. – sdcbr Jul 30 '18 at 14:28