2

I want to use an autoencoder for dimension reduction in Keras. The input is a table with discrete values 0,1,2,3,4 (each of these numbers show a category) in the columns. Each subject has a label 0/1 to show sick/healthy. Now I have two questions:

  1. Which activation function should I use in the last layer? Shall I use a combination of sigmoid and ReLU?

  2. I don't know if this kind of input variables need normalization (and if the answer is yes, how?)

frogatto
  • 28,539
  • 11
  • 83
  • 129
Sasha
  • 21
  • 1

1 Answers1

2

Which activation function should I use in the last layer? Shall I use a combination of sigmoid and ReLU?

The activation in the last layer should be sigmoid and use binary_crossentropy loss function for training.

I don't know if this kind of input variables need normalization (and if the answer is yes, how?)

It depends on the nature of discrete values you mentioned. As you know, inputs to a neural network represents the "intensity" of each neurons; higher values mean the neuron being more intensive/active. So, categorical values as input to a NN only makes sense if they map to a continuous range. For example if excellent=3, good=2, bad=1, terrible=0, it's okay to feed these values to a NN because it makes sense to calculate f(wx+b) (intensity of the neuron) as a value of 1.5 means somewhere between bad and good.

However if the categorical values are pure nomial values without any relationship between them (for example: apple=1, orange=2, banana=3), it really doen't make sense to calculate the f(wx+b). In this case what does value 1.5 mean? For this type of data as input to a NN you should convert them to a binary encoding. For example if you have only 3 fruits you can encode this way:

apple  = [1, 0, 0]
orange = [0, 1, 0]
banana = [0, 0, 1]

For this binary conversion, Keras has an utility function: to_categorical.

frogatto
  • 28,539
  • 11
  • 83
  • 129