2

I am working on simple MLP neural network for MNIST dataset using tensorflow as my homework. in the question we should implement a multilayer perceptron with tanh as activation function. I should use the data label with [-1,+1].For example for number 3 we have:

[-1,-1,-1,+1,-1,-1,-1,-1,-1,-1]

I know that for sigmoid function we can use on_hot such as:

mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)

in order to putting data in [0,1] like the following for number 3:

[0,0,0,1,0,0,0,0,0,0]

how can I encode label between [-1 ,+1]. thanks in advance for every help

m.ar
  • 33
  • 6
  • May I ask, why do you want to do that ? – Jérémy Blain Nov 23 '18 at 09:55
  • because using when we are using the tanh, we should put values in this range@JérémyBlain – m.ar Nov 23 '18 at 09:58
  • Yeah I understand why you want to use tanh, but why the data are [-1; +1] ? Is it a requirement ? The data were saved this way ? – Jérémy Blain Nov 23 '18 at 10:04
  • 1
    yeah, in fact in the question it has been said that putting the value of active class to one and all another to -1. – m.ar Nov 23 '18 at 10:08
  • 1
    That's a very awkward requirement. The reason for one-hot encoding with 0s and 1s is that this is the format which is expected by the common form of cross-entropy losses. (Also, this has nothing to do with logsig or tanh, for multiclass classification, the output layer usually uses a softmax.) – cheersmate Nov 23 '18 at 11:15
  • 1
    Neither sigmoid or tanh are typically used for multi-class classification, so I don't get what exactly you want to do. – Dr. Snoopy Nov 23 '18 at 12:56

1 Answers1

2

Unnecessary downvotes to the question. BTW.. if I understood it correctly, here's the answer.

What I understood is, instead of using sigmoid, you have to use tanh and so you want the output data in format of +1s and -1s instead of 0s and 1s.

Note that one hot encoding is specifically designed for getting outputs of 1s and 0s. That's why it is called one hot encoding - it outputs 1 for right answer and 0 for others.

Now, there is no built-in function to get the output you want. But I prefer a short and simple way by writing my own code. Don't get afraid - that's only 1 line of code.

import numpy as np
a = np.array([1, 1, 1, 0, 1, 0])
a[a==0]=-1

The output is:

array([1, 1, 1, -1, 1, -1])

You can use the same.. Take the one hot encoding labels as output using your code and then use this one line of code to get what you want.

a[a==0]=-1

Thank you..

Kadam Parikh
  • 422
  • 4
  • 17