binary threshold activation function in tensorflow

Question

I have a piece of code that uses sigmoid activation function for classification that outputs [0,1]. But I need a activation function that outputs binary values either 0 or 1.

        x = tf.placeholder("float", [None, COLUMN])
        Wh = tf.Variable(tf.random_normal([COLUMN, UNITS_OF_HIDDEN_LAYER], mean=0.0, stddev=0.05))
        h = tf.nn.sigmoid(tf.matmul(x, Wh))

        Wo = tf.Variable(tf.random_normal([UNITS_OF_HIDDEN_LAYER, COLUMN], mean=0.0, stddev=0.05))
        y = tf.nn.sigmoid(tf.matmul(h, Wo))

        # Objective functions
        y_ = tf.placeholder("float", [None, COLUMN])
        correct_prediction = tf.equal(tf.argmax(y, 1),tf.argmax(y, 1))
        cost = tf.reduce_sum(tf.cast(correct_prediction, "float"))/BATCH_SIZE

Can you please tell me how to replace sigmoid function with binary step one.

Ryan Jay · Accepted Answer · 2017-09-24T01:46:50.907

10

y = tf.round(tf.nn.sigmoid(tf.matmul(h,Wo))

that will give you 0 or 1 output.

edited Sep 24 '17 at 01:46

answered Sep 23 '17 at 12:47

Ryan Jay

838
8
9

score 6 · Answer 2 · answered Oct 11 '17 at 17:39

6

You don't need sigmoid in this case. Try relu(sign(x))

answered Oct 11 '17 at 17:39

Hongyu Sun

71
1

If you don't use a sigmoid would you then still be able to interpret it as a probability? – elgehelge Dec 29 '17 at 13:58
1

also used in BinaryNet https://github.com/itayhubara/BinaryNet.tf/blob/master/nnUtils.py – gizzmole Feb 22 '18 at 08:09

binary threshold activation function in tensorflow

2 Answers2