Random bits array by given probabilities with numpy

Question

I have a deterministic neural network and I want to make it stochastic.

Two questions:

I'm not sure if it means that I need to use the result of the sigmoid to determine the probabilities for the output, or if the probabilities are simply the neurons input, and a sigmoid function is now redundant.
How to do that efficiently with numpy? I know how to generate random bits, but how do you do that with given probabilities inside a large array? (My current sigmoid function is tanh if it matters)

Uri · Accepted Answer · 2013-04-24T17:52:15.147

0

The sigmoid function is still required, as the backpropagation works on computing the derivative of the sigmoid function, and not whether or not the neuron fired.
After computing the activation as before, I now run the result array x through this:

return numpy.random.ranf(x.shape) < x

My timing for this is 3.03323280772e-05

Also note that numpy treats boolean values as if they were 1 and 0 so no need to transfer the result back to int/float). Because this is now 0 and 1, I had to change my code a bit - before I used -1 and 1 for target values and inputs.

edited Apr 24 '13 at 17:52

answered Apr 24 '13 at 15:59

Uri

1

Or `return (numpy.random.ranf(x.shape) > x)*2 - 1` – askewchan Apr 24 '13 at 16:40
@askewchan I think it's < instead of >. Original mistake was mine :) – Uri Apr 24 '13 at 17:53
Yes, assuming `x` is an array of probabilities normalized to `1`. My point was simply that it's very easy to change boolean to `{-1, 1}` – askewchan Apr 24 '13 at 19:59

1 Answers1