3

So I was reading the pytorch document trying to learn and understand somethings(because I'm new to the machine learning ), I found the torch.bernoulli() and I understood (I miss understood it) that it approximates the tensors that the have the values between 1 and 0 to 1 or 0 depends on the value (like classic school less than 0.5 = 0 , more the than or equals 0.5 = 1)

After some experimentations on my own that yeah it works as expected

 >>>y = torch.Tensor([0.500])
 >>>x
 >>>  0.5000
     [torch.FloatTensor of size 1]    
 >>> torch.bernoulli(x)
 >>> 1
     [torch.FloatTensor of size 1]

But when I looked at the document something a bit weird

>>> a = torch.Tensor(3, 3).uniform_(0, 1) # generate a uniform random matrix with range [0, 1]
>>> a

 0.7544  0.8140  0.9842
**0.5282** 0.0595  0.6445
 0.1925  0.9553  0.9732
[torch.FloatTensor of size 3x3]

>>> torch.bernoulli(a)

 1  1  1
 **0**  0  1
 0  1  1
[torch.FloatTensor of size 3x3]

in the example the 0.5282 got approximated to 0 , how did that happen ? or it's a fault in the document because I tried it and the 0.5282 got approximated as expected to 1.

Mostafa Osama
  • 316
  • 3
  • 15

1 Answers1

6

Well, Bernoulli is a probability distribution. Specifically, torch.distributions.Bernoulli() samples from the distribution and returns a binary value (i.e. either 0 or 1). Here, it returns 1 with probability p and return 0 with probability 1-p.

Below example will make the understanding clear:

In [141]: m =  torch.distributions.Bernoulli(torch.tensor([0.63]))

In [142]: m.sample() # 63% chance 1; 37% chance 0
Out[142]: tensor([ 0.])

In [143]: m.sample() # 63% chance 1; 37% chance 0
Out[143]: tensor([ 1.])

In [144]: m.sample() # 63% chance 1; 37% chance 0
Out[144]: tensor([ 0.])

In [145]: m.sample() # 63% chance 1; 37% chance 0
Out[145]: tensor([ 0.])

In [146]: m.sample() # 63% chance 1; 37% chance 0
Out[146]: tensor([ 1.])

In [147]: m.sample() # 63% chance 1; 37% chance 0
Out[147]: tensor([ 1.])

In [148]: m.sample() # 63% chance 1; 37% chance 0
Out[148]: tensor([ 1.])

In [149]: m.sample() # 63% chance 1; 37% chance 0
Out[149]: tensor([ 1.])

In [150]: m.sample() # 63% chance 1; 37% chance 0
Out[150]: tensor([ 1.])

In [151]: m.sample() # 63% chance 1; 37% chance 0
Out[151]: tensor([ 1.])

So, we sampled it 10 times, out of which we got 1s 7 times which is approximately close to 63%. We need to sample this finitely large number of times to get the exact percentage of 37 and 63 for 0s and 1s respectively; This is because of the Law of Large Numbers.

kmario23
  • 57,311
  • 13
  • 161
  • 150
  • 2
    correct answer :) However the theorem you are thinking about is the law of large numbers – Ant Jul 18 '18 at 09:38
  • 1
    that means that I miss understood the function, does that mean that mean the more we're close to 1 the higher the chance to will 1 and vise versa, it's not about approximation ,thanks a lot for the answer – Mostafa Osama Jul 18 '18 at 11:49
  • 1
    Yes, it does not do any rounding; Instead, it just return a sample `0` or `1` based on the given probability `p` and `1-p` resp. – kmario23 Jul 18 '18 at 11:52
  • 1
    yeah i get it like in your example the probabilities were: p= 0,63 and 1 - p = 1-0.63 = 0.37 – Mostafa Osama Jul 18 '18 at 12:08