Binary features with 0,0,0, in NN always return 0.5

Question

Let's assume we have three columns, with binary features (0,1). One row in dataset is 0,0,0 with label 0.

The problem I am facing is:

When assigning weights to this row and activating sigmoid function, I'll always receive 0.5, because an array of zeros products on any weight with 0.

How to overcome this issue?

score 0 · Accepted Answer · answered Jul 11 '17 at 20:29

0

In addition to multiplication by a weight matrix, you can also add a bias (which is how it is typically done in neural networks), and hence you won't necessarily get a zero vector. You could also add more hidden layers (but as I said, even adding a bias vector will resolve the issue you mentioned).

answered Jul 11 '17 at 20:29

Miriam Farber

18,986
14
61
76

Thanks Miriam. May I just ask. Is it recomended to put bias for every hidden layer or it is enough for one (first one only). Bececause imagine, if I include bias in first hidden layer and countinue with propagantion to second layer, the result of second layer would act the same as my problem. – Makaroniiii Jul 11 '17 at 21:02
@Makaroniiii Typically every layer contains a bias vector. – Miriam Farber Jul 11 '17 at 21:11
Thank you very much. Have a nice day – Makaroniiii Jul 12 '17 at 10:24

Binary features with 0,0,0, in NN always return 0.5

1 Answers1