Machine Learning - SVM - How to calculate bias when calculate vector W?

Question

I am writing code of SVM Primal that uses SGD (Stochastic SubGradient Descent) for optimize the vector W.

The classification methos is sign(w*x + bias).
My question is how to find the best bias for it?
I guess that it has to do during the W optimizing, but how? I have no idea.

Python, but it's not relevant, I am not going to use scikit-learn, I want to do it myself. — zardav, Feb 06 '16 at 17:53

lejlot · Accepted Answer · 2016-02-06T22:56:02.677

1

Your hypothesis is sign(<w, x> + b), think for a second about x' = [x 1], then you could express your hypothesis as sign(<w', x'>), where w' = [w b]. I hope it clearly shows that b is not any different from w's (the only difference is that your regularizing term ||w||^2 does not involve b). Thus you just need d L/ d b where L is your loss function.

edited Feb 06 '16 at 22:56

answered Feb 06 '16 at 18:04

lejlot

64,777
8
131
164

Thanks. I don't understand why `x' = [x 1]` if `w' = [w b]`, can you explain it? – zardav Feb 06 '16 at 18:07
+ b = + <[b], [1]> = <[w b], [x 1]> = (by writing [x 1] I mean "x with concatenated 1 as a new dimension") – lejlot Feb 06 '16 at 18:09
Thank you very much! (the last sentece I understood myself...) – zardav Feb 06 '16 at 18:13

Machine Learning - SVM - How to calculate bias when calculate vector W?

1 Answers1