0

I tried to implementing the multiclass SVM classifier which uses the Hinge loss.

The below is the code I implemented

X_data, y_data = DF.drop(['label'], axis=1).values, DF.label.values
X_train, y_train, X_test, y_test = data_split(X_data, y_data)

X_train.shape, y_train.shape, X_test.shape, y_test.shape

shape = X_train[0].shape[0]
num_classes = len(np.unique(y_train))
loss = 0

w = np.random.uniform(0, 1, [num_classes, shape]) # initializing weights
delta = 1
lr = 0.0001

for x, y in zip(X_train, y_train): # training

    # inference
    dw = np.zeros_like(w)       # weight gradient
    scores = x.dot(w.T)
    correct_class_score = scores[y]

    # loss
    for i in range(num_classes):

        if i == y:
            continue

        res = scores[i] - correct_class_score + delta
        loss += np.max([0, res])

    # gradient
    for j in range(num_classes):

        if j == y:
            if scores[i] - correct_class_score + delta > 0:
                dw[j,:] = -x
        else:
            if scores[i] - correct_class_score + delta > 0 > 0:
                dw[j,:] = x

    w -=  (lr * dw)

the below is formula for this code from https://cs231n.github.io/linear-classify/#svm.

enter image description here

I want to know the code is right and better implementation for that.

Thanks in advance.

desertnaut
  • 57,590
  • 26
  • 140
  • 166
JaeJu
  • 85
  • 1
  • 11
  • Have you tested the code over some data? – Yonlif Apr 15 '20 at 11:11
  • @Yonlif Yeah i tested it with iris dataset. I got 0.33% accuracy, which is i think the model is not trained at all. So I wanna figure out is there any logical error – JaeJu Apr 15 '20 at 11:15
  • Did you [shuffle](https://stackoverflow.com/questions/61215675/data-shuffling-for-image-classification/61218707#61218707) the iris data before? – desertnaut Apr 15 '20 at 11:22
  • @desertnaut. Nah I did not shuffle it. – JaeJu Apr 15 '20 at 11:24
  • It can be an issue - take a look at the answer linked above and [this one](https://stackoverflow.com/questions/55302500/cross-val-score-is-not-working-with-roc-auc-and-multiclass/55309222#55309222). – desertnaut Apr 15 '20 at 11:26
  • @desertnaut. I checked it , but I do not think the problem is not for me. – JaeJu Apr 15 '20 at 11:40
  • Just sayin - it's always good to perform such sanity checks when trying ti implement something from scratch. In any case, if you don't *show* any issue here, the question is probably off-topic for SO. – desertnaut Apr 15 '20 at 11:43
  • @desertnaut Yep! – JaeJu Apr 15 '20 at 15:00

0 Answers0