Numerical Prediction with MX-Gluon ends up in a loop

Question

I'm trying to build a small MLP Network to predict some numerical values. The input are 6 different paramaters and the result should be a prediction of the target value. The MLP consists of 2 hidden layers and 1 output layer with exactly one Neuron. For the activation function I chose to try out Swish. I'm using MXNet Gluon. I already seperated my dataset in 3 parts and tried to use cross-validation. The validation set however is not evaluated in the code since my predictions are gibberish.

for epoch in range(num_epochs):
    for cross_it in range(k):
        indices = list(range((cross_it*n), ((cross_it+1)*n)))
        x_cvalidate = x_train[(cross_it*n):((cross_it+1)*n)]
        x_ctrain = np.delete(x_train, indices, axis=0)
        y_cvalidate = y_train[(cross_it*n):((cross_it+1)*n)]
        y_ctrain = np.delete(y_train, indices, axis=0)
        ## Declare Datasets
        dataset = mx.gluon.data.dataset.ArrayDataset(x_ctrain, y_ctrain)
        train_set = mx.gluon.data.DataLoader(dataset, batch_size=batch_size, shuffle=True)
        valid_set = mx.gluon.data.DataLoader(mx.gluon.data.dataset.ArrayDataset(x_cvalidate, y_cvalidate), batch_size=batch_size)
        ##
        for x,y in train_set:
            with mx.autograd.record():
                loss = mse(mlp_model(x), y)
            loss.backward()
            trainer.step(batch_size)
        loss = mse(mlp_model(nd.array(x_ctrain)), nd.array(y_ctrain))
        cr_loss.append(loss.mean().asnumpy())
        print('Cross-Validation Epoch: ' + str(cross_it) + ' Total Loss: ' + str(loss.mean().asnumpy()))

After executing this code with f.e. num_epochs = 3 I always get the same results.

Cross-Validation Epoch: 0 Total Loss: [22.890377]
Cross-Validation Epoch: 1 Total Loss: [25.228876]
Cross-Validation Epoch: 2 Total Loss: [17.016602]
Cross-Validation Epoch: 0 Total Loss: [22.890377]
Cross-Validation Epoch: 1 Total Loss: [25.228876]
Cross-Validation Epoch: 2 Total Loss: [17.016602]
Cross-Validation Epoch: 0 Total Loss: [22.890377]
Cross-Validation Epoch: 1 Total Loss: [25.228876]
Cross-Validation Epoch: 2 Total Loss: [17.016602]
Cross-Validation Epoch: 0 Total Loss: [22.890377]
Cross-Validation Epoch: 1 Total Loss: [25.228876]
Cross-Validation Epoch: 2 Total Loss: [17.016602]
Cross-Validation Epoch: 0 Total Loss: [22.890377]
Cross-Validation Epoch: 1 Total Loss: [25.228876]
Cross-Validation Epoch: 2 Total Loss: [17.016602]

It seems like the parameters are just updated in the inner loop. However this would never work with cross-validation since I simply can't just change the value k. Does anyone have an idea what I did wrong? I'm brainlagging hard here. Thanks!

Numerical Prediction with MX-Gluon ends up in a loop

0 Answers0