How to add a Validation and Test Set in Pytorch Model?

Question

I have Non Linear Regression Model ANN( X = [1000,3] , Y = [1000,8] ) with One hidden Layer(Nh = 6).

How to add a Validation(10% Dataset) and Test Set(10% Dataset) in this model ?

Model :

N, D_in, H, D_out = x.shape[0], x.shape[1], 6, y.shape[1]

model = nn.Sequential(OrderedDict([ ('fc1', nn.Linear(D_in, H)), 
                                    #('Sig', nn.Sigmoid()),
                                    ('ISRU', ISRU()), # Add ISRU
                                    ('fc2', nn.Linear(H, D_out))]))

# Error -----
loss_fn = torch.nn.L1Loss(reduction='mean')

# Train -----
optimizer = torch.optim.Adam(model.parameters(), lr=1,eps=2**(-EPS))
epoch = 250
for t in range(epoch):
    # Forward pass: compute predicted y by passing x to the model.
    clear_output(wait=True)
    y_pred = model(X)

    # Compute and print loss.
    loss = loss_fn(y_pred, Y)
    if t % 100 == 99:
        print(t, loss.item())

    optimizer.zero_grad() ;
    loss.backward() ;
    optimizer.step() ;
if loss.item() < diff : lista = np.vstack((lista, [loss.item(),2,EPS])) ; diff = loss.item()

For starters, have you implemented a [`Dataset`](https://pytorch.org/tutorials/beginner/data_loading_tutorial.html) for your data? — Iguananaut, Jan 07 '20 at 15:40
yes. x = torch.from_numpy(x).float() y = torch.from_numpy(y).float() — , Jan 09 '20 at 06:41

score 2 · Answer 1 · answered Jan 07 '20 at 15:42

Train/validation/test splits of data are "orthogonal" to the model.

To manage your data for training/testing you might want to use pytorch's TensorDataset. Then you might find Subset to be useful for splitting the dataset into train/validation/test subsets.

score 0 · Accepted Answer · edited Jan 09 '20 at 11:47

0

there are many ways to do this. you can use what @Shai suggested, I want to add what I would like to do. I ofen use train_test_split to split my data into train and test and then move forward to convert my train and test data to TensorDataset

if you like an easier solution, you can take a look at skorch it is a scikit learn wrapper for pytorch. I find it easy to use and more like the keras API, skorch will implement train test split automatically when you start training but you can also pass your custom train and test set to it.

edited Jan 09 '20 at 11:47

Shai

111,146
38
238
371

answered Jan 07 '20 at 15:48

basilisk

1,156
1
14
34

Ty. splitting dataset is not a problem, the problem is add validation and test analysis code. How to write the code line for add validation and test analysis in my case? – Jan 08 '20 at 09:09
@rerereL. skorch write the code for you and log it in a table like format to show the progress of valid and training loss/metric in every epoch. otherwise if you are using only pytorch then you can write it yourself, it is not so hard it depends only on what you want to monitor. there are many examples on github – basilisk Jan 08 '20 at 09:44

How to add a Validation and Test Set in Pytorch Model?

2 Answers2