2

My skorch Neural Net GridsearchCV code is running, it just appears to not be on the GPU so it is slow.

Any help appreciated.

It takes 13 seconds per epoch. I know that pytorch runs much faster on the GPU.

Code:

import skorch
import torch
import torch.nn as nn

from skorch.classifier import NeuralNetBinaryClassifier
from sklearn import model_selection
from torch.nn import functional as F

cuda = torch.device('cuda')
parameters = {'lr':[1e-4,1e-5]}

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.lin1 = nn.Linear(50, 30)
        self.lin2 = nn.Linear(30, 1)

    def forward(self, x):
        x = F.tanh(self.lin1(x))
        return self.lin2(x).squeeze()

class NLL(torch.nn.modules.loss._Loss):
    def forward(self, input, target):
        loss = F.nll_loss(input.unsqueeze(1),target)
        return loss   


model = Model()
model.to(cuda)

cl1 = NeuralNetBinaryClassifier(
    module=model,
    criterion=NLL,
    optimizer=torch.optim.Adam,
    max_epochs=100,
    train_split=None,
    device='cuda',
    )  

clf = model_selection.GridSearchCV(cl1, parameters, 
    iid=False, 
    scoring='neg_log_loss',
    cv=ps,
    verbose=1,
    n_jobs=1,
    return_train_score=True,
    )

clf.fit(X, Y)
nemo
  • 55,207
  • 13
  • 135
  • 135
John H
  • 21
  • 3
  • It would be helpful to make your code snippet runnable by including requisite imports, setting all variables, and including proper indents. You can check whether it is actually using the GPU by running nvidia-smi during training. – lebedov Jul 13 '18 at 16:36
  • I could not reproduce this behavior and the code looks good to me. Are you sure pytorch is able to use the GPU on your machine? – nemo Jul 17 '18 at 08:24
  • yes pytorch runs on this GPU about 1000 times faster. skorch actually does run the GPU but the GPU processor is running less than 10% on GPU Temp meter. Maybe something in my config? – John H Jul 18 '18 at 09:43
  • There was an issue that caused significant overhead for smaller datasets when using skorch that was fixed some time ago. If you want you could try running skorch directly from the repository as [explained in the README](https://github.com/dnouri/skorch#from-source) and see if that resolves your issue. – nemo Jul 23 '18 at 12:43

0 Answers0