I am trying to incorporate PyTorch functionalities into a scikit-learn
environment (in particular Pipelines and GridSearchCV) and therefore have been looking into skorch
. The standard documentation example for neural networks looks like
import torch.nn.functional as F
from torch import nn
from skorch import NeuralNetClassifier
class MyModule(nn.Module):
def __init__(self, num_units=10, nonlin=F.relu):
super(MyModule, self).__init__()
self.dense0 = nn.Linear(20, num_units)
self.nonlin = nonlin
self.dropout = nn.Dropout(0.5)
...
...
self.output = nn.Linear(10, 2)
...
...
where you explicitly pass the input and output dimensions by hardcoding them into the constructor. However, this is not really how scikit-learn
interfaces work, where the input and output dimensions are derived by the fit
method rather than being explicitly passed to the constructors. As a practical example consider
# copied from the documentation
net = NeuralNetClassifier(
MyModule,
max_epochs=10,
lr=0.1,
# Shuffle training data on each epoch
iterator_train__shuffle=True,
)
# any general Pipeline interface
pipeline = Pipeline([
('transformation', AnyTransformer()),
('net', net)
])
gs = GridSearchCV(net, params, refit=False, cv=3, scoring='accuracy')
gs.fit(X, y)
besides the fact that nowhere in the transformers must one specify the input and output dimensions, the transformers that are applied before the model may change the dimentionality of the training set (think at dimensionality reductions and similar), therefore hardcoding input and output in the neural network constructor just will not do.
Did I misunderstand how this is supposed to work or otherwise what would be a suggested solution (I was thinking of specifying the constructors into the forward
method where you do have X
available for fit already, but I am not sure this is good practice)?