pytorch seq2seq encoder forward method

Question

I'm following Pytorch seq2seq tutorial and below is how they define the encoder function.

class EncoderRNN(nn.Module):
    def __init__(self, input_size, hidden_size):
        super(EncoderRNN, self).__init__()
        self.hidden_size = hidden_size

        self.embedding = nn.Embedding(input_size, hidden_size)
        self.gru = nn.GRU(hidden_size, hidden_size)

    def forward(self, input, hidden):
        embedded = self.embedding(input).view(1, 1, -1)
        output = embedded
        output, hidden = self.gru(output, hidden)
        return output, hidden

    def initHidden(self):
        return torch.zeros(1, 1, self.hidden_size, device=device)

However, it seems like forward method is never really being called during the training.

Here is how the encoder forward method is being used in the tutorial:

    for ei in range(input_length):
        encoder_output, encoder_hidden = encoder(input_tensor[ei], encoder_hidden)
        encoder_outputs[ei] = encoder_output[0, 0]

isn't it supposed to be encoder.forward instead of just encoder? Is there some automatic 'forward' mechanism in Pytorch that I am not aware of?

score 2 · Accepted Answer · answered Jun 14 '18 at 04:50

In PyTorch, you write your own class by extending torch.nn.Module and define the forward method to express your desired computational steps that serve as the "paperwork" (e.g. calling hooks) in the model.__call__(...) method (which is what model(x) will call by python special name specifications).

If you are curious you can look at what model(x) does behind the scenes beyond calling model.forward(x) here: https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/module.py#L462

Also, you can see what is the difference between explicitly calling the .foward(x) method and just simply using model(x) here: https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/module.py#L72

pytorch seq2seq encoder forward method

1 Answers1