0

how to use NStepLSTM or BiNStepLSTM? I see reference it must supply argument as list of variables, what is this list?

How to use it to composite of other layer Link??

machen
  • 283
  • 2
  • 10

1 Answers1

1

NStepLSTM assumes that the inputs are a minibatch of sequences that may have different lengths. The input is a list of these sequences. Each sequence is represented by a variable of shape (T, D), where T is the length of the sequence and D is the dimensionality of each item in the sequence (if you are dealing with text data, D can be the dimensionality of the embedding layer).

Then, the NStepLSTM.__call__ returns a tuple of three: final hidden states, final cell states, and the output sequences in a similar format as the input sequences (a list of variables). You can combine it with other functions or links. For example, you can pass each variable in the output sequences to some loss function to get a loss.

Seiya Tokui
  • 341
  • 2
  • 3
  • Do you mean the input is list of minibatch? suppose I have an article of different lengths of sentence. each sentence is variable shape (T,D). D is word embedding dimension, so the input is whole article? that would be memory expensive! Because this will exhaust all GPU memory if we input all sequence to GPU? – machen Aug 24 '17 at 01:54
  • Do you mean the return 3rd value is also return all time step output and all sequence in each time step will return? – machen Aug 24 '17 at 01:55
  • If the whole sequence is too long to fit into the memory, you have to split the sentence into several pieces and do truncated BPTT (as the official language modeling example (ptb) of Chainer does). The third element of the returned tuple has the same length as the input to `NStepLSTM`. – Seiya Tokui Aug 28 '17 at 00:52
  • Does the input to the NStepLSTM list must have time order? I know each Variable is T x D must have time order, Does it means list of T x D , this list will strict to same order : the first element is first time happens event, the second continues the list's first element event ? – machen Nov 05 '17 at 15:33
  • does input of NStepLSTM that list of variable: between each variable inside this list need to have temporal correlation??? – machen Mar 01 '18 at 08:16
  • If I want to only use last time step to calculate loss, because NStepLSTM return 3 variable(final hidden states, final cell states, and the output sequences ) , should I use the the last one or the first one? – machen Mar 04 '18 at 09:48