4

Where can I find well-documented samples of LSTM models using CNTK C# (many-to-many, many-to-one, regression, classification)? Do they exist?

I have been trying to use CNTK in C# for about 4 months now, and it has been very hard to find good documentation in C#. So far, I have found these:

  1. CNTK C# sequence classifier (Github) https://github.com/Microsoft/CNTK/tree/master/Examples/TrainingCSharp/Common

Very hard to follow. Some of the nomenclature can be found at the (excellent) post: http://colah.github.io/posts/2015-08-Understanding-LSTMs/

But some parameters are confusing. What is LSTMDim, which becomes outputShape in function LSTMPComponentWithSelfStabilization? It seems that it is the length of the hidden state and not the output of the entire net. cellDim appears to be the dimension of the memory state. What exactly does CNTKLib.SequenceLast do?

  1. Sine wave predictor https://bhrnjica.net/2017/12/07/cntk-106-tutorial-time-series-prediction-with-lstm-using-c/

This example uses the LSTM Helper from the Github and is a port from the Python version. It uses cellDim = inDim = 5 in button1_Click event. I changed inDim to 15 and 10 to try some new configurations and got many errors in the process.

braaterAfrikaaner
  • 1,072
  • 10
  • 20
douglas125
  • 71
  • 1
  • 4

1 Answers1

4

CNTK C# support is rather poor compared to Python. There will be more and more examples, but they are merely an attempt to translate the same code from Python to C#. Python has built-in, clean and neat tools for creating deep learning stuff (for instance - cntk.layers module has practically anything you need). In C# you have to either write your own computation blocks(or mostly layers) via basic tools like CNTK.Parameter, CNTK.Function etc. or use their helpers. Thing is though, you may achieve different performance/convergence rates, because Python API is a bit more advanced.

Back to the questions:

  • LSTMDim parameter is the output dimension of that very layer
  • cellDim is the dimension of the cell state

You can see that LSTMDim and cellDim are set to be 25. In Python cellDim by default is None and output shape and cell shape may be different.

  • CNTKLib.SequenceLast returns the last element of the sequence
  • You must set int inDim = 10; in the beginning of the LSTMTimeSeries form as well, to get it started.

There is upcoming 2.7 release, but having worked with CNTK for more than a year (both Python and C#) I have decided to stick to the Python API as it has an edge practically over anything in C#

papadoble151
  • 646
  • 8
  • 19