CNTK.GPU Managed C# Eval API: System.Runtime.InteropServices.SEHException when calling outputVal.CopyVariableValueTo

Question

I have trained a model that labels a sequence by grabbing the label of the last word in the sequence:

Sequential([
            Embedding(emb_dim),
            cntk.ops.sequence.last(Recurrence(LSTM(hidden_dim), go_backwards=False)), 
            Dense(num_labels)
        ])

where num_labels = 8

I am using the GPU library from here https://github.com/Microsoft/CNTK/wiki/NuGet-Package to consume the model from C# (CNTK v2.0.beta8.0):

Variable outputVar = modelFunc.Outputs.Single(); 
var outputDataMap = new Dictionary<Variable, Value>();
outputDataMap.Add(outputVar, null);
modelFunc.Evaluate(inputDataMap, outputDataMap, device);
Value outputVal = outputDataMap[outputVar];

And outputVal is: Dimensions: Count = 3 Rank: 3 TotalSize: 8

while outputVar is: Dimensions: Count = 1 Rank: 1 TotalSize: 8

Is the above correct? I would expect that outputVar and outputVal would be of the same Dimensions/Rank/TotalSize. Also, how can I extract the class returned? Basically what C# type should I use for outputData? I tried the two nested lists as in the example on github but without any luck. outputVal.CopyVariableValueTo(outputVar, outputData);

Thank you

score 1 · Accepted Answer · answered Jan 20 '17 at 15:49

1

CopyVariableValueTo copies the data stored in the Value object into the provided buffer, either in the dense (the buffer is List>) or one-hot vector format (the buffer is List>). The one-hot vector format as output requires that each sample has only 1 non-zero value. If the output contains multiple non-zeor values, you should use the dense output. More details about API are here.

Thanks, Zhou

answered Jan 20 '17 at 15:49

Zhou

546
3
3

Changing the type of the list from uint to float solved the problem. Thank you! – malex Jan 20 '17 at 22:47

score 0 · Answer 2 · answered Jan 18 '17 at 09:07

Yes, it is correct. The shape of a Variable is bascially the tensor shape, but the shape of a Value object usually contains 2 additional dimensions: one is the sequence axis and the other is the batch axis, as a Value object could represent a batch of multiple sequences and each sequence has variable length samples. The shape of each sample should be the same as the Variable's shape. In your case, it seems that you have only 1 sample in output, so the sequence and batch length is 1 (you can check the outputVal.Shape.Dimensions1 and [2]), and the TotalSize is the same as outputVar.Shape.TotalSize.

To extract data from Value, you can use

outputVal.CopyVariableValueTo(outputVar, outputBuffer);

CopyVariableTo() copies the data stored in the Value object into the the buffer as a list of sequences with variable length samples, either in the denst format or the one-hot vector format. the outputVar denotes the shape and dynamic axes when copying data from this Value to the outputBuffer. The outputBuffer is a list of sequences with variable length. The number of items contained in the outer list of sequences is the number of sequences in the Value. Each element of the outer list represents a sequence. Each sequence,represented by List, contains a variable number of samples. Each sample consits of a fixed number of elements with type of T. The number of elements of a sample is determined by the shape of sampleVariable. You can find details about API here

What kind of issues do you have when using outputVal.CopyVariableValueTo(outputVar, outputData);?

Thanks

When I run: var outputData = new List>(); Value outputVal = outputDataMap[outputVar]; outputVal.CopyVariableValueTo(outputVar, outputData); I get " An unhandled exception of type 'System.Runtime.InteropServices.SEHException' occurred in CNTKLibraryManaged-2.0.dll" Additional information: External component has thrown an exception. — malex, Jan 18 '17 at 17:52
The exception is thrown when calling : outputVal.CopyVariableValueTo — malex, Jan 18 '17 at 18:00

score 0 · Answer 3 · answered Jan 27 '17 at 21:19

CopyVaraibleTo() has 2 variants, one is output using the dense format, and the other is using the one-hot vector format. For one-hot vecotr format, CopyVariableValueTo requires each sample in output has only 1 non-zero value, otherwise an exception is thrown. The Beta 9 has better exception handling which provides more precise exception info. Could you please have a try?

CNTK.GPU Managed C# Eval API: System.Runtime.InteropServices.SEHException when calling outputVal.CopyVariableValueTo

3 Answers3