Modifying RNN CuDNN example code to use CUDNN_DATA_INT8

Question

The RNN example (RNN_example.cu) that is in cudnn_samples_v7 is set up to use CUDNN_DATA_FLOAT. I'd like to modify it to use CUDNN_DATA_INT8. When I globally made this change, compiled and run, I get the following runtime errors:

$ ./RNN_int8 20 2 512 64 0
cuDNN Error: CUDNN_STATUS_NOT_SUPPORTED RNN_example_int8.cu 285
cuDNN Error: CUDNN_STATUS_NOT_SUPPORTED RNN_example_int8.cu 302
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 309
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 310
cuDNN Error: CUDNN_STATUS_NOT_SUPPORTED RNN_example_int8.cu 326
cuDNN Error: CUDNN_STATUS_NOT_SUPPORTED RNN_example_int8.cu 328
CUDA Error: out of memory RNN_example_int8.cu 330
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 373
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 402
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 373
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 402
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 373
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 402
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 373
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 402
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 482
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 516
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 541

For example, the error at line 309 is running this code:

cudnnErrCheck(cudnnSetFilterNdDescriptor(wDesc, CUDNN_DATA_INT8, CUDNN_TENSOR_NCHW, 3, dimW));

I suspect the format type (CUDNN_TENSOR_HCHW) isn't correct for the data type - is that the correct hypothesis? If so, what format is needed here?

I suggest providing a [mcve]. See item 1 [here](https://stackoverflow.com/help/on-topic). Among other requirements, int8 requires proper underlying support in the hardware. You may also just want to search the [developer guide](https://docs.nvidia.com/deeplearning/sdk/cudnn-developer-guide/index.html) for every instance of int8. — Robert Crovella, Aug 27 '18 at 23:59
I'm concluding based on the CUDNN_STATUS_NOT_SUPPORTED that CUDNN_DATA_INT8 is not supported for these RNN model API calls eve though the Nvidia docs do not stipulate any such restriction. Oh if only CuDNN was open source so I could just look at the code and see for sure. — aneccodeal, Aug 29 '18 at 16:14

score 1 · Answer 1 · edited Sep 22 '18 at 16:59

1

CuDNN currently doesn't support INT8 RNN, we suggest you use FP16 input output to achieve better performance than float. If you use Volta be sure to turn on CUDNN_TENSOR_OP_MATH to get hardware acceleration.

All supported config combinations of cuDNN RNN can be found here https://docs.nvidia.com/deeplearning/sdk/cudnn-developer-guide/index.html#features-of-rnn-functions Let us know if you have any other issues!

edited Sep 22 '18 at 16:59

Stephen Kennedy

20,585
22
95
108

answered Sep 22 '18 at 08:47

Y X

11
1

@Y_X: A while back I tried modifying that example (RNN_example.cu) to use CUDNN_DATA_HALF, but run into lots of issues. Later on I found out about CUDNN_TENSOR_OP_MATH and the CUDNN_TENSOR_OP_MATH_ALLOW_CONVERSION and tried those with the original example code, but didn't see any speedup. – aneccodeal Sep 24 '18 at 20:27

Modifying RNN CuDNN example code to use CUDNN_DATA_INT8

1 Answers1