0

I want to train a model with mxnet gluon with mixed precision on a dataset with a huge number of classes. However, the number of classes exceeds the limit of float16 and some of the labels and predictions are inf. How do I solve this?

swmeng
  • 1

1 Answers1

0

You'll want to use a mixture of Float16 and Float32 (it's mixed precision training afterall!). Your data, label and certain other parts of your network (e.g. exp and softmax) shouldn't be converted to Float16, otherwise you'll get Inf issues as you're finding. I recommend checking out MXNet AMP (Automatic Mixed Precision). See this tutorial.

Thom Lane
  • 993
  • 9
  • 9