Before I know the automatic mixed precision, I manually half the model and data using half() for training with half precision. But the training result is not good at all.
Then I used the automatic mixed precision to train a network, which returns desent results. But when I save the checkpoint, the parameters in the checkpoints are still in fp32. I want to save a checkpoint with fp16. Therefore, I want to ask if and how I can save the checkpoints with fp16. And this also makes me wonder: when performing conv2d with autocast, does the parameters of conv2d also halfed? or is it only the data halfed?