Any pointers on how to improve Inference for BigBird finetuned on Multiclass Classification? Inference is done on 16GB GPU(NVIDIA).
I have already tried Deepspeed and ONNX. ONNX Runtime is not supported for Bigbird and Deepspeed Zero Stage 3 doesn't any better performance.
