I want decode h264 dirctively to cv::Mat or cv::cuda::GpuMat without sws_scale to saving CPUs.
So is there any way to make it realized? thanks in advance.
No. If the video is encoded as YUV, it must be converted using something like swscale. There is not “directly” decode to a format other than the one it was encoded as.