I have implemented hardware decoding on Linux using VAAPI via FFmpeg. Since I have an OpenGL application, I am converting the decoded VAAPI surfaces to OpenGL textures using vaCopySurfaceGLX. This is working fine except that there is a copy (on the GPU) that is made. I was told that I could directly use the VAAPI surface as OpenGL textures using EGL. I have looked at some examples (mainly Kodi source code) but I'm not able to create the EGLImageKHR. The function eglCreateImageKHR returns 0, and when I check for errors, I get a EGL_BAD_ATTRIBUTE error but I don't understand why.
Below is how I'm converting the VAAPI surface.
During initialization, I set up EGL this way:
// currentDisplay comes from call to glXGetCurrentDisplay() and is also used when getting the VADisplay like this: vaGetDisplay(currentDisplay)
EGLint major, minor;
_eglDisplay = eglGetDisplay(currentDisplay);
eglInitialize(_eglDisplay, &major, &minor);
eglBindAPI(EGL_OPENGL_API);
Then later, to create my EGL image, this is what I do:
// _vaapiContext.vaDisplay comes from vaGetDisplay(currentDisplay)
// surface is the VASurfaceID of the surface I want to use in OpenGL
vaDeriveImage(_vaapiContext.vaDisplay, surface, &_vaapiContext.vaImage);
VABufferInfo buf_info;
memset(&buf_info, 0, sizeof(buf_info));
buf_info.mem_type = VA_SURFACE_ATTRIB_MEM_TYPE_DRM_PRIME;
vaAcquireBufferHandle(_vaapiContext.vaDisplay, _vaapiContext.vaImage.buf, &buf_info);
EGLint attribs[] = {
EGL_WIDTH, _vaapiContext.vaImage.width,
EGL_HEIGHT, _vaapiContext.vaImage.height,
EGL_LINUX_DRM_FOURCC_EXT, fourcc_code('R', '8', ' ', ' '),
EGL_DMA_BUF_PLANE0_FD_EXT, buf_info.handle,
EGL_DMA_BUF_PLANE0_OFFSET_EXT, _vaapiContext.vaImage.offsets[0],
EGL_DMA_BUF_PLANE0_PITCH_EXT, _vaapiContext.vaImage.pitches[0],
EGL_NONE
};
EGLImageKHR eglImage = eglCreateImageKHR(_eglDisplay, EGL_NO_CONTEXT, EGL_LINUX_DMA_BUF_EXT, (EGLClientBuffer)NULL, attribs);
Looking at what could cause this error in the following document https://www.khronos.org/registry/egl/extensions/EXT/EGL_EXT_image_dma_buf_import.txt, I also tried to add the following options which should not matter since my format is not planar
EGL_YUV_COLOR_SPACE_HINT_EXT, EGL_ITU_REC601_EXT,
EGL_SAMPLE_RANGE_HINT_EXT, EGL_YUV_FULL_RANGE_EXT,
EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT, EGL_YUV_CHROMA_SITING_0_EXT,
EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT, EGL_YUV_CHROMA_SITING_0_EXT
The code that I'm using is similar to all the examples I've seen so I'm not sure what the error is.
Note that I have removed all the error checks for this post. All the calls above are successful except for eglCreateImageKHR.