How do I create and initialise a DXGI_FORMAT_NV12 resource in DX12 (source is AVFrame)

Question

I'm trying to create an NV12 resource as source for a video encoder in DX12. While I intend to eventually populate a resource from GPU, what I'm trying to do now is take an ffmpeg AVFrame I already have (in AV_PIX_FMT_YUV420P format) and create a texture in DXGI_FORMAT_NV12 format using that data.

I understand the NV12 format (https://learn.microsoft.com/en-us/windows/win32/medfound/recommended-8-bit-yuv-formats-for-video-rendering#nv12) has U and V interleaved while the AV_PIX_FMT_YUV420P doesn't.

My main question is what does the D3D12_RESOURCE_DESC look like for an NV12 texture - do I tell it I need more than one array/mip level to make it planar? Or do I just give it a single memory address with both planes layed out as per the NV12 format, and it figures out subresources for me based on the format?
I understand that to read the data I define two SRVs, one for Y mapped to the Red channel and a second for U and V, but it's how I initialise it that's confusing me.

score 2 · Accepted Answer · answered Jan 05 '23 at 07:09

2

Just create the resource as normal, and then when you query the layout description, it will be planar.

D3D12_RESOURCE_DESC desc = {};
desc.Dimension = D3D12_RESOURCE_DIMENSION_TEXTURE2D;
desc.Format = DXGI_FORMAT_NV12;
desc.MipLevels = 1;
desc.DepthOrArraySize = 1;
desc.Width = 1024;
desc.Height = 720;
desc.SampleDesc.Count = 1;

const CD3DX12_HEAP_PROPERTIES defaultHeapProperties(D3D12_HEAP_TYPE_DEFAULT);

ComPtr<ID3D12Resource> res;
HRESULT hr = device->CreateCommittedResource(
    &defaultHeapProperties,
    D3D12_HEAP_FLAG_NONE,
    &desc,
    D3D12_RESOURCE_STATE_COMMON,
    nullptr,
    IID_PPV_ARGS(res.GetAddressOf()));
if (FAILED(hr))
{
   // error
}

D3D12_FEATURE_DATA_FORMAT_INFO formatInfo = { DXGI_FORMAT_NV12, 0 };
if (FAILED(device->CheckFeatureSupport(D3D12_FEATURE_FORMAT_INFO, &formatInfo, sizeof(formatInfo))))
{
    formatInfo = {};
}

D3D12_PLACED_SUBRESOURCE_FOOTPRINT footprint[2] = {};
UINT numRows;
UINT64 rowBytes, totalBytes;
    
device->GetCopyableFootprints(&desc, 0, 2, 0, footprint, &numRows, &rowBytes, &totalBytes);

The formatInfo.PlaneCount is 2, which is why you have to ask for two subresources.

footprint[0].Format is DXGI_FORMAT_R8_TYPELESS with 1024x720 size. The footprint[0].Offset is likely 0.

footprint[1].Format is DXGI_FORMAT_R8G8_TYPELESS with 512x360 size. The footprint[1].Offset is something other than 0.

In Direct3D 12 Video the layouts are very simple to understand. In Direct3D 11 Video, it was all implicitly defined so it was a bit of a mess. That said, DDS files were defined as non-planar data, so you may want to examine how these are handled in DirectXTex.

answered Jan 05 '23 at 07:09

Chuck Walbourn

38,259
2
58
81

afraid I'm struggling to upload my data, would appreciate some help if possible. I'm using a 1280*720 source and creating a texture with those dimensions. My input data has a section of 720 rows that are 1280 wide (Y data), followed by a section of 360 rows which are also 1280 wide (interleaved UV data) – mike Jan 05 '23 at 14:06
I'm using the utility methods `UpdateSubresources` which in turn uses `MemcpySubresource`. (via MiniEngine code I'm trying to extend) The `RowPitch` returned by `GetCopyableFootprints` is **1536**, I don't understand where this comes from, given the input dimensions of 1280x720. When using `UpdateSubResources`, I'm setting the `.pData` of each `D3D12_SUBRESOURCE_DATA` to be the start of each of the two sections in my input data above. Obviously the `RowPitch` not matching my data is going to be a problem, is my approach otherwise OK? – mike Jan 05 '23 at 14:14
ah so I see [here](https://learn.microsoft.com/en-us/windows/win32/api/d3d12/nf-d3d12-id3d12device-getcopyablefootprints) "pRowSizeInBytes should not be confused with row pitch, as examining pLayouts and getting the row pitch from that will give you 256 as it is aligned to D3D12_TEXTURE_DATA_PITCH_ALIGNMENT." still trying to get this data in though – mike Jan 05 '23 at 16:39
Should I be populating the `rowPitch` of `D3D12_SUBRESOURCE_DATA pSrcData` of `UpdateSubresources` with RowPitch in the footprints above (1536), or `pRowSizeInBytes` from `GetCopyableFootprints` (1280), or just set to my input source dimensions (which correspond to the rowSizeInBytes value or the Width in the footprints)? Seems like it should be the latter? – mike Jan 05 '23 at 16:53
... I seem to be able to create the texture now by just populating `D3D12_SUBRESOURCE_DATA pSrcData` with the dimensions of my source data (i.e. use rowPitch = 1280), funnily enough :-D Got myself in a bit of a muddle there, checking the behaviour of `MemCpySubResource` cleared it up – mike Jan 05 '23 at 17:46

How do I create and initialise a DXGI_FORMAT_NV12 resource in DX12 (source is AVFrame)

1 Answers1