How to pass an existing IDXGIAdapter to D3D11CreateDevice() to create a D3D11Device for a specific adapter?

Question

Goal:

I would like to pass a previously instantiated IDXGIAdapter to ID3D11CreateDevice(...) so that I have the control which adapter is used when creating the D3D11 device.

Setup:

I am using the following sources of inspiration and in-dept analysis:

• CUDA sample for interop with DirectX 11 (code can be seen here)
• DirectX basic construct that allows me to experiment (code can be seen here - click on Main.cpp to reveal the full code)

My system is:

• Windows 10 incl. Windows 8 and 10 SDK
• Visual Studio 2017 Pro incl. the MSVC toolkit that comes with it
• CMake 3.12 (part of VS2017)
• CUDA 11.2
• Latest Nvidia drivers for RTX 3080 (notebook)

Problem:

Whenever I try to pass anything but NULL as IDXGIAdapter to ID3D11CreateDevice(IDXGIAdapter *adapter, ...) I get an HRESULT equal to E_INVALIDARG.

Details:

My goal is to be able to use CUDA DX11 interop to process D3D11 buffers using CUDA (incl. libtorch, that is PyTorch in C++).

Many PCs contain at least one DX-capable GPU, however not every such GPU is CUDA-capable (e.g. integrated GPU or AMD/Intel dedicated GPU). Multiple GPUs are also not excluded in my case. Therefore the starting point for me is answering the question:

On a given system which GPU is CUDA capable?

The CUDA sample I have linked above contains two important functions namely findCUDADevice() as well as findDXDevice(char* dev_name). The first one detects if CUDA is at all available, while the second (although not in a perfect way) detects a DX-compatible device that also supports CUDA.

I adopted and modified the second function like this (it's not without bugs...):

bool getAdapter(IDXGIAdapter** adapter, char* devname)
{
    HRESULT hr = S_OK;
    cudaError cuStatus;
    UINT adapter_idx = 0;
    IDXGIFactory *pFactory;

    hr = CreateDXGIFactory(__uuidof(IDXGIFactory), (void **)(&pFactory));

    if (FAILED(hr)) {
        printf("> No DXGI Factory created.\n");
        return false;
    }

    //for (; !adapter; ++adapter_idx) {
    for (; !(*adapter); ++adapter_idx) {
        // Get a candidate DXGI adapter
        IDXGIAdapter *pAdapter = NULL;
        hr = pFactory->EnumAdapters(adapter_idx, &pAdapter);

        if (FAILED(hr)) {
            break;  // no compatible adapters found
        }

        // Query to see if there exists a corresponding compute device
        int cuDevice;
        cuStatus = cudaD3D11GetDevice(&cuDevice, pAdapter);
        printLastCudaError("cudaD3D11GetDevice failed");

        if (cudaSuccess == cuStatus) {
            // If so, mark it as the one against which to create our d3d10 device
            (*adapter) = pAdapter;
            (*adapter)->AddRef();
            break;
        }

        pAdapter->Release();
    }

    printf("> Found %d D3D11 Adapater(s).\n", (int)adapter);

    pFactory->Release();

    if (!adapter) {
        printf("> Found 0 D3D11 Adapater(s) /w Compute capability.\n");
        return false;
    }

    DXGI_ADAPTER_DESC adapterDesc;
    (*adapter)->GetDesc(&adapterDesc);
    wcstombs(devname, adapterDesc.Description, 128);

    printf("> Found 1 D3D11 Adapater(s) /w Compute capability.\n");
    printf("> %s\n", devname);

    return true;
}

The function still needs further refinement (currently it's just taking the first DX11-CUDA capable device and returns it) but the result is enough for me to investigate further.

Using the DirectX 11 basic setup I posted at the beginning I added

char devname_dx[128];
bool device_ok = getAdapter(&adapter, devname_dx);

right after the initialization of the swap chain and replaced the previously used

// Create the D3D11 device and assign a swap chain to it
hr = D3D11CreateDeviceAndSwapChain(
    NULL,     // instruct DX to take default adapter
    D3D_DRIVER_TYPE_HARDWARE,
    NULL,
    NULL,
    NULL,
    NULL,
    D3D11_SDK_VERSION,
    &scd,
    &swapchain,
    &dev,
    NULL,
    &devcon
);

with

// Create just the D3D11 device, do not add any swap chain
hr = D3D11CreateDevice(
    adapter,  // instruct DX to take a previously retrieved adapter
    D3D_DRIVER_TYPE_HARDWARE,
    NULL,
    NULL,
    NULL,
    NULL,
    D3D11_SDK_VERSION,
    &dev,
    NULL,
    &devcon
);

with the intention to

• use the adapter retrieved in getAdapter(...)
• add the swap chain afterwards

DirectX however disagrees with me and whenever I run my code I get an E_INVALIDARG for that call. I nailed it down to the first argument namely the adapter that is passed onto the function. If I set it to NULL, no error occurs.

How do I instruct DX that I want to use a specific adapter whenever I create a new D3D11 device?

Answer 1

As Simon says, an adapter cannot be provided when the D3D_DRIVER_TYPE parameter is set to anything other than D3D_DRIVER_TYPE_UNKNOWN. This is documented in the API reference for D3D11CreateDevice here:

If you set the pAdapter parameter to a non-NULL value, you must also set the DriverType parameter to the D3D_DRIVER_TYPE_UNKNOWN value. If you set the pAdapter parameter to a non-NULL value and the DriverType parameter to the D3D_DRIVER_TYPE_HARDWARE value, D3D11CreateDevice returns an HRESULT of E_INVALIDARG.

In addition, I second his suggestion to enable the debug layer when working with DirectX and D3D. You can do that by changing your D3DCreateDevice call to the following:

// Create just the D3D11 device, do not add any swap chain
hr = D3D11CreateDevice(
    adapter,  // instruct DX to take a previously retrieved adapter
    D3D_DRIVER_TYPE_UNKNOWN,
    NULL,
    D3D11_CREATE_DEVICE_DEBUG,
    NULL,
    0,
    D3D11_SDK_VERSION,
    &dev,
    NULL,
    &devcon
);

I'm fairly certain the debug layer is built into Windows, so nothing should need to be installed to enable it. You can check by looking for d3d11sdklayers.dll in C:\Windows\System32. This is where its installed on my Windows 10 machine (10.0.19043).

I'd also recommend providing a list of feature levels to the function, as this will allow you to identify hardware that does not support the minimum API requirements you need, rather than silently failing when you try to use those features on down-level hardware. It will also allow you to use D3D11.1 and D3D11.2 in the future. The first link Simon provided contains a good example of how to do this (here).