rust gpu_allocator bufferDeviceAddress must be enabbled

Question

I am using ash and gpu_allocator to try to port some C++ code to rust.

I am running into a validation error that the C++ code never runs into:

"Validation Error: [ VUID-VkMemoryAllocateInfo-flags-03331 ] Object 0: handle = 0x56443b02d140, name = Logical device from NVIDIA GeForce GTX 1070, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xf972dfbf | If VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT is set, bufferDeviceAddress must be enabled. The Vulkan spec states: If VkMemoryAllocateFlagsInfo::flags includes VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT, the bufferDeviceAddress feature must be enabled (https://vulkan.lunarg.com/doc/view/1.3.211.0/linux/1.3-extensions/vkspec.html#VUID-VkMemoryAllocateInfo-flags-03331)"', src/vulkan/hardware_interface.rs:709:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

It seems, from the message I need to enable an extension.

Confusion n1, it seems this should always be enabbled, according to the docs: https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_KHR_buffer_device_address.html

Because it was moved to be a core extension? Bbut maybe I miss understand what core means, so I tried enabling it manually.

I Have a list of extension names that I pass to the device upon creation:

const DEVICE_EXTENSIONS : &'static [&str] =
&[
    "VK_KHR_swapchain",
    "VK_KHR_dynamic_rendering",
    "VK_EXT_buffer_device_address",
    "VK_EXT_extended_dynamic_state",
    "VK_EXT_conservative_rasterization",
];

    let extensions : Vec<CString> = DEVICE_EXTENSIONS.iter().map(
        |extension_name| CString::new(extension_name.to_string()).unwrap()
    ).collect();

    let raw_extensions : Vec<*const i8> = extensions.iter().map(
        |extension| extension.as_ptr()
    ).collect();

    let mut dynamic_rendering =
        vk::PhysicalDeviceDynamicRenderingFeaturesKHR {
            dynamic_rendering : true as u32,
            ..Default::default()
        };
    type PDDRF = vk::PhysicalDeviceDynamicRenderingFeaturesKHR;
    let indexing_features =
        vk::PhysicalDeviceDescriptorIndexingFeatures
        {
            p_next : (&mut dynamic_rendering) as *mut PDDRF as *mut c_void,
            shader_sampled_image_array_non_uniform_indexing : true as u32,
            ..Default::default()
        };

    // The enabledLayerCount and ppEnabledLayerNames parameters are deprecated,
    // they should always be 0 and nullptr.
    // https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VkDeviceCreateInfo.html
    let create_info =
        vk::DeviceCreateInfo {
            p_next : &indexing_features as *const _ as *const c_void,
            queue_create_info_count : 1,
            p_queue_create_infos : &queue_create_info,
            enabled_extension_count : requested_device_extensions.len() as u32,
            pp_enabled_extension_names : raw_extensions.as_ptr(),
            p_enabled_features : &device_features,
            ..Default::default()
        };

A little convoluted, I know, but so far the swapchain extension seems to load just fine, so I don't understand why the device address extension is never enabled.

Someone mentioned I should enable the feature alongside the extension:

let buffer_address =
        vk::PhysicalDeviceBufferAddressFeaturesEXT
        {
            buffer_device_address : vk::TRUE,
            ..Default::default()
        };

    let dynamic_rendering =
        vk::PhysicalDeviceDynamicRenderingFeaturesKHR {
            p_next : (&buffer_address) as *const _ as *mut c_void,
            dynamic_rendering : true as u32,
            ..Default::default()
        };

    type PDDRF = vk::PhysicalDeviceDynamicRenderingFeaturesKHR;
    let indexing_features =
        vk::PhysicalDeviceDescriptorIndexingFeatures
        {
            p_next : (&dynamic_rendering) as *const PDDRF as *mut c_void,
            shader_sampled_image_array_non_uniform_indexing : true as u32,
            ..Default::default()
        };

    // The enabledLayerCount and ppEnabledLayerNames parameters are deprecated,
    // they should always be 0 and nullptr.
    // https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VkDeviceCreateInfo.html
    let create_info =
        vk::DeviceCreateInfo
        {
            p_next : &indexing_features as *const _ as *const c_void,
            queue_create_info_count : 1,
            p_queue_create_infos : &queue_create_info,
            enabled_extension_count : requested_device_extensions.len() as u32,
            pp_enabled_extension_names : raw_extensions.as_ptr(),
            p_enabled_features : &device_features,
            ..Default::default()
        };

But I still get the same error even with this.

score 1 · Accepted Answer · answered Aug 13 '22 at 09:38

You already mentioned on the Khronos Vulkan Discord that you got it working with the piece of code I posted using ash's builder pattern and push_next features, but I think I see why your original code triggers the validation layer.

Enabling the feature

You're enabling VK_EXT_buffer_device_address and using its structure VkPhysicalDeviceBufferDeviceAddressFeaturesEXT,
but VK_EXT_buffer_device_address was deprecated by VK_KHR_buffer_device_address; it was promoted to a KHR extension.

Worth nothing here is, that VkPhysicalDeviceBufferDeviceAddressFeaturesKHR is not an alias for VkPhysicalDeviceBufferDeviceAddressFeaturesEXT, it's a different structure (... with identical layout).
They therefore have different sType values:

// Provided by VK_EXT_buffer_device_address
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_BUFFER_DEVICE_ADDRESS_FEATURES_EXT = 1000244000,

// Provided by VK_VERSION_1_2
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_BUFFER_DEVICE_ADDRESS_FEATURES = 1000257000,
// Provided by VK_KHR_buffer_device_address
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_BUFFER_DEVICE_ADDRESS_FEATURES_KHR = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_BUFFER_DEVICE_ADDRESS_FEATURES,

Now let's check the relevant piece of code for the validation layer, where it checks for the VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT flag for vkAllocateMemory:

VkBool32 buffer_device_address = false;
// --snip--
const auto *bda_features = LvlFindInChain<VkPhysicalDeviceBufferDeviceAddressFeatures>(device_createinfo_pnext);
if (bda_features) {
  capture_replay = bda_features->bufferDeviceAddressCaptureReplay;
  buffer_device_address = bda_features->bufferDeviceAddress;
}

It tries to find a VkPhysicalDeviceBufferDeviceAddressFeatures in the pNext chain, which is an alias for VkPhysicalDeviceBufferDeviceAddressFeaturesKHR, not the EXT one, as can be seen from the source code for LvlFindInChain:

// Find an entry of the given type in the const pNext chain
template <typename T> const T *LvlFindInChain(const void *next) {
// --snip--
    if (LvlTypeMap<T>::kSType == current->sType) {

// Map type VkPhysicalDeviceBufferDeviceAddressFeatures to id VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_BUFFER_DEVICE_ADDRESS_FEATURES
template <> struct LvlTypeMap<VkPhysicalDeviceBufferDeviceAddressFeatures> {
    static const VkStructureType kSType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_BUFFER_DEVICE_ADDRESS_FEATURES;
};

That is, the validation layer looks for a structure with
sType == VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_BUFFER_DEVICE_ADDRESS_FEATURES == 1000257000,
not one with VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_BUFFER_DEVICE_ADDRESS_FEATURES_EXT == 1000244000,
therefore it concludes that bufferDeviceAddress feature is not enabled.

You needed to use the promoted KHR one to satisfy the validation layer, even though your Vulkan driver probably supported the deprecated EXT one as well.

gpu-allocator

As for why using the gpu-allocator crate seemingly necessitated enabling buffer device address,
I'd suggest checking if you accidentally set gpu_allocator::vulkan::AllocatorCreatorDesc::buffer_device_address to true.
This is what prompts the library to use VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT when calling vkAllocateMemory:

let allocation_flags = vk::MemoryAllocateFlags::DEVICE_ADDRESS;
let mut flags_info = vk::MemoryAllocateFlagsInfo::builder().flags(allocation_flags);
// TODO(manon): Test this based on if the device has this feature enabled or not
let alloc_info = if buffer_device_address {
  alloc_info.push_next(&mut flags_info)

rust gpu_allocator bufferDeviceAddress must be enabbled

1 Answers1

Enabling the feature

gpu-allocator