0

I want to use multiple immutable samplers within one render command in vulkan, but I don't know how to select them within the fragment shader. To understand what I want and what I mean I start first with explaining the working version.

I wrote a render pipeline for multiple textures using vulkan. The first version, which only supported ARGB (VK_FORMAT_R8G8B8A8_UNORM) textures, works fine.

To record the command buffer I did the following:

  1. vkBeginCommandBuffer
  2. vkCmdBeginRenderPass
  3. vkCmdBindPipeline
  4. for each texture
    1. vkCmdBindVertexBuffers
    2. vkCmdBindIndexBuffer
    3. vkCmdBindDescriptorSets (sets two ubos and the image view)
    4. vkCmdDrawIndexed
  5. vkCmdEndRenderPass
  6. vkEndCommandBuffer

Each texture was drawn correctly, because in 4.3. I bound it's image view to the fragment shader which included the texture sampler.

So now to the part I am struggling with:

The usage of ARGB textures is very ineffizient, if they are updated frequently. So I tried to use YCbCr textures like I420 directly within the shader. After figuring out, how to set them up, I noticed, that they have to be used as immutable samplers (VkDescriptorSetLayoutBinding pImmutableSamplers) which could only be bound statically when the descriptor set layout is defined (vkCreateDescriptorSetLayout). That means using one descriptor set layout over and over again, does not work anymore. Correct?

To fix that, I provided multiple descriptor set layouts (vkCreatePipelineLayout) to the graphics pipeline (vkCreateGraphicsPipelines). Now I am able, to have one binding per texture to the pipeline, which means, I can have multiple immutable samplers statically bound, never exchanged. So far so good, but my fragment shader uses only the first sampler (to be exact, the first descriptor set) to render the output. I thought at first, the descriptor set would be taken from the vkCmdBindDescriptorSets, but I was wrong.

Here is my fragment shader:

#extension GL_ARB_separate_shader_objects : enable

layout(binding = 1) uniform sampler2DArray texSampler;

layout(binding = 2) uniform UniformBufferObject {
    float alpha;
    uint layer;
} ubo;

layout(location = 0) in vec2 fragTexCoord;

layout(location = 0) out vec4 outColor;

void main() {
    vec4 color = texture(texSampler, vec3(fragTexCoord, ubo.layer));
    outColor = vec4(color.rgb, color[3] * ubo.alpha);
}

I already figured out, that it is possible to provide a set for the layout of the shader (eg. layout(set=1, binding=1)), but I need to update this set within the rendering task (per texture), like I did with all the other bindings before drawing.

Any suggestions? What is the recommended solution here?

lexos
  • 21
  • 1
  • 3
  • "*The usage of ARGB textures is very ineffizient, if they are updated frequently.*" What in particular is inefficient about them? "*That means using one descriptor set layout over and over again, does not work anymore.*" Um, why not? The sampler is just part of the descriptor set, rather than being an object bound to the set. Unless you need to *change* the sampler, there shouldn't be a problem. – Nicol Bolas Apr 10 '20 at 13:30
  • @NicolBolas thanks for your response. On on hand, because it increases the memory transfer of the texture (an I420 texture is only 3/8 of the ARGB). On the other hand, I am getting the textures from a video decoder, which provides I420 natively. So forcing the decoder to ARGB means, that each frame has to be color space converted, which costs ~8ms per delivered frame. That sums up if you decode more than one video at once. I420 textures with YCbCr completely removes the effort of the color space conversion and also reduces the memory bandwidth, that the shader consumes, when using the texture. – lexos Apr 10 '20 at 14:59
  • @NicolBolas and to the descriptor set. A YCbCr sampler does not work without being used as immutable sampler. This means, I can not update the binding within the recorded command buffer execution. I have to switch to another descriptor set, to use another sampler. I don't know why, but if I switch them, the conversion is lost und the green becomes red. It interprets the texture as RGB instead of YUV. – lexos Apr 10 '20 at 15:02
  • "*This means, I can not update the binding within the recorded command buffer execution.*" You aren't allowed to change descriptor bindings while the descriptor set is being used *regardless* of immutable samplers. Also, it's not clear why you are trying to change *samplers*. Now, remember that "sampler" means a different thing to the Vulkan API than it does to GLSL. – Nicol Bolas Apr 10 '20 at 15:14
  • @NicolBolas ok, not sure if we talk about the same thing. I created for the first example a pipeline layout (vkCreatePipelineLayout), with one discriptor set layout (vkCreateDescriptorSetLayout) which I updated before each draw of a texture (vkCmdDrawIndexed) with a corresponding descriptor set (vkAllocateDescriptorSets) in the command buffer (vkCmdBindDescriptorSets). Each of this descriptor sets had one image info (VkDescriptorImageInfo) bound (VkWriteDescriptorSet) with an update command (vkUpdateDescriptorSets). But this doesn't work for immutable samplers. What would you recommend? – lexos Apr 11 '20 at 14:22

0 Answers0