5

Context

In the specification it talks about which pipeline stage the loadOp and storeOp of an attachment occur in (Here - The 2nd and 3rd paragraph after the bulletpoints), and which access types they use (Here):

Load operations for attachments with a depth/stencil format execute in the VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT pipeline stage.

Load operations for attachments with a color format execute in the VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT pipeline stage.

VK_ATTACHMENT_LOAD_OP_LOAD specifies that the previous contents of the image within the render area will be preserved. For attachments with a depth/stencil format, this uses the access type VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT. For attachments with a color format, this uses the access type VK_ACCESS_COLOR_ATTACHMENT_READ_BIT.

And so on similarly for the other loadOps and storeOps.

Also it mentions how the loadOp, storeOp, and resolve ops that occur in a subpass are included in the synchronisation scopes of subpass dependencies (Here - The 2nd and 3rd paragraph after the bulletpoints):

the first set of commands includes all commands submitted as part of the subpass instance identified by srcSubpass and any load, store or multisample resolve operations on attachments used in srcSubpass

The Question

The above parts of the specification imply that in some cases this information is needed so you can ensure correct synchronisation of these load and store operations with other accesses (Either in other subpasses or outside that renderpass), in the same way you need to ensure resolve operations are synchronised with other accesses to the same attachment.

My question is whether you need to consider the load and store ops when synchronising subpasses with other subpasses, and subpasses with accesses outside the renderpass instance? And in what cases is synchronisation required?

I believe I must be missunderstanding something as I can't find references to this in any of the vulkan books, or anyone discussing it.

Additional Detail

I recognise that in most cases there is no need to consider this as the pipeline stage and access type of the loadOp or storeOp of the attachment is the same as the pipeline stage and access type used in the commands recorded in the subpass instance that those loadOps or storeOps would occur in. But, there are cases where they aren't the same.

For example:

A renderpass RP that has one subpass RP1.

That subpass uses image view IV as an input attachment with loadOp = VK_ATTACHMENT_LOAD_OP_LOAD.

IV has a colour format and is in VK_IMAGE_LAYOUT_GENERAL.

At the start of RP1 the loadOp will run, in the pipeline stage VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT, with access type VK_ACCESS_COLOR_ATTACHMENT_READ_BIT. (See per the spec. See Context).

But as IV is used as an input attachment the subpass will access it using pipeline stage VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT and access type VK_ACCESS_INPUT_ATTACHMENT_READ_BIT.

Lets say some writes are performed prior to RP on IV, with a external subpass dependency like this:

VkSubpassDependency subpassDependency;

subpassDependency.srcSubpass = VK_SUBPASS_EXTERNAL;
subpassDependency.dstSubpass = 0;

//source
subpassDependency.srcStageMask = /*Whatever the prior write was E.g. VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT*/;
subpassDependency.srcAccessMask = /*Whatever the prior write was E.g. VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT*/;

//destination
subpassDependency.dstStageMask = VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;
subpassDependency.dstAccessMask = VK_ACCESS_INPUT_ATTACHMENT_READ_BIT;

This would, even though it seems right, possibly allow the loadOp of IV to run concurrently with the prior write to IV, clearly being invalid.

Aliasing

Finally, this would also seem to cause complexity due to the need to sync storeOps and loadOps of aliasing attachments, in addition to sync'ing, the accesses to those aliasing attachments themselves.

Thank you in advance.

Community
  • 1
  • 1
AlastairHolmes
  • 427
  • 2
  • 10

1 Answers1

2

My answer to "do you have to synchronize X with Y" regarding Vulkan is yes. There are about 2-3 exceptions, and there is no immediate harm to oversynchronize.

You do need to synchronize with the outside of renderpass. If you have written the resource before, and it is gonna be loaded (read, or write if clear op), there must be a barrier (usually VK_SUBPASS_EXTERNAL dependency). Similarly if you stored a resource (written) and it is gonna be later read.

The spec says:

Applications must ensure that all accesses to memory that backs image subresources used as attachments in a given renderpass instance either happen-before the load operations for those attachments, or happen-after the store operations for those attachments.

Subpasses are gonna likely be synchronized by subsequent operations dependent on the load. E.g. the color attachment gets written also in VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT and you would naturally provide a dependency for subsequent subpasses that would need that output.

As for the perceived subpass self-dependency in your example, the load is guaranteed to happen before the first use. Similarly store happen after last use. Spec quote:

The load operation for each sample in an attachment happens-before any recorded command which accesses the sample in the first subpass where the attachment is used.

The store operation for each sample in an attachment happens-after any recorded command which accesses the sample in the last subpass where the attachment is used.

So, in my interpretation, it is as if you added an imaginary vkCmdLoadResource command to your first subpass that uses it, and a subsequent vkCmdPipelineBarrier dependency between that imaginary load command and a first use of that resource within that subpass.

krOoze
  • 12,301
  • 1
  • 20
  • 34