5

Hypothetically, say I wanted to use a compute shader to run Kernel_X using thread dimensions of (8, 1, 1).

I could set it up as:

In Script:

Shader.Dispatch(Kernel_X, 8, 1, 1);

In Shader:

[numthreads(1,1,1)]
void Kernel_X(uint id : SV_DispatchThreadID) { ... }

or I could set it up like this:

In Script:

Shader.Dispatch(Kernel_X, 1, 1, 1);

In Shader:

[numthreads(8,1,1)]
void Kernel_X(uint id : SV_DispatchThreadID) { ... }

I understand that at the end of this code, the dimensions would come out to be (8, 1, 1); however, I was wondering how switching up the numbers actually differed from each other. My guess would be that running Dispatch (Kernel_X, 8, 1, 1), "ran" a kernel of 1x1x1 8 times, while running numthreads(8,1,1) would run an 8x1x1 kernel once.

TakeMeHomeCountryRoads
  • 169
  • 1
  • 2
  • 8

2 Answers2

10

To understand the difference, a bit of hardware knowledge is required:

Internally, a GPU works on so-called wave fronts, which are SIMD-style processing units (Like a group of threads, where each thread can have it's own data, but they all have to execute the exact same instruction at the exact same time, allways). The number of Threads per wave front is hardware dependent, but is usual either 32 (NVidia) or 64 (AMD).

Now, with [numthreads(8,1,1)] you request a shader thread group size of 8 x 1 x 1 = 8 threads, which the hardware is free to distribute among it's wave fronts. So, with 32 threads per wave front, the hardware would schedule one wave front per shader group, with 8 active threads in that wave front (the other 24 threads are "inactive", meaning they do the same work, but are discarding any memory writes). Then, with Dispatch(1, 1, 1), you are dispatching one such shader group, meaning there will be one wave front running on the hardware.

Would you use [numthreads(1,1,1)] instead, only one thread in a wave front could be active. So, by calling Dispatch(8, 1, 1) on that one, the hardware would require to run 8 shader groups (= 8 wave fronts), each one running just with 1/32 active threads, so while you would get the same result, you would waste a lot more computational power.

So, in general, for optimal performance you want to have shader group sizes that are multiples of 32 (or 64), while trying to call Dispatch with as low numbers as reasonable possible.

3Dave
  • 28,657
  • 18
  • 88
  • 151
Bizzarrus
  • 1,194
  • 6
  • 10
2

The Dispatch() call determines the number of thread groups you are invoking. This way you invoke 8 times 1 times 1 = 8 groups.

Shader.Dispatch(Kernel_X, 8, 1, 1);

And in the shader the [numthreads] tag specifies the size of the thread groups. This for example declares 8 times 1 times 1 = 8 threads for every group.

[numthreads(8,1,1)] void Kernel_X(uint id : SV_DispatchThreadID)
{ }

If you want to achieve a total of 8 threads, you can invoke a single group with 8 threads per group, or 8 groups with a single thread per group. The end result is going to be the same, though performance is not. Usually, you may want to have a threadgroup size that is a power of 2, and with nvidia you usually set it at least at 32 while AMD cards are optimized for at least 64 threads per group.

Btw, you usually dispatch way more than 8 threads, as it’s rather pointless to code a compute shader for just 8 threads and your cpu would probably be faster. So, you may want to call:

Shader.Dispatch(Kernel_X, Mathf.CeilToInt((float)wantedThreadNumber/wantedGroupSize), 1, 1);
kefren
  • 1,042
  • 7
  • 12