0

In the online racecheck documentation, the severity level has this description of hazard level WARNING:
An example of this are hazards due to warp level programming that make the assumption that threads are proceeding in groups.

The statement is confusing because threads are processed in groups. (The SM executes code across a warp.) If they are not processed in groups, then how are they processed?
What does "warp level programming" mean? (What would non warp level programming be?)

einpoklum
  • 118,144
  • 57
  • 340
  • 684
Doug
  • 2,783
  • 6
  • 33
  • 37

1 Answers1

1

Its true that all processing is handled in warps. Warp level programming also called warp synchronous programming depends on this to ensure correctness of code/behavior. Many or perhaps most codes do not depend on the concept of a warp or that there are 32 threads per warp to deliver correct behavior.

There are at least two concerns. First in the presence of control structures such as if/then/else its possible that threads in a warp are not all executing in lockstep. Second there is no guarantee that future architectures will preserve the concept of a warp or 32 threads per warp

Robert Crovella
  • 143,785
  • 11
  • 213
  • 257
  • I get the feeling that WLP is an instance where the # of threads that are executing is less than WarpSize. i.e. blockDim – Doug Sep 25 '13 at 18:32
  • No. Warp synchronous programming makes the assumption that a group of threads are executing in lockstep, and takes advantage of that knowledge in some way (for example to ensure the sequence of operations between threads in the same warp.) This assumption can be made and used successfully (if appropriate care is taken) whether only 2 threads are executing or all 32 threads are executing in a warp, and independent of the number of warps or block dimensions. But if, for example, a future architecture declared a new warp size of 16 instead of 32, such an assumption might be unreliable. – Robert Crovella Sep 25 '13 at 18:44
  • By lockstep you mean every thread in the block takes the exact same branches? i.e. The IF/ELSE in every thread evaluates the same, and takes the same branch. Not, one thread takes the THEN branch and the next 4 take the ELSE branch. – Doug Sep 25 '13 at 19:57
  • Yes every thread in a warp that is following the same code path is executing in lockstep. This behavior can be exploited in the code. – Robert Crovella Sep 25 '13 at 21:41
  • 2
    Note that incorrectly synchronized warp-synchronous programs could also fail to work simply due to changes in compiler optimizations, no new hardware required. – harrism Sep 26 '13 at 04:36