Modern GPUs support atomic operations in different memory spaces. These are different in implementation and in consequences on execution flow than atomic operations on CPUs.
On modern GPUs, atomic operations in global device memory may require synchronization among thousands of logical threads (or hundreds of warps/wavefronts). A GPU may also support atomic operations on an individual processing core's memory (shared memory in CUDA parlance, local memory in OpenCL parlance) - which behave differently (performance-wise and execution-flow-wise) than global memory atomics.
Reading on GPU atomics:
- The OpenCL reference guide's section on atomic operations.
- Intel's guide to using atomic operations with OpelCL.
- The section on atomic operations in nVIDIA's CUDA Programming guide.
