1

Recently, I have been making program (FDTD Operation) using the CUDA development environment, OS is Windows server 2008 , Graphic card is TeslaC2070, compiler is VS2010. This program calculates using single and double precision floating-point.

I was reading the CUDA programming guide 3.2 and 4.0 . In appendix, guide tell me sin(), cos() has maximum accuracy of 2 ULP. My original CPU program produces results which are different to the CUDA Version.

I want to make results correctly same. Is it possible?

talonmies
  • 70,661
  • 34
  • 192
  • 269
오승택
  • 45
  • 1
  • 5
  • 3
    What makes you believe that `sin` or `cos` are to blame? What is the ULP of the host library you are using? Floating point arithmetic is not associative. Even parallelisation of simple operations like summation can change results. – talonmies Aug 01 '12 at 20:01
  • 1
    @talonmies, why do you post answers in comments? Fine, I'll bite. :-) – harrism Aug 01 '12 at 22:07

1 Answers1

3

To quote Goldberg (a paper that every Computer Scientist, Computational Scientist, and possibly even every scientist who programs, should read):

Due to roundoff errors, the associative laws of algebra do not necessarily hold for floating-point numbers.

This means that when you change the order of operations—even when using ostensibly associative arithmetic—you are likely to get slightly different answers.

Parallelism, by definition, results in different ordering of operations relative to serial arithmetic. "Embarrasingly parallel" computations, that is, computations where each output element is computed independently from all others, sometimes do not have to worry about this. But collective operations, like reductions or scans, and spatial neighborhood computations, such stencils (as in FDTD), do experience this effect.

In practice, even using a different compiler (and even different compiler options) can change the result of floating point computation, even when compiling the same code, with or without parallelism.

harrism
  • 26,505
  • 2
  • 57
  • 88
  • 1
    The following white paper from NVIDIA goes over various reasons why CPU results may differ from GPU results: http://developer.download.nvidia.com/assets/cuda/files/NVIDIA-CUDA-Floating-Point.pdf – njuffa Aug 02 '12 at 03:59