Also known as algorithmic differentiation, short AD. Techniques that take a procedure evaluating a numerical function and transform it into a procedure that additionally evaluates directional derivatives, gradients, higher order derivatives.
Also known as algorithmic differentiation, short AD. Techniques that take a procedure evaluating a numerical function and transform it into a procedure that additionally evaluates directional derivatives, gradients, higher order derivatives.
Techniques include operator
- overloading for dual numbers,
- operator overloading to extract the operations sequence as a tape,
- code analysis and transformation.
For a function with input of dimension n and output of dimension n, requiring L elementary operations for its evaluation, one directional derivative or one gradient can be computed with 3*L operations.
The accuracy of the derivative is, automatically, nearly as good as the accuracy of the function evaluation.
Other differentiation method are
- symbolic differentiation, where the expanded expression for the derivatives is obtained first, which can be large depending on the implementation, and
- numerical differentiation by divided differences, which provides less accuracy with comparable effort, or comparable accuracy with a higher effort.
See wikipedia and autodiff.org
