
A combined automatic differentiation and array library for C++

Automatic differentiation features
Adept enables C++ algorithms to
be automatically
differentiated. For each mathematical statement involving
scalars or arrays of a special "active" type, Adept stores the
corresponding differential statement symbolically on a stack. The
stack may then be used to perform the following computations:
 Full Jacobian matrix. Given the nonlinear
function y=f(x) coded in C or C++, Adept will
compute the matrix H=∂y/x, where the
element at row i and column j of H
is H_{i,j}=∂y_{i}/∂x_{j}. This
matrix will be computed much more rapidly and accurately than if you
simply recompute the function multiple times perturbing each element
of x one by one. The Jacobian matrix is used in
the GaussNewton
and LevenbergMarquardt
minimization algorithms.
 Reversemode differentiation. This is a key component in
optimization problems where a nonlinear function needs to be
minimized but the state vector x is too large for it to make
sense to compute the full Jacobian matrix. Atmospheric data
assimilation is the canonical example in Meteorology. Given a
nonlinear function y=f(x) and a vector of
adjoints ∂J/∂y (where J is the scalar
function to be minimized), Adept will compute the vector of adjoints
∂J/∂x. Formally,
∂J/∂x is given by the matrixvector
product H^{T}∂J/∂y, but it is
computed here without computing the full Jacobian
matrix H. The adjoint may then be used in
a quasiNewton
minimization scheme.
 Forwardmode differentiation. Given the nonlinear
function y=f(x) and a vector of perturbations
δx, Adept will compute the corresponding vector
δy arising from a linearization of the
function f. Formally, δy is given by the
matrixvector product Hδx, but it is computed
here without computing the full Jacobian matrix H. Note that
Adept is optimized for the reverse case, so might not be as fast
(and will certainly not be as economical in memory) in the forward
mode as libraries written especially for that purpose.
For full details of how to use Adept's automatic differentiation
features, please consult Chapter 2 of the
User Guide.
Speed
The way that
expression
templates have been used and several other important
optimizations mean that reversemode differentiation is
significantly faster than other tools that provide equivalent
functionality
(ADOLC, CppAD
and Sacado)
and less memory is used. In fact, Adept is also often only around
1025% slower than an adjoint code you might write by hand, but
immeasurably faster in terms of user time; adjoint coding is very
time consuming and errorprone. For further details of the
comparison with other tools,
read the Adept paper.
Missing features
At present Adept is missing some functionality that you may
require:
 Differentiation is firstorder only: it cannot directly compute
higherorder derivatives such as the Hessian matrix. However, in a
large class of minimization problems the Hessian matrix can be
computed from the Jacobian matrix (see the FAQ in the
User Guide).
 No capability to differentiate expressions involving complex
numbers.
See also
