Adept: A fast automatic differentiation library for C++
Adept (Automatic Differentiation using Expression Templates) is a
software library that enables algorithms written in C and C++ to
be automatically
differentiated. It uses an operator overloading approach, so
very little code modification is required. Differentiation can be
performed in forward mode, reverse mode (to compute the adjoint),
or the full Jacobian matrix can be computed. Moreover, the way that
expression
templates have been used and several other important
optimizations mean that reversemode differentiation is
significantly faster than other libraries that provide equivalent
functionality
(ADOLC, CppAD
and Sacado)
and less memory is used. In fact, Adept is also often only around
1025% slower than an adjoint code you might write by hand, but
immeasurably faster in terms of user time; adjoint coding is very
time consuming and errorprone. For further details,
read the Adept paper below.
News
 January 2016: Array functionality is being added to Adept
version 2.0, included in the beta
release adept1.9.4.tar.gz.
 June 2015: Adept 1.1 is now
on GitHub! The
new version has a more permissive license, supports OpenMP
parallelization of the calculation of Jacobian matrices, has better
Windows support and is accompanied by a new mailing list.
Functionality
Adept provides the following functionality:
 Full Jacobian matrix. Given the nonlinear
function y=f(x) coded in C or C++, Adept will
compute the matrix H=∂y/x, where the
element at row i and column j of H
is H_{i,j}=∂y_{i}/∂x_{j}. This
matrix will be computed much more rapidly and accurately than if you
simply recompute the function multiple times perturbing each element
of x one by one. The Jacobian matrix is used in
the GaussNewton
and LevenbergMarquardt
minimization algorithms.
 Reversemode differentiation. This is a key component in
optimization problems where a nonlinear function needs to be
minimized but the state vector x is too large for it to make
sense to compute the full Jacobian matrix. Atmospheric data
assimilation is the canonical example in Meteorology. Given a
nonlinear function y=f(x) and a vector of
adjoints ∂J/∂y (where J is the scalar
function to be minimized), Adept will compute the vector of adjoints
∂J/∂x. Formally,
∂J/∂x is given by the matrixvector
product H^{T}∂J/∂y, but it is
computed here without computing the full Jacobian
matrix H. The adjoint may then be used in
a quasiNewton
minimization scheme.
 Forwardmode differentiation. Given the nonlinear
function y=f(x) and a vector of perturbations
δx, Adept will compute the corresponding vector
δy arising from a linearization of the
function f. Formally, δy is given by the
matrixvector product Hδx, but it is computed
here without computing the full Jacobian matrix H. Note that
Adept is optimized for the reverse case, so might not be as fast
(and will certainly not be as economical in memory) in the forward
mode as libraries written especially for that purpose.
Note that at present Adept is missing some functionality that you
may require:
 Differentiation is firstorder only: it cannot directly compute
higherorder derivatives such as the Hessian matrix.
 Limited support for complex numbers; no support for mathematical
functions of complex numbers, and expressions involving operations
on complex numbers are not optimized.
 Up to version 1.1, your code must only operate on variables
individually: they can be stored in Cstyle arrays
or std::vector types, but Adept does not work fully with
containers that allow operations on entire arrays such as
the std::valarray type. However, version 1.9 and higher
includes its own array functionality.
 C and C++ only; this library could not be written in Fortran since
the language provides no template capability.
Further information
 Paper: Hogan, R. J., 2014: Fast reversemode automatic
differentiation using expression templates in C++. ACM
Trans. Math. Softw., 40,
26:126:16: PDF
 Documentation: Version 1.1 User Guide: PDF
 Mailing
list: Subscribe
to the list
or visit
the archive.
 GitHub
repository: https://github.com/rjhogan/Adept
 you can also browse the code here
 Talk to data assimilation research group, University of
Reading, May
2012: Powerpoint
 Talk at 14th Euro Automatic Differentiation Workshop,
Oxford, December 2013: Powerpoint
If you use Adept in a publication, please cite the Hogan (2014)
paper. This is a request and not a condition of the license.
Download
 Version 1.9.4 (beta release for upcoming version
2.0): adept1.9.4.tar.gz (27
January 2016).
 Major rewrite to combine automatic
differentiation and array functionality into a single library,
which is the only way to do fast automatic differentiation of
array expressions. There is still considerable work to do to
include all the necessary functionality.
 Further information available in
the Version 2 User Guide
(PDF).
 Version 1.1
(latest stable version): adept1.1.tar.gz (3
June 2015). The main changes since version 1.0 are:
 Changed license from GNU General Public License to the more
permissive Apache
License, Version 2.0. Users are permitted to use instead the
ESA Software Community License Type 1, if they prefer.
 Build system now uses
an autotoolsgenerated configure
script.
 Support for additional mathematical functions: asinh,
acosh, atanh, expm1, log1p, cbrt,
erf, erfc, exp2 and log2.
 Jacobian calculation can use OpenMP parallelization.
 Windows support via the capability to compile code to be
differentiated without the need to link to an external library.
 More test programs and a benchmarking tool.
 Version 1.0: adept1.0.tar.gz (3
September 2013). The main changes since version 0.9 are:
 LIFO requirement removed in allocation of active objects in
memory: more efficient for codes that don't deallocate objects in
the reverse order from their allocation.
 Interface change: independent variables no longer need to be
initialized using set_value; they can now be initialized
using the ordinary assignment operator (=) provided that
the new_recording function is called immediately after
they are initialized (see section 5.2 of the User Guide).
 Ability to interface to code components that compute their own
Jacobian (e.g. those written in Fortran).
 More test programs, including one that interfaces to
the GNU Scientific
Library to perform a real minimization.
 C++ exceptions are thrown when an error occurs.
 Recording of derivative information can be "paused" so that the
same function can be called from within the program both with and
without automatic differentiation.
 Source code can be compiled twice to produce two versions of a
function, one with automatic differentiation and one without.
 A detailed User Guide.
 Version 0.9: adept0.9.tar.gz
(2 May 2012).
If you use the code and have any comments, queries, requests or
bugfixes then please
contact Robin Hogan. You
can also post to
the mailing
list. I also welcome emails to tell me how Adept is being used.
License and copyright
The Adept library version 1.1 and earlier is copyright ©
20122015 University of Reading. Later versions include code that is
copyright © 20152016 European Centre for Medium Range Weather
Forecasts.
Adept is duallicensed, which means that you may choose to
be bound by either of the following licenses.
 The
Apache License, Version 2.0. This is a permissive,
freesoftware license that will be the choice for almost all users.
It permits you to use and modify the library for any purpose, and to
redistribute it under the same terms. Modified versions, or
derivative works incorporating Adept, may be distributed in source
or object form with a few restrictions that are stated in the
license, but provided these conditions are met it may be
incorporated into proprietary software.
 The ESA Software Community License (ESCL), Type
1. Recognising that Adept was written largely with funding from
the European Space Agency, you may alternatively use the Type1
ESCL. It is more restrictive than Apache 2.0 in that derivative
works can only be released under the same license, and the license
is limited to users in ESA's Member States. The latter condition
means that it is not an opensource license. There are several
versions of the Type1 ESCL in circulation; you may use any official
version sanctioned by ESA.
In addition to choosing only one of the two licenses above, you are
permitted to duallicense your own derivative work using the same two
licenses, if you prefer.
See also
