Adept: A fast automatic differentiation library for C++

Adept (Automatic Differentiation using Expression Templates) is a software library that enables algorithms written in C and C++ to be automatically differentiated. It uses an operator overloading approach, so very little code modification is required. Differentiation can be performed in forward mode, reverse mode (to compute the adjoint), or the full Jacobian matrix can be computed. Moreover, the way that expression templates have been used and several other important optimizations mean that reverse-mode differentiation is significantly faster than other libraries that provide equivalent functionality (ADOL-C, CppAD and Sacado) and less memory is used. In fact, Adept is also often only around 10-25% slower than an adjoint code you might write by hand, but immeasurably faster in terms of user time; adjoint coding is very time consuming and error-prone. For further details, read the Adept paper below.

News

  • Adept 1.1 is now on GitHub! (June 2015) The new version has a more permissive license, supports OpenMP parallelization of the calculation of Jacobian matrices, has better Windows support and is accompanied by a new mailing list.

Functionality

Adept provides the following functionality:

  • Full Jacobian matrix. Given the non-linear function y=f(x) coded in C or C++, Adept will compute the matrix H=∂y/x, where the element at row i and column j of H is Hi,j=∂yi/∂xj. This matrix will be computed much more rapidly and accurately than if you simply recompute the function multiple times perturbing each element of x one by one. The Jacobian matrix is used in the Gauss-Newton and Levenberg-Marquardt minimization algorithms.
  • Reverse-mode differentiation. This is a key component in optimization problems where a non-linear function needs to be minimized but the state vector x is too large for it to make sense to compute the full Jacobian matrix. Atmospheric data assimilation is the canonical example in Meteorology. Given a non-linear function y=f(x) and a vector of adjoints ∂J/∂y (where J is the scalar function to be minimized), Adept will compute the vector of adjoints ∂J/∂x. Formally, ∂J/∂x is given by the matrix-vector product HTJ/∂y, but it is computed here without computing the full Jacobian matrix H. The adjoint may then be used in a quasi-Newton minimization scheme.
  • Forward-mode differentiation. Given the non-linear function y=f(x) and a vector of perturbations δx, Adept will compute the corresponding vector δy arising from a linearization of the function f. Formally, δy is given by the matrix-vector product Hδx, but it is computed here without computing the full Jacobian matrix H. Note that Adept is optimized for the reverse case, so might not be as fast (and will certainly not be as economical in memory) in the forward mode as libraries written especially for that purpose.

Note that at present Adept is missing some functionality that you may require:

  • Differentiation is first-order only: it cannot directly compute higher-order derivatives such as the Hessian matrix.
  • Limited support for complex numbers; no support for mathematical functions of complex numbers, and expressions involving operations on complex numbers are not optimized.
  • Your code must only operate on variables individually: they can be stored in C-style arrays or std::vector types, but Adept does not work fully with containers that allow operations on entire arrays such as the std::valarray type.
  • C and C++ only; this library could not be written in Fortran since the language provides no template capability.

Further information

  • Paper: Hogan, R. J., 2014: Fast reverse-mode automatic differentiation using expression templates in C++. ACM Trans. Math. Softw., 40, 26:1-26:16: PDF
  • Documentation: User Guide: PDF
  • Mailing list: Subscribe to the list or visit the archive.
  • GitHub repository: https://github.com/rjhogan/Adept - you can also browse the code here
  • Talk to data assimilation research group, University of Reading, May 2012: Powerpoint
  • Talk at 14th Euro Automatic Differentiation Workshop, Oxford, December 2013: Powerpoint

If you use Adept in a publication, please cite the Hogan (2014) paper. This is a request and not a condition of the license.

Download

  • Version 1.1 (latest): adept-1.1.tar.gz (3 June 2015). The main changes since version 1.0 are:
    • Changed license from GNU General Public License to the more permissive Apache License, Version 2.0. Users are permitted to use instead the ESA Software Community License Type 1, if they prefer.
    • Build system now uses an autotools-generated configure script.
    • Support for additional mathematical functions: asinh, acosh, atanh, expm1, log1p, cbrt, erf, erfc, exp2 and log2.
    • Jacobian calculation can use OpenMP parallelization.
    • Windows support via the capability to compile code to be differentiated without the need to link to an external library.
    • More test programs and a benchmarking tool.
  • Version 1.0: adept-1.0.tar.gz (3 September 2013). The main changes since version 0.9 are:
    • LIFO requirement removed in allocation of active objects in memory: more efficient for codes that don't deallocate objects in the reverse order from their allocation.
    • Interface change: independent variables no longer need to be initialized using set_value; they can now be initialized using the ordinary assignment operator (=) provided that the new_recording function is called immediately after they are initialized (see section 5.2 of the User Guide).
    • Ability to interface to code components that compute their own Jacobian (e.g. those written in Fortran).
    • More test programs, including one that interfaces to the GNU Scientific Library to perform a real minimization.
    • C++ exceptions are thrown when an error occurs.
    • Recording of derivative information can be "paused" so that the same function can be called from within the program both with and without automatic differentiation.
    • Source code can be compiled twice to produce two versions of a function, one with automatic differentiation and one without.
    • A detailed User Guide.
  • Version 0.9: adept-0.9.tar.gz (2 May 2012).

If you use the code and have any comments, queries, requests or bug-fixes then please contact Robin Hogan. You can also post to the mailing list. I also welcome emails to tell me how Adept is being used.

License and copyright

The Adept library is copyright © 2012-2015 University of Reading.

Adept is dual-licensed, which means that you may choose to be bound by either of the following licenses.

  • The Apache License, Version 2.0. This is a permissive, free-software license that will be the choice for almost all users. It permits you to use and modify the library for any purpose, and to redistribute it under the same terms. Modified versions, or derivative works incorporating Adept, may be distributed in source or object form with a few restrictions that are stated in the license, but provided these conditions are met it may be incorporated into proprietary software.
  • The ESA Software Community License (ESCL), Type 1. Recognising that Adept was written largely with funding from the European Space Agency, you may alternatively use the Type-1 ESCL. It is more restrictive than Apache 2.0 in that derivative works can only be released under the same license, and the license is limited to users in ESA's Member States. The latter condition means that it is not an open-source license. There are several versions of the Type-1 ESCL in circulation; you may use any official version sanctioned by ESA.
In addition to choosing only one of the two licenses above, you are permitted to dual-license your own derivative work using the same two licenses, if you prefer.

Acknowledgements

Adept 1.0 was developed by Robin Hogan with funding from European Space Agency contract 40001041528/11/NL/CT.
Some of the modifications to produce version 1.1 were funded by a National Centre for Earth Observation Mission Support grant (Natural Environment Research Council grant NE/H003894/1).

See also

Return to Clouds Group | Department of Meteorology | University of ReadingThis page is maintained by Robin Hogan