ad-3.0: Automatic Differentiation

Portability GHC only experimental ekmett@gmail.com None

Numeric.AD.Mode.Reverse

Description

Mixed-Mode Automatic Differentiation.

For reverse mode AD we use `StableName` to recover sharing information from the tape to avoid combinatorial explosion, and thus run asymptotically faster than it could without such sharing information, but the use of side-effects contained herein is benign.

Synopsis

# Gradient

grad :: (Traversable f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f a -> f aSource

The `grad` function calculates the gradient of a non-scalar-to-scalar function with `Reverse` AD in a single pass.

grad' :: (Traversable f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f a -> (a, f a)Source

The `grad'` function calculates the result and gradient of a non-scalar-to-scalar function with `Reverse` AD in a single pass.

gradWith :: (Traversable f, Num a) => (a -> a -> b) -> (forall s. Mode s => f (AD s a) -> AD s a) -> f a -> f bSource

`grad g f` function calculates the gradient of a non-scalar-to-scalar function `f` with reverse-mode AD in a single pass. The gradient is combined element-wise with the argument using the function `g`.

``` grad == gradWith (\_ dx -> dx)
id == gradWith const
```

gradWith' :: (Traversable f, Num a) => (a -> a -> b) -> (forall s. Mode s => f (AD s a) -> AD s a) -> f a -> (a, f b)Source

`grad' g f` calculates the result and gradient of a non-scalar-to-scalar function `f` with `Reverse` AD in a single pass the gradient is combined element-wise with the argument using the function `g`.

``` grad' == gradWith' (\_ dx -> dx)
```

# Jacobian

jacobian :: (Traversable f, Functor g, Num a) => (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> g (f a)Source

The `jacobian` function calculates the jacobian of a non-scalar-to-non-scalar function with reverse AD lazily in `m` passes for `m` outputs.

jacobian' :: (Traversable f, Functor g, Num a) => (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> g (a, f a)Source

The `jacobian'` function calculates both the result and the Jacobian of a nonscalar-to-nonscalar function, using `m` invocations of reverse AD, where `m` is the output dimensionality. Applying `fmap snd` to the result will recover the result of `jacobian` | An alias for `gradF'`

jacobianWith :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> g (f b)Source

'jacobianWith g f' calculates the Jacobian of a non-scalar-to-non-scalar function `f` with reverse AD lazily in `m` passes for `m` outputs.

Instead of returning the Jacobian matrix, the elements of the matrix are combined with the input using the `g`.

``` jacobian == jacobianWith (\_ dx -> dx)
jacobianWith const == (\f x -> const x <\$> f x)
```

jacobianWith' :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> g (a, f b)Source

`jacobianWith` g f' calculates both the result and the Jacobian of a nonscalar-to-nonscalar function `f`, using `m` invocations of reverse AD, where `m` is the output dimensionality. Applying `fmap snd` to the result will recover the result of `jacobianWith`

Instead of returning the Jacobian matrix, the elements of the matrix are combined with the input using the `g`.

``` jacobian' == jacobianWith' (\_ dx -> dx)
```

# Hessian

hessian :: (Traversable f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f a -> f (f a)Source

Compute the hessian via the jacobian of the gradient. gradient is computed in reverse mode and then the jacobian is computed in reverse mode.

However, since the `'grad f :: f a -> f a'` is square this is not as fast as using the forward-mode Jacobian of a reverse mode gradient provided by `hessian`.

hessianF :: (Traversable f, Functor g, Num a) => (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> g (f (f a))Source

Compute the order 3 Hessian tensor on a non-scalar-to-non-scalar function via the reverse-mode Jacobian of the reverse-mode Jacobian of the function.

Less efficient than `hessianF`.

# Derivatives

diff :: Num a => (forall s. Mode s => AD s a -> AD s a) -> a -> aSource

diff' :: Num a => (forall s. Mode s => AD s a -> AD s a) -> a -> (a, a)Source

The `d'` function calculates the value and derivative, as a pair, of a scalar-to-scalar function.

diffF :: (Functor f, Num a) => (forall s. Mode s => AD s a -> f (AD s a)) -> a -> f aSource

diffF' :: (Functor f, Num a) => (forall s. Mode s => AD s a -> f (AD s a)) -> a -> f (a, a)Source

# Unsafe Variadic Gradient

vgrad :: Grad i o o' a => i -> oSource

vgrad' :: Grad i o o' a => i -> o'Source

class Num a => Grad i o o' a | i -> a o o', o -> a i o', o' -> a i oSource

Instances

 Num a => Grad (AD Reverse a) [a] (a, [a]) a Grad i o o' a => Grad (AD Reverse a -> i) (a -> o) (a -> o') a