Portability GHC only experimental ekmett@gmail.com None

Description

Mixed-Mode Automatic Differentiation.

Each combinator exported from this module chooses an appropriate AD mode. The following basic operations are supported, modified as appropriate by the suffixes below:

• `grad` computes the gradient (partial derivatives) of a function at a point
• `jacobian` computes the Jacobian matrix of a function at a point
• `diff` computes the derivative of a function at a point
• `du` computes a directional derivative of a function at a point
• `hessian` compute the Hessian matrix (matrix of second partial derivatives) of a function at a point

The suffixes have the following meanings:

• `'` -- also return the answer
• `With` lets the user supply a function to blend the input with the output
• `F` is a version of the base function lifted to return a `Traversable` (or `Functor`) result
• `s` means the function returns all higher derivatives in a list or f-branching `Stream`
• `T` means the result is transposed with respect to the traditional formulation.
• `0` means that the resulting derivative list is padded with 0s at the end.

Synopsis

grad :: (Traversable f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f a -> f aSource

The `grad` function calculates the gradient of a non-scalar-to-scalar function with `Reverse` AD in a single pass.

````>>> ````grad (\[x,y,z] -> x*y+z) [1,2,3]
```[2,1,1]
```

grad' :: (Traversable f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f a -> (a, f a)Source

The `grad'` function calculates the result and gradient of a non-scalar-to-scalar function with `Reverse` AD in a single pass.

````>>> ````grad' (\[x,y,z] -> 4*x*exp y+cos z) [1,2,3]
```(28.566231899122155,[29.5562243957226,29.5562243957226,-0.1411200080598672])
```

gradWith :: (Traversable f, Num a) => (a -> a -> b) -> (forall s. Mode s => f (AD s a) -> AD s a) -> f a -> f bSource

`grad g f` function calculates the gradient of a non-scalar-to-scalar function `f` with reverse-mode AD in a single pass. The gradient is combined element-wise with the argument using the function `g`.

``` `grad` = `gradWith` (_ dx -> dx)
`id` = `gradWith` const
```

gradWith' :: (Traversable f, Num a) => (a -> a -> b) -> (forall s. Mode s => f (AD s a) -> AD s a) -> f a -> (a, f b)Source

`grad' g f` calculates the result and gradient of a non-scalar-to-scalar function `f` with `Reverse` AD in a single pass the gradient is combined element-wise with the argument using the function `g`.

``grad'` == `gradWith'` (_ dx -> dx)`

grads :: (Traversable f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f a -> Cofree f aSource

# Jacobians (Sparse or Reverse)

jacobian :: (Traversable f, Functor g, Num a) => (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> g (f a)Source

Calculate the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs.

If you know the relative number of inputs and outputs, consider `jacobian` or `jacobian`.

jacobian' :: (Traversable f, Functor g, Num a) => (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> g (a, f a)Source

Calculate both the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward- and reverse- mode AD based on the relative, based on the number of inputs

If you know the relative number of inputs and outputs, consider `jacobian'` or `jacobian'`.

jacobianWith :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> g (f b)Source

`jacobianWith g f` calculates the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs.

The resulting Jacobian matrix is then recombined element-wise with the input using `g`.

If you know the relative number of inputs and outputs, consider `jacobianWith` or `jacobianWith`.

jacobianWith' :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> g (a, f b)Source

`jacobianWith' g f` calculates the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between sparse and reverse mode AD based on the number of inputs and outputs.

The resulting Jacobian matrix is then recombined element-wise with the input using `g`.

If you know the relative number of inputs and outputs, consider `jacobianWith'` or `jacobianWith'`.

# Higher Order Jacobian (Sparse-on-Reverse)

jacobians :: (Traversable f, Functor g, Num a) => (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> g (Cofree f a)Source

# Transposed Jacobians (Forward Mode)

jacobianT :: (Traversable f, Functor g, Num a) => (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> f (g a)Source

A fast, simple, transposed Jacobian computed with forward-mode AD.

jacobianWithT :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> f (g b)Source

A fast, simple, transposed Jacobian computed with `Forward` mode `AD` that combines the output with the input.

# Hessian (Sparse-On-Reverse)

hessian :: (Traversable f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f a -> f (f a)Source

Compute the Hessian via the Jacobian of the gradient. gradient is computed in reverse mode and then the Jacobian is computed in sparse (forward) mode.

hessian' :: (Traversable f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f a -> (a, f (a, f a))Source

# Hessian Tensors (Sparse or Sparse-On-Reverse)

hessianF :: (Traversable f, Functor g, Num a) => (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> g (f (f a))Source

Compute the order 3 Hessian tensor on a non-scalar-to-non-scalar function using `Sparse` or 'Sparse'-on-'Reverse'

# Hessian Tensors (Sparse)

hessianF' :: (Traversable f, Functor g, Num a) => (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f a -> g (a, f (a, f a))Source

# Hessian Vector Products (Forward-On-Reverse)

hessianProduct :: (Traversable f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f (a, a) -> f aSource

`hessianProduct f wv` computes the product of the hessian `H` of a non-scalar-to-scalar function `f` at `w = fst \$ wv` with a vector `v = snd \$ wv` using "Pearlmutter's method" from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.6143, which states:

``` H v = (d/dr) grad_w (w + r v) | r = 0
```

Or in other words, we take the directional derivative of the gradient. The gradient is calculated in reverse mode, then the directional derivative is calculated in forward mode.

hessianProduct' :: (Traversable f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f (a, a) -> f (a, a)Source

`hessianProduct' f wv` computes both the gradient of a non-scalar-to-scalar `f` at `w = fst \$ wv` and the product of the hessian `H` at `w` with a vector `v = snd \$ wv` using "Pearlmutter's method". The outputs are returned wrapped in the same functor.

``` H v = (d/dr) grad_w (w + r v) | r = 0
```

Or in other words, we return the gradient and the directional derivative of the gradient. The gradient is calculated in reverse mode, then the directional derivative is calculated in forward mode.

# Derivatives (Forward Mode)

diff :: Num a => (forall s. Mode s => AD s a -> AD s a) -> a -> aSource

The `diff` function calculates the first derivative of a scalar-to-scalar function by forward-mode `AD`

````>>> ````diff sin 0
```1.0
```

diffF :: (Functor f, Num a) => (forall s. Mode s => AD s a -> f (AD s a)) -> a -> f aSource

The `diffF` function calculates the first derivatives of scalar-to-nonscalar function by `Forward` mode `AD`

````>>> ````diffF (\a -> [sin a, cos a]) 0
```[1.0,-0.0]
```

diff' :: Num a => (forall s. Mode s => AD s a -> AD s a) -> a -> (a, a)Source

The `diff'` function calculates the result and first derivative of scalar-to-scalar function by `Forward` mode `AD`

``` `diff'` `sin` == `sin` `&&&` `cos`
`diff'` f = f `&&&` d f
```
````>>> ````diff' sin 0
```(0.0,1.0)
```
````>>> ````diff' exp 0
```(1.0,1.0)
```

diffF' :: (Functor f, Num a) => (forall s. Mode s => AD s a -> f (AD s a)) -> a -> f (a, a)Source

The `diffF'` function calculates the result and first derivatives of a scalar-to-non-scalar function by `Forward` mode `AD`

````>>> ````diffF' (\a -> [sin a, cos a]) 0
```[(0.0,1.0),(1.0,-0.0)]
```

# Derivatives (Tower)

diffs :: Num a => (forall s. Mode s => AD s a -> AD s a) -> a -> [a]Source

diffsF :: (Functor f, Num a) => (forall s. Mode s => AD s a -> f (AD s a)) -> a -> f [a]Source

diffs0 :: Num a => (forall s. Mode s => AD s a -> AD s a) -> a -> [a]Source

diffs0F :: (Functor f, Num a) => (forall s. Mode s => AD s a -> f (AD s a)) -> a -> f [a]Source

# Directional Derivatives (Forward Mode)

du :: (Functor f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f (a, a) -> aSource

Compute the directional derivative of a function given a zipped up `Functor` of the input values and their derivatives

du' :: (Functor f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f (a, a) -> (a, a)Source

Compute the answer and directional derivative of a function given a zipped up `Functor` of the input values and their derivatives

duF :: (Functor f, Functor g, Num a) => (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f (a, a) -> g aSource

Compute a vector of directional derivatives for a function given a zipped up `Functor` of the input values and their derivatives.

duF' :: (Functor f, Functor g, Num a) => (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f (a, a) -> g (a, a)Source

Compute a vector of answers and directional derivatives for a function given a zipped up `Functor` of the input values and their derivatives.

# Directional Derivatives (Tower)

dus :: (Functor f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f [a] -> [a]Source

dus0 :: (Functor f, Num a) => (forall s. Mode s => f (AD s a) -> AD s a) -> f [a] -> [a]Source

dusF :: (Functor f, Functor g, Num a) => (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f [a] -> g [a]Source

dus0F :: (Functor f, Functor g, Num a) => (forall s. Mode s => f (AD s a) -> g (AD s a)) -> f [a] -> g [a]Source

# Taylor Series (Tower)

taylor :: Fractional a => (forall s. Mode s => AD s a -> AD s a) -> a -> a -> [a]Source

taylor0 :: Fractional a => (forall s. Mode s => AD s a -> AD s a) -> a -> a -> [a]Source

# Maclaurin Series (Tower)

maclaurin :: Fractional a => (forall s. Mode s => AD s a -> AD s a) -> a -> [a]Source

maclaurin0 :: Fractional a => (forall s. Mode s => AD s a -> AD s a) -> a -> [a]Source