Numeric.Backprop

Provides the BP monad and the BVar type; after manipulating BVars (inputs to your function) to produce a result, the library tracks internal data dependencies, which are used to perform back-propagation (reverse-mode automatic differentiation) to calculate the gradient of the output with respect to the inputs.

Similar to automatic differentiation from the ad library and Numeric.AD.Mode.Reverse, except for a few key differences:

1. Most importantly, this library implements heterogeneous back-propagation, so you can manipulate values of different types (like different matrix and vector types, and product and sum types). This is essential for things like back-propagation for neural networks.
2. This module allows you to explicitly build your data dependency graph if you wish, which allows the library to perform optimizations and reduce extra allocation, which may or may not provide advantages over Numeric.AD.Mode.Reverse's unsafePerformIO-based implicit graph building.

See the README for more information and links to demonstrations and tutorials. If you want to plunge right in, you can also look directly at the main types, BP, BPOp, BVar, Op, and the main functions, backprop and opVar.

Types

Backprop types

data BP s rs a

A Monad allowing you to explicitly build hetereogeneous data dependency graphs and that the library can perform back-propagation on.

A BP s rs a is a BP action that uses an environment of rs returning a a. When "run", it will compute a gradient that is a tuple of rs. (The phantom parameter s is used to ensure that any BVars aren't leaked out of the monad)

Note that you can only "run" a BP s rs that produces a BVar -- that is, things of the form

BP s rs (BVar s rs a)

The above is a BP action that returns a BVar containing an a. When this is run, it'll produce a result of type a and a gradient of that is a tuple of rs. (This form has a type synonym, BPOp, for convenience)

For example, a BP s '[ Int, Double, Double ] is a monad that represents a computation with an Int, Double, and Double as inputs. And, if you ran a

BP s '[ Int, Double, Double ] (BVar s '[ Int, Double, Double ] Double)

Or, using the BPOp type synonym:

BPOp s '[ Int, Double, Double ] Double

with backprop or gradBPOp, it'll return a gradient on the inputs (Int, Double, and Double) and produce a value of type Double.

Now, one powerful thing about this type is that a BP is itself an Op (or more precisely, an OpB, which is a subtype of OpM). So, once you create your fancy BP computation, you can transform it into an OpM using bpOp.

Instances

Monad (BP s rs)

Methods

(>>=) :: BP s rs a -> (a -> BP s rs b) -> BP s rs b

(>>) :: BP s rs a -> BP s rs b -> BP s rs b

return :: a -> BP s rs a

fail :: String -> BP s rs a

Functor (BP s rs)

Methods

fmap :: (a -> b) -> BP s rs a -> BP s rs b

(<$) :: a -> BP s rs b -> BP s rs a

Applicative (BP s rs)

Methods

pure :: a -> BP s rs a

(<*>) :: BP s rs (a -> b) -> BP s rs a -> BP s rs b

(*>) :: BP s rs a -> BP s rs b -> BP s rs b

(<*) :: BP s rs a -> BP s rs b -> BP s rs a

type BPOp s rs a = BP s rs (BVar s rs a)

A handy type synonym representing a BP action that returns a BVar. This is handy because this is the form of BP actions that backprop and gradBPOp (etc.) expects.

A value of type:

BPOp s rs a

is an action that takes an input environment of rs and produces a BVar containing a value of type a. Because it returns a BVar, the library can track the data dependencies between the BVar and the input environment and perform back-propagation.

See documentation for BP for an explanation of the phantom type parameter s.

type BPOpI s rs a = Prod (BVar s rs) rs -> BVar s rs a

An "implicit" operation on BVars that can be backpropagated. A value of type:

BPOpI s rs a

takes a bunch of BVars containg rs and uses them to (purely) produce a BVar containing an a.

foo :: BPOpI s '[ Double, Double ] Double
foo (x :< y :< Ø) = x + sqrt y

If you are exclusively doing implicit back-propagation by combining BVars and using BPOpIs, you are probably better off just importing Numeric.Backprop.Implicit, which provides better tools. This type synonym exists in Numeric.Backprop just for the implicitly function, which can convert "implicit" backprop functions like a BPOpI s rs a into an "explicit" graph backprop function, a BPOp s rs a.

data BVar :: Type -> [Type] -> Type -> Type

The basic unit of manipulation inside BP (or inside an implicit-graph backprop function). Instead of directly working with values, you work with BVars contating those values. When you work with a BVar, the backprop library can keep track of what values refer to which other values, and so can perform back-propagation to compute gradients.

A BVar s rs a refers to a value of type a, with an environment of values of the types rs. The phantom parameter s is used to ensure that stray BVars don't leak outside of the backprop process.

(That is, if you're using implicit backprop, it ensures that you interact with BVars in a polymorphic way. And, if you're using explicit backprop, it ensures that a BVar s rs a never leaves the BP s rs that it was created in.)

BVars have Num, Fractional, Floating, etc. instances, so they can be manipulated using polymorphic functions and numeric functions in Haskell. You can add them, subtract them, etc., in "implicit" backprop style.

(However, note that if you directly manipulate BVars using those instances or using liftB, it delays evaluation, so every usage site has to re-compute the result/create a new node. If you want to re-use a BVar you created using + or - or liftB, use bindVar to force it first. See documentation for bindVar for more details.)

Instances

Floating a => Floating (BVar s rs a)

See note for Num instance.

Methods

pi :: BVar s rs a

exp :: BVar s rs a -> BVar s rs a

log :: BVar s rs a -> BVar s rs a

sqrt :: BVar s rs a -> BVar s rs a

(**) :: BVar s rs a -> BVar s rs a -> BVar s rs a

logBase :: BVar s rs a -> BVar s rs a -> BVar s rs a

sin :: BVar s rs a -> BVar s rs a

cos :: BVar s rs a -> BVar s rs a

tan :: BVar s rs a -> BVar s rs a

asin :: BVar s rs a -> BVar s rs a

acos :: BVar s rs a -> BVar s rs a

atan :: BVar s rs a -> BVar s rs a

sinh :: BVar s rs a -> BVar s rs a

cosh :: BVar s rs a -> BVar s rs a

tanh :: BVar s rs a -> BVar s rs a

asinh :: BVar s rs a -> BVar s rs a

acosh :: BVar s rs a -> BVar s rs a

atanh :: BVar s rs a -> BVar s rs a

log1p :: BVar s rs a -> BVar s rs a

expm1 :: BVar s rs a -> BVar s rs a

log1pexp :: BVar s rs a -> BVar s rs a

log1mexp :: BVar s rs a -> BVar s rs a

Fractional a => Fractional (BVar s rs a)

See note for Num instance.

Methods

(/) :: BVar s rs a -> BVar s rs a -> BVar s rs a

recip :: BVar s rs a -> BVar s rs a

fromRational :: Rational -> BVar s rs a

Num a => Num (BVar s rs a)

Note that if you use the Num instance to create BVars, the resulting BVar is deferred/delayed. At every location you use it, it will be recomputed, and a separate graph node will be created. If you are using a BVar you made with the Num instance in multiple locations, use bindVar first to force it and prevent recomputation.

Methods

(+) :: BVar s rs a -> BVar s rs a -> BVar s rs a

(-) :: BVar s rs a -> BVar s rs a -> BVar s rs a

(*) :: BVar s rs a -> BVar s rs a -> BVar s rs a

negate :: BVar s rs a -> BVar s rs a

abs :: BVar s rs a -> BVar s rs a

signum :: BVar s rs a -> BVar s rs a

fromInteger :: Integer -> BVar s rs a

type Op as a = forall m. Monad m => OpM m as a

An Op as a describes a differentiable function from as to a.

For example, a value of type

Op '[Int, Bool] Double

is a function from an Int and a Bool, returning a Double. It can be differentiated to give a gradient of an Int and a Bool if given a total derivative for the Double. If we call Bool \(2\), then, mathematically, it is akin to a:

\[ f : \mathbb{Z} \times 2 \rightarrow \mathbb{R} \]

See runOp, gradOp, and gradOpWith for examples on how to run it, and Op for instructions on creating it.

This type is abstracted over using the pattern synonym with constructor Op, so you can create one from scratch with it. However, it's simplest to create it using op2', op1', op2', and op3' helper smart constructors And, if your function is a numeric function, they can even be created automatically using op1, op2, op3, and opN with a little help from Numeric.AD from the ad library.

Note that this type is a subset or subtype of OpM (and also of OpB). So, if a function ever expects an OpM m as a (or a OpB), you can always provide an Op as a instead.

Many functions in this library will expect an OpM m as a (or an OpB s as a), and in all of these cases, you can provide an Op as a.

type OpB s as a = OpM (ST s) as a

A subclass of OpM (and superclass of Op), representing Ops that the backprop library uses to perform backpropation.

An

OpB s rs a

represents a differentiable function that takes a tuple of rs and produces an a a, which can be run on BVar ss and also inside BP ss. For example, an OpB s '[ Int, Double ] Bool takes an Int and a Double and produces a Bool, and does it in a differentiable way.

OpB is a superset of Op, so, if you see any function that expects an OpB (like opVar' and ~$, for example), you can give them an Op, as well.

You can think of OpB as a superclass/parent class of Op in this sense, and of Op as a subclass of OpB.

Tuple types

Prod is a heterogeneous list/tuple type, which allows you to tuple together multiple values of different types and operate on them generically.

A Prod f '[a, b, c] contains an f a, an f b, and an f c, and is constructed by consing them together with :< (using 'Ø' as nil):

I "hello" :< I True :< I 7.8 :< Ø    :: Prod I '[String, Bool, Double]
C "hello" :< C "world" :< C "ok" :< Ø  :: Prod (C String) '[a, b, c]
Proxy :< Proxy :< Proxy :< Ø           :: Prod Proxy '[a, b, c]

(I is the identity functor, and C is the constant functor)

So, in general:

x :: f a
y :: f b
z :: f c
x :< y :< z :< Ø :: Prod f '[a, b, c]

If you're having problems typing 'Ø', you can use only:

only z           :: Prod f '[c]
x :< y :< only z :: Prod f '[a, b, c]

Tuple is provided as a convenient type synonym for Prod I, and has a convenient pattern synonym ::< (and only_), which can also be used for pattern matching:

x :: a
y :: b
z :: c

only_ z             :: Tuple '[c]
x ::< y ::< z ::< Ø :: Tuple '[a, b, c]
x ::< y ::< only_ z :: Tuple '[a, b, c]

data Prod k f a :: forall k. (k -> *) -> [k] -> * where

Constructors

Ø :: Prod k f ([] k)

(:<) :: Prod k f ((:) k a1 as) infixr 5

Instances

Witness ØC ØC (Prod k f (Ø k))

Associated Types

type WitnessC (ØC :: Constraint) (ØC :: Constraint) (Prod k f (Ø k)) :: Constraint

Methods

(\\) :: ØC => (ØC -> r) -> Prod k f (Ø k) -> r

IxFunctor1 k [k] (Index k) (Prod k)

Methods

imap1 :: (forall a. i b a -> f a -> g a) -> t f b -> t g b

IxFoldable1 k [k] (Index k) (Prod k)

Methods

ifoldMap1 :: Monoid m => (forall a. i b a -> f a -> m) -> t f b -> m

IxTraversable1 k [k] (Index k) (Prod k)

Methods

itraverse1 :: Applicative h => (forall a. i b a -> f a -> h (g a)) -> t f b -> h (t g b)

Functor1 [k] k (Prod k)

Methods

map1 :: (forall a. f a -> g a) -> t f b -> t g b

Foldable1 [k] k (Prod k)

Methods

foldMap1 :: Monoid m => (forall a. f a -> m) -> t f b -> m

Traversable1 [k] k (Prod k)

Methods

traverse1 :: Applicative h => (forall a. f a -> h (g a)) -> t f b -> h (t g b)

TestEquality k f => TestEquality [k] (Prod k f)

Methods

testEquality :: f a -> f b -> Maybe ((Prod k f :~: a) b)

BoolEquality k f => BoolEquality [k] (Prod k f)

Methods

boolEquality :: f a -> f b -> Boolean ((Prod k f == a) b)

Eq1 k f => Eq1 [k] (Prod k f)

Methods

eq1 :: f a -> f a -> Bool

neq1 :: f a -> f a -> Bool

Ord1 k f => Ord1 [k] (Prod k f)

Methods

compare1 :: f a -> f a -> Ordering

(<#) :: f a -> f a -> Bool

(>#) :: f a -> f a -> Bool

(<=#) :: f a -> f a -> Bool

(>=#) :: f a -> f a -> Bool

Show1 k f => Show1 [k] (Prod k f)

Methods

showsPrec1 :: Int -> f a -> ShowS

show1 :: f a -> String

Read1 k f => Read1 [k] (Prod k f)

Methods

readsPrec1 :: Int -> ReadS (Some (Prod k f) f)

(Known [k] (Length k) as, Every k (Known k f) as) => Known [k] (Prod k f) as

Associated Types

type KnownC (Prod k f) (as :: Prod k f -> *) (a :: Prod k f) :: Constraint

Methods

known :: as a

(Witness p q (f a1), Witness s t (Prod a f as)) => Witness (p, s) (q, t) (Prod a f ((:<) a a1 as))

Associated Types

type WitnessC ((p, s) :: Constraint) ((q, t) :: Constraint) (Prod a f ((:<) a a1 as)) :: Constraint

Methods

(\\) :: (p, s) => ((q, t) -> r) -> Prod a f ((a :< a1) as) -> r

ListC ((<$>) Constraint * Eq ((<$>) * k f as)) => Eq (Prod k f as)

Methods

(==) :: Prod k f as -> Prod k f as -> Bool

(/=) :: Prod k f as -> Prod k f as -> Bool

(ListC ((<$>) Constraint * Eq ((<$>) * k f as)), ListC ((<$>) Constraint * Ord ((<$>) * k f as))) => Ord (Prod k f as)

Methods

compare :: Prod k f as -> Prod k f as -> Ordering

(<) :: Prod k f as -> Prod k f as -> Bool

(<=) :: Prod k f as -> Prod k f as -> Bool

(>) :: Prod k f as -> Prod k f as -> Bool

(>=) :: Prod k f as -> Prod k f as -> Bool

max :: Prod k f as -> Prod k f as -> Prod k f as

min :: Prod k f as -> Prod k f as -> Prod k f as

ListC ((<$>) Constraint * Show ((<$>) * k f as)) => Show (Prod k f as)

Methods

showsPrec :: Int -> Prod k f as -> ShowS

show :: Prod k f as -> String

showList :: [Prod k f as] -> ShowS

type WitnessC ØC ØC (Prod k f (Ø k))

type WitnessC ØC ØC (Prod k f (Ø k)) = ØC

type KnownC [k] (Prod k f) as

type KnownC [k] (Prod k f) as = (Known [k] (Length k) as, Every k (Known k f) as)

type WitnessC (p, s) (q, t) (Prod a f ((:<) a a1 as))

type WitnessC (p, s) (q, t) (Prod a f ((:<) a a1 as)) = (Witness p q (f a1), Witness s t (Prod a f as))

type Tuple = Prod * I

A Prod of simple Haskell types.

newtype I a :: * -> *

Constructors

I

Fields

• getI :: a

Instances

Monad I

Methods

(>>=) :: I a -> (a -> I b) -> I b

(>>) :: I a -> I b -> I b

return :: a -> I a

fail :: String -> I a

Functor I

Methods

fmap :: (a -> b) -> I a -> I b

(<$) :: a -> I b -> I a

Applicative I

Methods

pure :: a -> I a

(<*>) :: I (a -> b) -> I a -> I b

(*>) :: I a -> I b -> I b

(<*) :: I a -> I b -> I a

Foldable I

Methods

fold :: Monoid m => I m -> m

foldMap :: Monoid m => (a -> m) -> I a -> m

foldr :: (a -> b -> b) -> b -> I a -> b

foldr' :: (a -> b -> b) -> b -> I a -> b

foldl :: (b -> a -> b) -> b -> I a -> b

foldl' :: (b -> a -> b) -> b -> I a -> b

foldr1 :: (a -> a -> a) -> I a -> a

foldl1 :: (a -> a -> a) -> I a -> a

toList :: I a -> [a]

null :: I a -> Bool

length :: I a -> Int

elem :: Eq a => a -> I a -> Bool

maximum :: Ord a => I a -> a

minimum :: Ord a => I a -> a

sum :: Num a => I a -> a

product :: Num a => I a -> a

Traversable I

Methods

traverse :: Applicative f => (a -> f b) -> I a -> f (I b)

sequenceA :: Applicative f => I (f a) -> f (I a)

mapM :: Monad m => (a -> m b) -> I a -> m (I b)

sequence :: Monad m => I (m a) -> m (I a)

Witness p q a => Witness p q (I a)

Associated Types

type WitnessC (p :: Constraint) (q :: Constraint) (I a) :: Constraint

Methods

(\\) :: p => (q -> r) -> I a -> r

Eq a => Eq (I a)

Methods

(==) :: I a -> I a -> Bool

(/=) :: I a -> I a -> Bool

Num a => Num (I a)

Methods

(+) :: I a -> I a -> I a

(-) :: I a -> I a -> I a

(*) :: I a -> I a -> I a

negate :: I a -> I a

abs :: I a -> I a

signum :: I a -> I a

fromInteger :: Integer -> I a

Ord a => Ord (I a)

Methods

compare :: I a -> I a -> Ordering

(<) :: I a -> I a -> Bool

(<=) :: I a -> I a -> Bool

(>) :: I a -> I a -> Bool

(>=) :: I a -> I a -> Bool

max :: I a -> I a -> I a

min :: I a -> I a -> I a

Show a => Show (I a)

Methods

showsPrec :: Int -> I a -> ShowS

show :: I a -> String

showList :: [I a] -> ShowS

type WitnessC p q (I a)

type WitnessC p q (I a) = Witness p q a

BP

Backprop

backprop :: forall rs a. Every Num rs => (forall s. BPOp s rs a) -> Tuple rs -> (a, Tuple rs)

Perform back-propagation on the given BPOp. Returns the result of the operation it represents, as well as the gradient of the result with respect to its inputs. See module header for Numeric.Backprop and package documentation for examples and usages.

evalBPOp

Arguments

:: (forall s. BPOp s rs a)

BPOp to run

-> Tuple rs

input

-> a

output

Simply run the BPOp on an input tuple, getting the result without bothering with the gradient or with back-propagation.

gradBPOp

Arguments

:: Every Num rs

=> (forall s. BPOp s rs a)

BPOp to differentiate

-> Tuple rs

input

-> Tuple rs

gradient

Run the BPOp on an input tuple and return the gradient of the result with respect to the input tuple.

backprop' :: Prod Summer rs -> Prod Unity rs -> (forall s. BPOp s rs a) -> Tuple rs -> (a, Tuple rs)

A version of backprop taking explicit Summers and Unitys, so it can be run with types that aren't instances of Num.

gradBPOp'

Arguments

:: Prod Summer rs

-> Prod Unity rs

-> (forall s. BPOp s rs a)

BPOp to differentiate'

-> Tuple rs

input

-> Tuple rs

gradient

A version of gradBPOp taking explicit Summers and Unitys, so it can be run with types that aren't instances of Num.

Utility combinators

withInps :: Known Length rs => (Prod (BVar s rs) rs -> BP s rs a) -> BP s rs a

Runs a continuation on a Prod of all of the input BVars.

Handy for bringing the environment into scope and doing stuff with it:

foo :: BPOp '[Double, Int] a
foo = withInps $ \(x :< y :< Ø) -> do
    -- do stuff with inputs

Looks kinda like foo (x :< y :< Ø) = -- ..., don't it?

Note that the above is the same as

foo :: BPOp '[Double, Int] a
foo = do
    case inpVars of
      x ::< Ø - do
        -- do stuff with inputs

But just a little nicer!

implicitly :: (Known Length rs, Num a) => BPOpI s rs a -> BPOp s rs a

Convert a BPOpI into a BPOp. That is, convert a function on a bundle of BVars (generating an implicit graph) into a fully fledged BPOp that you can run backprop on. See BPOpI for more information.

If you are going to write exclusively using implicit BVar operations, it might be more convenient to use Numeric.Backprop.Implicit instead, which is geared around that use case.

withInps' :: Length rs -> (Prod (BVar s rs) rs -> BP s rs a) -> BP s rs a

A version of withInps taking explicit Length, indicating the number of inputs required and their types.

Mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the list of inputs. If you ever actually explicitly write down rs as a list of types, you should be able to just use withInps.

implicitly' :: Length rs -> Summer a -> BPOpI s rs a -> BPOp s rs a

A version of implicitly taking explicit Length and an explicit Summer, indicating the number of inputs required and their types, and also allowing it to work on types that aren't instances of Num.

Requiring an explicit Length is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the list of inputs. If you ever actually explicitly write down rs as a list of types, you should be able to just use implicitly.

Vars

constVar :: a -> BVar s rs a

Create a BVar that represents just a specific value, that doesn't depend on any other BVars.

inpVar :: Index rs a -> BVar s rs a

Create a BVar given an index into the input environment. For an example,

inpVar IZ

would refer to the first input variable (the Int in a BP s '[Int, Bool]), and

inpVar (IS IZ)

Would refer to the second input variable (the Bool in a BP s '[Int, Bool])

Typically, there shouldn't be any reason to use inpVar directly. It's cleaner to get all of your input BVars together using withInps or inpVars.

inpVars :: Known Length rs => Prod (BVar s rs) rs

Get a Prod (tupling) of BVars for all of the input environment (rs) of the BP s rs

For example, if your BP has an Int and Double in its input environment (a BP s '[Int, Double]), this would return a BVar pointing to the Int and a BVar pointing to the Double.

case (inpVars :: Prod (BVar s '[Int, Double]) '[Int, Double]) of
  x :< y :< Ø -> do
    -- the first item, x, is a var to the input Int
    -- x :: BVar s '[Int, Double] Int
    -- the second item, y, is a var to the input Double
    -- y :: BVar s '[Int, Double] Double

bpOp :: (Every Num rs, Known Length rs) => BPOp s rs a -> OpB s rs a

Turn a BPOp into an OpB. Basically converts a BP taking an rs and producing an a into an Op taking an rs and returning an a, with all of the powers and utility of an Op, including all of its gradient-finding glory.

Really just reveals the fact that any BPOp s rs a is itself an Op, an OpB s rs a, which makes it a differentiable function.

Handy because an OpB can be used with almost all of the Op-related functions in this moduel, including opVar, ~$, etc.

bindVar :: Num a => BVar s rs a -> BP s rs (BVar s rs a)

Concretizes a delayed BVar. If you build up a BVar using numeric functions like + or * or using liftB, it'll defer the evaluation, and all of its usage sites will create a separate graph node.

Use bindVar if you ever intend to use a BVar in more than one location.

-- bad
errSquared :: Num a => BP s '[a, a] a
errSquared = withInp $ \(r :< t :< Ø) -> do
    let err = r - t
    return (err * err)   -- err is used twice!

-- good
errSquared :: Num a => BP s '[a, a] a
errSquared = withInps $ \(r :< t :< Ø) -> do
    let err = r - t
    e <- bindVar err     -- force e, so that it's safe to use twice!
    return (e * e)

-- better
errSquared :: Num a => BP s '[a, a] a
errSquared = withInps $ \(r :< t :< Ø) -> do
    let err = r - t
    e <- bindVar err
    bindVar (e * e)      -- result is forced so user doesn't have to worry

Note the relation to opVar ~$ liftB / .$:

opVar o xs    = bindVar (liftB o xs)
o ~$ xs       = bindVar (o .$ xs)
op2 (*) ~$ (x :< y :< Ø) = bindVar (x * y)

So you can avoid bindVar altogether if you use the explicitly binding ~$ and opVar etc.

Note that bindVar on BVars that are already forced is a no-op.

inpVars' :: Length rs -> Prod (BVar s rs) rs

A version of inpVars taking explicit Length, indicating the number of inputs required and their types.

Mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the list of inputs. If you ever actually explicitly write down rs as a list of types, you should be able to just use inpVars.

bpOp' :: Prod Summer rs -> Prod Unity rs -> BPOp s rs a -> OpB s rs a

bpOp, but taking explicit Summers and Unitys, for the situation where the rs are not instance of Num.

bindVar' :: Summer a -> BVar s rs a -> BP s rs (BVar s rs a)

A version of bindVar that requires an explicit Summer, so that you can use it on values whose types aren't instances of Num.

From Ops

opVar :: Num a => OpB s as a -> Prod (BVar s rs) as -> BP s rs (BVar s rs a)

Apply an OpB to a Prod (tupling) of BVars.

If you had an OpB s '[a, b, c] d, this function will expect a 3-Prod of a BVar s rs a, a BVar s rs b, and a BVar s rs c, and the result will be a BVar s rs d:

myOp :: OpB s '[a, b, c] d
x    :: BVar s rs a
y    :: BVar s rs b
z    :: BVar s rs c

x :< y :< z :< Ø              :: Prod (BVar s rs) '[a, b, c]
opVar myOp (x :< y :< z :< Ø) :: BP s rs (BVar s rs d)

Note that OpB is a superclass of Op, so you can provide any Op here, as well (like those created by op1, op2, constOp, op0 etc.)

opVar has an infix alias, ~$, so the above example can also be written as:

myOp ~$ (x :< y :< z :< Ø) :: BP s rs (BVar s rs d)

to let you pretend that you're applying the myOp function to three inputs.

Also note the relation between opVar and liftB and bindVar:

opVar o xs = bindVar (liftB o xs)

opVar can be thought of as a "binding" version of liftB.

(~$) :: Num a => OpB s as a -> Prod (BVar s rs) as -> BP s rs (BVar s rs a) infixr 5

Infix synonym for opVar, which lets you pretend that you're applying OpBs as if they were functions:

myOp :: OpB s '[a, b, c] d
x    :: BVar s rs a
y    :: BVar s rs b
z    :: BVar s rs c

x :< y :< z :< Ø           :: Prod (BVar s rs) '[a, b, c]
myOp ~$ (x :< y :< z :< Ø) :: BP s rs (BVar s rs d)

Note that OpB is a superclass of Op, so you can pass in any Op here, as well (like those created by op1, op2, constOp, op0 etc.)

~$ can also be thought of as a "binding" version of .$:

o ~$ xs = bindVar (o .$ xs)

opVar1 :: Num b => OpB s '[a] b -> BVar s rs a -> BP s rs (BVar s rs b)

Convenient wrapper over opVar that takes an OpB with one argument and a single BVar argument. Lets you not have to type out the entire Prod.

opVar1 o x = opVar o (x :< 'Ø')

myOp :: Op '[a] b
x    :: BVar s rs a

opVar1 myOp x :: BP s rs (BVar s rs b)

Note that OpB is a superclass of Op, so you can pass in an Op here (like one made with op1) as well.

opVar2 :: Num c => OpB s '[a, b] c -> BVar s rs a -> BVar s rs b -> BP s rs (BVar s rs c)

Convenient wrapper over opVar that takes an OpB with two arguments and two BVar arguments. Lets you not have to type out the entire Prod.

opVar2 o x y = opVar o (x :< y :< 'Ø')

myOp :: Op '[a, b] c
x    :: BVar s rs a
y    :: BVar s rs b

opVar2 myOp x y :: BP s rs (BVar s rs c)

Note that OpB is a superclass of Op, so you can pass in an Op here (like one made with op2) as well.

opVar3 :: Num d => OpB s '[a, b, c] d -> BVar s rs a -> BVar s rs b -> BVar s rs c -> BP s rs (BVar s rs d)

Convenient wrapper over opVar that takes an OpB with three arguments and three BVar arguments. Lets you not have to type out the entire Prod.

opVar3 o x y z = opVar o (x :< y :< z :< 'Ø')

myOp :: Op '[a, b, c] d
x    :: BVar s rs a
y    :: BVar s rs b
z    :: BVar s rs c

opVar3 myOp x y z :: BP s rs (BVar s rs d)

Note that OpB is a superclass of Op, so you can pass in an Op here (like one made with op3) as well.

(-$) :: (Every Num as, Known Length as, Num a) => BPOp s as a -> Prod (BVar s rs) as -> BPOp s rs a infixr 5

Lets you treat a BPOp s as b as an Op as b, and "apply" arguments to it just like you would with an Op and ~$ / opVar.

Basically a convenient wrapper over bpOp and ~$:

o -$ xs = bpOp o ~$ xs

So for a BPOp s as b, you can "plug in" BVars to as, and get a b as a result.

Useful for running a BPOp s as b that you got from a different function, and "plugging in" its as inputs with BVars from your current environment.

opVar' :: forall s rs as a. Summer a -> OpB s as a -> Prod (BVar s rs) as -> BP s rs (BVar s rs a)

A version of opVar taking an explicit Summer, so can be used on values of types that aren't instances of Num.

opVar1' :: Summer b -> OpB s '[a] b -> BVar s rs a -> BP s rs (BVar s rs b)

A version of opVar1 taking an explicit Summer, so can be used on values of types that aren't instances of Num.

opVar2' :: Summer c -> OpB s '[a, b] c -> BVar s rs a -> BVar s rs b -> BP s rs (BVar s rs c)

A version of opVar2 taking an explicit Summer, so can be used on values of types that aren't instances of Num.

opVar3' :: Summer d -> OpB s '[a, b, c] d -> BVar s rs a -> BVar s rs b -> BVar s rs c -> BP s rs (BVar s rs d)

A version of opVar3 taking an explicit Summer, so can be used on values of types that aren't instances of Num.

Var manipulation

As parts

partsVar :: forall s rs bs b. (Every Num bs, Known Length bs) => Iso' b (Tuple bs) -> BVar s rs b -> BP s rs (Prod (BVar s rs) bs)

Use an Iso (or compatible Iso from the lens library) to "pull out" the parts of a data type and work with each part as a BVar.

If there is an isomorphism between a b and a Tuple as (that is, if an a is just a container for a bunch of as), then it lets you break out the as inside and work with those.

data Foo = F Int Bool

fooIso :: Iso' Foo (Tuple '[Int, Bool])
fooIso = iso (\(F i b)         -> i ::< b ::< Ø)
             (\(i ::< b ::< Ø) -> F i b        )

partsVar fooIso :: BVar rs Foo -> BP s rs (Prod (BVar s rs) '[Int, Bool])

stuff :: BP s '[Foo] a
stuff = withInps $ \(foo :< Ø) -> do
    i :< b :< Ø <- partsVar fooIso foo
    -- now, i is a BVar pointing to the Int inside foo
    -- and b is a BVar pointing to the Bool inside foo
    -- you can do stuff with the i and b here

You can use this to pass in product types as the environment to a BP, and then break out the type into its constituent products.

Note that for a type like Foo, fooIso can be generated automatically with Generic from GHC.Generics and Generic from Generics.SOP and generics-sop, using the gTuple iso. See gSplit for more information.

Also, if you are literally passing a tuple (like BP s '[Tuple '[Int, Bool]) then you can give in the identity isomorphism (id) or use splitVars.

(#<~) :: (Every Num bs, Known Length bs) => Iso' b (Tuple bs) -> BVar s rs b -> BP s rs (Prod (BVar s rs) bs) infixr 1

A useful infix alias for partsVar.

Building on the example from partsVar:

data Foo = F Int Bool

fooIso :: Iso' Foo (Tuple '[Int, Bool])
fooIso = iso (\(F i b)         -> i ::< b ::< Ø)
             (\(i ::< b ::< Ø) -> F i b        )

stuff :: BP s '[Foo] a
stuff = withInps $ \(foo :< Ø) -> do
    i :< b :< Ø <- fooIso #<~ foo
    -- now, i is a BVar pointing to the Int inside foo
    -- and b is a BVar pointing to the Bool inside foo
    -- you can do stuff with the i and b here

See gSplit for an example usage of splitting up an arbitrary product type (like Foo) using GHC.Geneics and Generics.SOP.

withParts :: (Every Num bs, Known Length bs) => Iso' b (Tuple bs) -> BVar s rs b -> (Prod (BVar s rs) bs -> BP s rs a) -> BP s rs a

A continuation-based version of partsVar. Instead of binding the parts and using it in the rest of the block, provide a continuation to handle do stuff with the parts inside.

Building on the example from partsVar:

data Foo = F Int Bool

fooIso :: Iso' Foo (Tuple '[Int, Bool])
fooIso = iso (\(F i b)         -> i ::< b ::< Ø)
             (\(i ::< b ::< Ø) -> F i b        )

stuff :: BP s '[Foo] a
stuff = withInps $ \(foo :< Ø) -> do
    withParts fooIso foo $ \(i :< b :< Ø) -> do
      -- now, i is a BVar pointing to the Int inside foo
      -- and b is a BVar pointing to the Bool inside foo
      -- you can do stuff with the i and b here

Useful so that you can work with the internal parts of the data type in a closure, so the parts don't leak out to the rest of your BP. But, mostly just a stylistic choice.

splitVars :: forall s rs as. (Every Num as, Known Length as) => BVar s rs (Tuple as) -> BP s rs (Prod (BVar s rs) as)

Split out a BVar of a tuple into a tuple (Prod) of BVars.

-- the environment is a single Int-Bool tuple, tup
stuff :: BP s '[ Tuple '[Int, Bool] ] a
stuff = withInps $ \(tup :< Ø) -> do
    i :< b :< Ø <- splitVars tup
    -- now, i is a BVar pointing to the Int inside tup
    -- and b is a BVar pointing to the Bool inside tup
    -- you can do stuff with the i and b here

Note that

splitVars = partsVar id

gSplit :: (Every Num bs, Known Length bs, Generic b, Code b ~ '[bs]) => BVar s rs b -> BP s rs (Prod (BVar s rs) bs)

Using Generic from GHC.Generics and Generic from Generics.SOP, split a BVar containing a product type into a tuple (Prod) of BVars pointing to each value inside.

Building on the example from partsVar:

import qualified Generics.SOP as SOP

data Foo = F Int Bool
  deriving Generic

instance SOP.Generic Foo

gSplit :: BVar rs Foo -> BP s rs (Prod (BVar s rs) '[Int, Bool])

stuff :: BP s '[Foo] a
stuff = withInps $ \(foo :< Ø) -> do
    i :< b :< Ø <- gSplit foo
    -- now, i is a BVar pointing to the Int inside foo
    -- and b is a BVar pointing to the Bool inside foo
    -- you can do stuff with the i and b here

Because Foo is a straight up product type, gSplit can use GHC.Generics and take out the items inside.

Note that because

gSplit = splitVars gTuple

Then, you can also use gTuple with #<~:

stuff :: BP s '[Foo] a
stuff = withInps $ \(foo :< Ø) -> do
    i :< b :< Ø <- gTuple #<~ foo
    -- now, i is a BVar pointing to the Int inside foo
    -- and b is a BVar pointing to the Bool inside foo
    -- you can do stuff with the i and b here

gTuple :: (Generic a, Code a ~ '[as]) => Iso' a (Tuple as)

An Iso between a type that is a product type, and a tuple that contains all of its components. Uses Generics.SOP and the Generic typeclass.

>>> import qualified Generics.SOP as SOP
>>> data Foo = A Int Bool      deriving Generic
>>> instance SOP.Generic Foo
>>> view gTuple (A 10 True)
10 ::< True ::< Ø
>>> review gTuple (15 ::< False ::< Ø)
A 15 False

partsVar' :: forall s rs bs b. Prod Summer bs -> Prod Unity bs -> Iso' b (Tuple bs) -> BVar s rs b -> BP s rs (Prod (BVar s rs) bs)

A version of partsVar taking explicit Summers and Unitys, so it can be run with internal types that aren't instances of Num.

withParts' :: Prod Summer bs -> Prod Unity bs -> Iso' b (Tuple bs) -> BVar s rs b -> (Prod (BVar s rs) bs -> BP s rs a) -> BP s rs a

A version of withParts taking explicit Summers and Unitys, so it can be run with internal types that aren't instances of Num.

splitVars' :: forall s rs as. Prod Summer as -> Prod Unity as -> BVar s rs (Tuple as) -> BP s rs (Prod (BVar s rs) as)

A version of splitVars taking explicit Summers and Unitys, so it can be run with types that aren't instances of Num.

gSplit' :: (Generic b, Code b ~ '[bs]) => Prod Summer bs -> Prod Unity bs -> BVar s rs b -> BP s rs (Prod (BVar s rs) bs)

A version of gSplit taking explicit Summers and Unitys, so it can be run with internal types that aren't instances of Num.

As sums

choicesVar :: forall s rs bs b. (Every Num bs, Known Length bs) => Iso' b (Sum I bs) -> BVar s rs b -> BP s rs (Sum (BVar s rs) bs)

Use an Iso (or compatible Iso from the lens library) to "pull out" the different constructors of a sum type and return a (choice) sum of BVars that you can pattern match on.

If there is an isomorphism between a b and a Sum I as (that is, if an a is just a sum type for every type in as), then it lets you branch on which constructor is used inside the b.

Essentially implements pattern matching on BVar values.

data Bar = A Int | B Bool | C String

barIso :: Iso' Bar (Sum I '[Int, Bool, String])
barIso = iso (\case A i -> InL (I i)
                      B b -> InR (InL (I b))
                      C s -> InR (InR (InL (I s))
               )
               (\case InL (I i)           -> A i
                      InR (InL (I b))       -> B b
                      InR (InR (InL (I s))) -> C s
               )

choicesVar barIso :: BVar rs Bar -> BP s rs (Sum I (BVar s rs) '[Int, Bool, String])

stuff :: BP s '[Bar] a
stuff = withInps $ \(bar :< Ø) -> do
    c <- choicesVar barIso bar
    case c of
      InL i -> do
         -- in this branch, bar was made with the A constructor
         -- i is the Int inside it
      InR (InL b) -> do
         -- in this branch, bar was made with the B constructor
         -- b is the Bool inside it
      InR (InR (InL s)) -> do
         -- in this branch, bar was made with the B constructor
         -- s is the String inside it

You can use this to pass in sum types as the environment to a BP, and then branch on which constructor the value was made with.

See Numeric.Backprop for a mini-tutorial on Sum.

(?<~) :: (Every Num bs, Known Length bs) => Iso' b (Sum I bs) -> BVar s rs b -> BP s rs (Sum (BVar s rs) bs) infixr 1

A useful infix alias for choicesVar.

Building on the example from choicesVar:

data Bar = A Int | B Bool | C String

barIso :: Iso' Bar (Sum I '[Int, Bool, String])
barIso = iso (\case A i -> InL (I i)
                      B b -> InR (InL (I b))
                      C s -> InR (InR (InL (I s))
               )
               (\case InL (I i)           -> A i
                      InR (InL (I b))       -> B b
                      InR (InR (InL (I s))) -> C s
               )

stuff :: BP s '[Bar] a
stuff = withInps $ \(bar :< Ø) -> do
    c <- barIso ?<~ bar
    case c of
      InL i -> do
         -- in this branch, bar was made with the A constructor
         -- i is the Int inside it
      InR (InL b) -> do
         -- in this branch, bar was made with the B constructor
         -- b is the Bool inside it
      InR (InR (InL s)) -> do
         -- in this branch, bar was made with the B constructor
         -- s is the String inside it

withChoices :: forall s rs bs b a. (Every Num bs, Known Length bs) => Iso' b (Sum I bs) -> BVar s rs b -> (Sum (BVar s rs) bs -> BP s rs a) -> BP s rs a

A continuation-based version of choicesVar. Instead of binding the parts and using it in the rest of the block, provide a continuation that will handle every possible constructor/case of the type of the value the BVar points to.

Building on the example from choicesVar:

data Bar = A Int | B Bool | C String

barIso :: Iso' Bar (Sum I '[Int, Bool, String])
barIso = iso (\case A i -> InL (I i)
                      B b -> InR (InL (I b))
                      C s -> InR (InR (InL (I s))
               )
               (\case InL (I i)           -> A i
                      InR (InL (I b))       -> B b
                      InR (InR (InL (I s))) -> C s
               )

choicesVar barIso :: BVar rs Bar -> BP s rs (Sum I (BVar s rs) '[Int, Bool, String])

stuff :: BP s '[Bar] a
stuff = withInps $ \(bar :< Ø) -> do
    withChoices barIso bar $ case
      InL i -> do
         -- in this branch, bar was made with the A constructor
         -- i is the Int inside it
      InR (InL b) -> do
         -- in this branch, bar was made with the B constructor
         -- b is the Bool inside it
      InR (InR (InL s)) -> do
         -- in this branch, bar was made with the B constructor
         -- s is the String inside it

Nicer than choicesVar directly, because you don't have to give the result a superfluous name before pattern matching on it. You can just directly pattern match in the lambda, so there's a lot less syntactical noise.

choicesVar' :: forall s rs bs b. Prod Summer bs -> Prod Unity bs -> Iso' b (Sum I bs) -> BVar s rs b -> BP s rs (Sum (BVar s rs) bs)

A version of choicesVar taking explicit Summers and Unitys, so it can be run with internal types that aren't instances of Num.

withChoices' :: forall s rs bs b a. Prod Summer bs -> Prod Unity bs -> Iso' b (Sum I bs) -> BVar s rs b -> (Sum (BVar s rs) bs -> BP s rs a) -> BP s rs a

A version of withChoices taking explicit Summers and Unitys, so it can be run with internal types that aren't instances of Num.

Like the Prod type (see mini-tutorial at Numeric.Backprop), the Sum type lets you make arbitrary sum types over different types and work with them generically.

A Sum f '[a, b, c] contains either an f a, an f b, or an f c, and is constructed with the constructors InL and InR, which are analogous to Left and Right.

For a value of type Sum f '[Int, Bool, String], there are three constructors:

InL             :: f Int    -> Sum f '[Int, Bool, String]
InL . InR       :: f Bool   -> Sum f '[Int, Bool, String]
InL . InR . InR :: f String -> Sum f '[Int, Bool, String]

Each InR "pushes deeper" into the Sum.

Likewise, if you have a value of type Sum f '[Int, Bool, String], you can see which constructor it was made (and what type it contains) with by pattern matching:

foo :: Sum f '[Int, Bool, String]

case foo of
  InL i         -> -- foo contains an "f Int"
  InR (InL b)   -> -- foo contains an "f Bool"
  InR (InR (InL s)) -> -- foo contains an "f String"

data Sum k f a :: forall k. (k -> *) -> [k] -> * where

Constructors

InL :: Sum k f ((:) k a1 as)

InR :: Sum k f ((:) k a1 as)

Instances

Witness p q (f a) => Witness p q (Sum k f ((:) k a ([] k)))

Associated Types

type WitnessC (p :: Constraint) (q :: Constraint) (Sum k f ((:) k a ([] k))) :: Constraint

Methods

(\\) :: p => (q -> r) -> Sum k f ((k ': a) [k]) -> r

(Witness p q (f a1), Witness p q (Sum a f ((:<) a b as))) => Witness p q (Sum a f ((:<) a a1 ((:<) a b as)))

Associated Types

type WitnessC (p :: Constraint) (q :: Constraint) (Sum a f ((:<) a a1 ((:<) a b as))) :: Constraint

Methods

(\\) :: p => (q -> r) -> Sum a f ((a :< a1) ((a :< b) as)) -> r

IxFunctor1 k [k] (Index k) (Sum k)

Methods

imap1 :: (forall a. i b a -> f a -> g a) -> t f b -> t g b

IxFoldable1 k [k] (Index k) (Sum k)

Methods

ifoldMap1 :: Monoid m => (forall a. i b a -> f a -> m) -> t f b -> m

IxTraversable1 k [k] (Index k) (Sum k)

Methods

itraverse1 :: Applicative h => (forall a. i b a -> f a -> h (g a)) -> t f b -> h (t g b)

Functor1 [k] k (Sum k)

Methods

map1 :: (forall a. f a -> g a) -> t f b -> t g b

Foldable1 [k] k (Sum k)

Methods

foldMap1 :: Monoid m => (forall a. f a -> m) -> t f b -> m

Traversable1 [k] k (Sum k)

Methods

traverse1 :: Applicative h => (forall a. f a -> h (g a)) -> t f b -> h (t g b)

Eq1 k f => Eq1 [k] (Sum k f)

Methods

eq1 :: f a -> f a -> Bool

neq1 :: f a -> f a -> Bool

Ord1 k f => Ord1 [k] (Sum k f)

Methods

compare1 :: f a -> f a -> Ordering

(<#) :: f a -> f a -> Bool

(>#) :: f a -> f a -> Bool

(<=#) :: f a -> f a -> Bool

(>=#) :: f a -> f a -> Bool

Show1 k f => Show1 [k] (Sum k f)

Methods

showsPrec1 :: Int -> f a -> ShowS

show1 :: f a -> String

Read1 k f => Read1 [k] (Sum k f)

Methods

readsPrec1 :: Int -> ReadS (Some (Sum k f) f)

ListC ((<$>) Constraint * Eq ((<$>) * k f as)) => Eq (Sum k f as)

Methods

(==) :: Sum k f as -> Sum k f as -> Bool

(/=) :: Sum k f as -> Sum k f as -> Bool

(ListC ((<$>) Constraint * Eq ((<$>) * k f as)), ListC ((<$>) Constraint * Ord ((<$>) * k f as))) => Ord (Sum k f as)

Methods

compare :: Sum k f as -> Sum k f as -> Ordering

(<) :: Sum k f as -> Sum k f as -> Bool

(<=) :: Sum k f as -> Sum k f as -> Bool

(>) :: Sum k f as -> Sum k f as -> Bool

(>=) :: Sum k f as -> Sum k f as -> Bool

max :: Sum k f as -> Sum k f as -> Sum k f as

min :: Sum k f as -> Sum k f as -> Sum k f as

ListC ((<$>) Constraint * Show ((<$>) * k f as)) => Show (Sum k f as)

Methods

showsPrec :: Int -> Sum k f as -> ShowS

show :: Sum k f as -> String

showList :: [Sum k f as] -> ShowS

type WitnessC p q (Sum a f ((:<) a a1 ((:<) a b as)))

type WitnessC p q (Sum a f ((:<) a a1 ((:<) a b as))) = (Witness p q (f a1), Witness p q (Sum a f ((:<) a b as)))

type WitnessC p q (Sum k f ((:) k a ([] k)))

type WitnessC p q (Sum k f ((:) k a ([] k))) = Witness p q (f a)

As sums of products

sopVar :: forall s rs bss b. (Known Length bss, Every (Every Num ∧ Known Length) bss) => Iso' b (Sum Tuple bss) -> BVar s rs b -> BP s rs (Sum (Prod (BVar s rs)) bss)

A combination of partsVar and choicesVar, that lets you split a type into a sum of products. Using an Iso (or compatible Iso from the lens library), you can pull out a type that is a sum of products into a sum of products of BVars.

Implements branching on the constructors of a value that a BVar contains, and also splitting out the different items inside each constructor.

data Baz = A Int    Bool
         | B String Double


bazIso :: Iso' Baz (Sum Tuple '[ '[Int, Bool], '[String, Double] ])
bazIso = iso (\case A i b -> InL (I (i ::< b ::< Ø))
                      B s d -> InR (InL (I (s ::< d ::< Ø)))
               )
               (\case InL (I (i ::::< Ø))     - A i b
                      InR (InL (I (s ::::< Ø))) - B s d
               )

sopVar bazIso :: BVar rs Baz -> BP s rs (Sum (Prod (BVar s rs)) '[ '[Int, Bool], '[String, Double] ])

stuff :: BP s '[Baz] a
stuff = withInps $ \(baz :< Ø) -> do
    c <- sopVar barIso baz
    case c of
      InL (i ::< Ø) - do
         -- in this branch, baz was made with the A constructor
         -- i and b are the Int and Bool inside it
      InR (InL (s ::< Ø)) - do
         -- in this branch, baz was made with the B constructor
         -- s and d are the String and Double inside it

Essentially exists to implement "pattern matching" on multiple constructors and fields for the value inside a BVar.

Note that for a type like Baz, bazIso can be generated automatically with Generic from GHC.Generics and Generic from Generics.SOP and generics-sop, with gSOP. See gSplits for more information.

See Numeric.Backprop for a mini-tutorial on Sum.

gSplits :: forall s rs b. (Generic b, Known Length (Code b), Every (Every Num ∧ Known Length) (Code b)) => BVar s rs b -> BP s rs (Sum (Prod (BVar s rs)) (Code b))

Using Generic from GHC.Generics and Generic from Generics.SOP, split a BVar containing a sum of products (any simple ADT, essentialy) into a Sum of each constructor, each containing a tuple (Prod) of BVars pointing to each value inside.

Building on the example from sopVar:

import qualified Generics.SOP as SOP

data Baz = A Int    Bool
         | B String Double
  deriving Generic

instance SOP.Generic Baz

gSplits :: BVar rs Baz -> BP s rs (Sum (Prod (BVar s rs)) '[ '[Int, Bool], '[String, Double] ])

stuff :: BP s '[Baz] a
stuff = withInps $ \(baz :< Ø) -> do
    c <- gSplits baz
    case c of
      InL (i ::< Ø) - do
         -- in this branch, baz was made with the A constructor
         -- i and b are the Int and Bool inside it
      InR (InL (s ::< Ø)) - do
         -- in this branch, baz was made with the B constructor
         -- s and d are the String and Double inside it

Because Foo is a straight up sum-of-products type, gSplits can use GHC.Generics and take out the items inside.

Note:

gSplit = splitVars gSOP

See Numeric.Backprop for a mini-tutorial on Sum.

gSOP :: Generic a => Iso' a (Sum Tuple (Code a))

An Iso between a sum type whose constructors are products, and a sum (Sum) of products (Tuple). Uses Generics.SOP and the Generic typeclass.

>>> import qualified Generics.SOP as SOP
>>> data Bar = A Int Bool | B String Double
>>> instance SOP.Generic Bar
>>> 'view' 'gSOP' (A 10 True)
'InL' (10 ::< True ::< Ø)
>>> 'view' 'gSOP' (B "hello" 3.4)
'InR' ('InL' ("hello" ::< 3.4 ::< Ø))
>>> 'review' 'gTuple' ('InL' (15 ::< False ::< Ø))
A 15 False
>>> 'review' 'gTuple' ('InR' ('InL' ("bye" ::< 9.8 ::< Ø)))
B "bye" 9.8

sopVar' :: forall s rs bss b. Prod (Prod Summer) bss -> Prod (Prod Unity) bss -> Iso' b (Sum Tuple bss) -> BVar s rs b -> BP s rs (Sum (Prod (BVar s rs)) bss)

A version of sopVar taking explicit Summers and Unitys, so it can be run with internal types that aren't instances of Num.

gSplits' :: forall s rs b. Generic b => Prod (Prod Summer) (Code b) -> Prod (Prod Unity) (Code b) -> BVar s rs b -> BP s rs (Sum (Prod (BVar s rs)) (Code b))

A version of gSplits taking explicit Summers and Unitys, so it can be run with internal types that aren't instances of Num.

Combining

liftB :: OpB s as a -> Prod (BVar s rs) as -> BVar s rs a

Apply OpB over a Prod of BVars, as inputs. Provides "implicit-graph" back-propagation, with deferred evaluation.

If you had an OpB s '[a, b, c] d, this function will expect a 3-Prod of a BVar s rs a, a BVar s rs b, and a BVar s rs c, and the result will be a BVar s rs d:

myOp :: OpB s '[a, b, c] d
x    :: BVar s rs a
y    :: BVar s rs b
z    :: BVar s rs c

x :< y :< z :< Ø              :: Prod (BVar s rs) '[a, b, c]
liftB myOp (x :< y :< z :< Ø) :: BVar s rs d

Note that OpB is a superclass of Op, so you can provide any Op here, as well (like those created by op1, op2, constOp, op0 etc.)

liftB has an infix alias, .$, so the above example can also be written as:

myOp .$ (x :< y :< z :< Ø) :: BVar s rs d

to let you pretend that you're applying the myOp function to three inputs.

The result is a new deferred BVar. This should be fine in most cases, unless you use the result in more than one location. This will cause evaluation to be duplicated and multiple redundant graph nodes to be created. If you need to use it in two locations, you should use opVar instead of liftB, or use bindVar:

opVar o xs = bindVar (liftB o xs)

liftB can be thought of as a "deferred evaluation" version of opVar.

(.$) :: OpB s as a -> Prod (BVar s rs) as -> BVar s rs a infixr 5

Infix synonym for liftB, which lets you pretend that you're applying OpBs as if they were functions:

myOp :: OpB s '[a, b, c] d
x    :: BVar s rs a
y    :: BVar s rs b
z    :: BVar s rs c

x :< y :< z :< Ø           :: Prod (BVar s rs) '[a, b, c]
myOp .$ (x :< y :< z :< Ø) :: BVar s rs d

Note that OpB is a superclass of Op, so you can pass in any Op here, as well (like those created by op1, op2, constOp, op0 etc.)

See the documentation for liftB for all the caveats of this usage.

.$ can also be thought of as a "deferred evaluation" version of ~$:

o ~$ xs = bindVar (o .$ xs)

liftB1 :: OpB s '[a] b -> BVar s rs a -> BVar s rs b

Convenient wrapper over liftB that takes an OpB with one argument and a single BVar argument. Lets you not have to type out the entire Prod.

liftB1 o x = liftB o (x :< 'Ø')

myOp :: Op '[a] b
x    :: BVar s rs a

liftB1 myOp x :: BVar s rs b

Note that OpB is a superclass of Op, so you can pass in an Op here (like one made with op1) as well.

See the documentation for liftB for caveats and potential problematic situations with this.

liftB2 :: OpB s '[a, b] c -> BVar s rs a -> BVar s rs b -> BVar s rs c

Convenient wrapper over liftB that takes an OpB with two arguments and two BVar arguments. Lets you not have to type out the entire Prod.

liftB2 o x y = liftB o (x :< y :< 'Ø')

myOp :: Op '[a, b] c
x    :: BVar s rs a
y    :: BVar s rs b

liftB2 myOp x y :: BVar s rs c

Note that OpB is a superclass of Op, so you can pass in an Op here (like one made with op2) as well.

See the documentation for liftB for caveats and potential problematic situations with this.

liftB3 :: OpB s '[a, b, c] d -> BVar s rs a -> BVar s rs b -> BVar s rs c -> BVar s rs d

Convenient wrapper over liftB that takes an OpB with three arguments and three BVar arguments. Lets you not have to type out the entire Prod.

liftB3 o x y z = liftB o (x :< y :< z :< 'Ø')

myOp :: Op '[a, b, c] d
x    :: BVar s rs a
y    :: BVar s rs b
z    :: BVar s rs c

liftB3 myOp x y z :: BVar s rs d

Note that OpB is a superclass of Op, so you can pass in an Op here (like one made with op3) as well.

See the documentation for liftB for caveats and potential problematic situations with this.

Op

op1 :: Num a => (forall s. AD s (Forward a) -> AD s (Forward a)) -> Op '[a] a

Automatically create an Op of a numerical function taking one argument. Uses diff, and so can take any numerical function polymorphic over the standard numeric types.

>>> gradOp' (op1 (recip . negate)) (5 ::< Ø)
(-0.2, 0.04 ::< Ø)

op2 :: Num a => (forall s. Reifies s Tape => Reverse s a -> Reverse s a -> Reverse s a) -> Op '[a, a] a

Automatically create an Op of a numerical function taking two arguments. Uses grad, and so can take any numerical function polymorphic over the standard numeric types.

>>> gradOp' (op2 (\x y -> x * sqrt y)) (3 ::< 4 ::< Ø)
(6.0, 2.0 ::< 0.75 ::< Ø)

op3 :: Num a => (forall s. Reifies s Tape => Reverse s a -> Reverse s a -> Reverse s a -> Reverse s a) -> Op '[a, a, a] a

Automatically create an Op of a numerical function taking three arguments. Uses grad, and so can take any numerical function polymorphic over the standard numeric types.

>>> gradOp' (op3 (\x y z -> (x * sqrt y)**z)) (3 ::< 4 ::< 2 ::< Ø)
(36.0, 24.0 ::< 9.0 ::< 64.503 ::< Ø)

opN :: (Num a, Known Nat n) => (forall s. Reifies s Tape => Vec n (Reverse s a) -> Reverse s a) -> Op (Replicate n a) a

Automatically create an Op of a numerical function taking multiple arguments. Uses grad, and so can take any numerical function polymorphic over the standard numeric types.

>>> gradOp' (opN (\(x :+ y :+ Ø) -> x * sqrt y)) (3 ::< 4 ::< Ø)
(6.0, 2.0 ::< 0.75 ::< Ø)

composeOp

Arguments

:: (Monad m, Known Length as, Every Num as)

=> Prod (OpM m as) bs

Prod of OpMs taking as and returning different b in bs

-> OpM m bs c

OpM taking eac of the bs from the input Prod.

-> OpM m as c

Composed OpM

Compose OpMs together, similar to .. But, because all OpMs are \(\mathbb{R}^N \rightarrow \mathbb{R}\), this is more like sequence for functions, or liftAN.

That is, given an OpM m as b1, an OpM m as b2, and an OpM m as b3, it can compose them with an OpM m '[b1,b2,b3] c to create an OpM m as c.

composeOp1 :: (Monad m, Known Length as, Every Num as) => OpM m as b -> OpM m '[b] c -> OpM m as c

Convenient wrappver over composeOp for the case where the second function only takes one input, so the two OpMs can be directly piped together, like for ..

(~.) :: (Monad m, Known Length as, Every Num as) => OpM m '[b] c -> OpM m as b -> OpM m as c infixr 9

Convenient infix synonym for (flipped) composeOp1. Meant to be used just like .:

op1 negate            :: Op '[a]   a
op2 (+)               :: Op '[a,a] a

op1 negate ~. op2 (+) :: Op '[a, a] a

op1' :: (a -> (b, Maybe b -> a)) -> Op '[a] b

Create an Op of a function taking one input, by giving its explicit derivative. The function should return a tuple containing the result of the function, and also a function taking the derivative of the result and return the derivative of the input.

If we have

\[ \eqalign{ f &: \mathbb{R} \rightarrow \mathbb{R}\cr y &= f(x)\cr z &= g(y) } \]

Then the derivative \( \frac{dz}{dx} \), it would be:

\[ \frac{dz}{dx} = \frac{dz}{dy} \frac{dy}{dx} \]

If our Op represents \(f\), then the second item in the resulting tuple should be a function that takes \(\frac{dz}{dy}\) and returns \(\frac{dz}{dx}\).

If the input is Nothing, then \(\frac{dz}{dy}\) should be taken to be \(1\).

As an example, here is an Op that squares its input:

square :: Num a => Op '[a] a
square = op1' $ \x -> (x*x, \case Nothing -> 2 * x
                                  Just d  -> 2 * d * x
                      )

Remember that, generally, end users shouldn't directly construct Ops; they should be provided by libraries or generated automatically.

For numeric functions, single-input Ops can be generated automatically using op1.

op2' :: (a -> b -> (c, Maybe c -> (a, b))) -> Op '[a, b] c

Create an Op of a function taking two inputs, by giving its explicit gradient. The function should return a tuple containing the result of the function, and also a function taking the derivative of the result and return the derivative of the input.

If we have

\[ \eqalign{ f &: \mathbb{R}^2 \rightarrow \mathbb{R}\cr z &= f(x, y)\cr k &= g(z) } \]

Then the gradient \( \left< \frac{\partial k}{\partial x}, \frac{\partial k}{\partial y} \right> \) would be:

\[ \left< \frac{\partial k}{\partial x}, \frac{\partial k}{\partial y} \right> = \left< \frac{dk}{dz} \frac{\partial z}{dx}, \frac{dk}{dz} \frac{\partial z}{dy} \right> \]

If our Op represents \(f\), then the second item in the resulting tuple should be a function that takes \(\frac{dk}{dz}\) and returns \( \left< \frac{\partial k}{dx}, \frac{\partial k}{dx} \right> \).

If the input is Nothing, then \(\frac{dk}{dz}\) should be taken to be \(1\).

As an example, here is an Op that multiplies its inputs:

mul :: Num a => Op '[a, a] a
mul = op2' $ \x y -> (x*y, \case Nothing -> (y  , x  )
                                 Just d  -> (d*y, x*d)
                     )

Remember that, generally, end users shouldn't directly construct Ops; they should be provided by libraries or generated automatically.

For numeric functions, two-input Ops can be generated automatically using op2.

op3' :: (a -> b -> c -> (d, Maybe d -> (a, b, c))) -> Op '[a, b, c] d

Create an Op of a function taking three inputs, by giving its explicit gradient. See documentation for op2' for more details.

Utility

pattern (:>) :: forall k f a b. f a -> f b -> Prod k f ((:) k a ((:) k b ([] k))) infix 6

Construct a two element Prod. Since the precedence of (:>) is higher than (:<), we can conveniently write lists like:

>>> a :< b :> c

Which is identical to:

>>> a :< b :< c :< Ø

only :: f a -> Prod k f ((:) k a ([] k))

Build a singleton Prod.

head' :: Prod k f ((:<) k a as) -> f a

pattern (::<) :: forall a as. a -> Tuple as -> Tuple ((:<) * a as) infixr 5

Cons onto a Tuple.

only_ :: a -> Tuple ((:) * a ([] *))

Singleton Tuple.

newtype Summer a

Instructions on how to "sum" a list of values of a given type. Basically used as an explicit witness for a Num instance.

For most types, the only meaningful value of type Summer a is Summer sum. However, using Summer lets us use BP with types that are not instances of Num. Any type can be used, as long as you provide a way to "sum" it!

For most of the functions in this library, you can completely ignore this, as they will be generated automatically. You only need to work with this directly if you want to use custom types that aren't instances of Num with this library.

If 'Num a' is satisfied, one can create the canonical Summer using known :: Num a => Summer a.

Constructors

Summer

Fields

• runSummer :: [a] -> a

Instances

Num a => Known * Summer a

If a is an instance of Num, then the canonical Summer a is Summer sum.

Associated Types

type KnownC Summer (a :: Summer -> *) (a :: Summer) :: Constraint

Methods

known :: a a

type KnownC * Summer a

type KnownC * Summer a = Num a

newtype Unity a

A canonical "unity" (the multiplicative identity) for a given type. Basically used as an explicit witness for a Num instance.

For most types, the only meaningful value of type Unity a is Unity 1'. However, using Unity lets us use BP with types that are not instances of Num. Any type can be used, as long as you provide a way to get a multiplicative identity in it!

For most of the functions in this library, you can completely ignore this, as they will be generated automatically. You only need to work with this directly if you want to use custom types that aren't instances of Num with this library.

If 'Num a' is satisfied, one can create the canonical Unity using known :: Num a => Unity a.

Constructors

Unity

Fields

• getUnity :: a

Instances

Functor Unity

Methods

fmap :: (a -> b) -> Unity a -> Unity b

(<$) :: a -> Unity b -> Unity a

Show a => Show (Unity a)

Methods

showsPrec :: Int -> Unity a -> ShowS

show :: Unity a -> String

showList :: [Unity a] -> ShowS

Num a => Known * Unity a

If a is an instance of Num, then the canonical Unity a is Unity 1.

Associated Types

type KnownC Unity (a :: Unity -> *) (a :: Unity) :: Constraint

Methods

known :: a a

type KnownC * Unity a

type KnownC * Unity a = Num a

summers :: (Every Num as, Known Length as) => Prod Summer as

If all the types in as are instances of Num, generate a Prod Summer as, or a tuple of Summers for every type in as.

unities :: (Every Num as, Known Length as) => Prod Unity as

If all the types in as are instances of Num, generate a Prod Unity as, or a tuple of Unitys for every type in as.

summers' :: Every Num as => Length as -> Prod Summer as

Like summers, but requiring an explicit witness for the number of types in the list as.

unities' :: Every Num as => Length as -> Prod Unity as

Like unities, but requiring an explicit witness for the number of types in the list as.