Copyright | (c) Justin Le 2017 |
---|---|
License | BSD3 |
Maintainer | justin@jle.im |
Stability | experimental |
Portability | non-portable |
Safe Haskell | None |
Language | Haskell2010 |
Provides a monomorphic interface to the library and to the Numeric.Backprop module.
They are monomorphic in the sense that all of the inputs have to be of the same type. So, something like
BP
s '[Double, Double, Double] Int
From Numeric.Backprop would, in this module, be:
BP
sN3
Double Int
Instead of dealing with Prod
s and Tuple
s, this module works with
VecT
s and Vec
s, respectively. These are fixed-length vectors whose
length are encoded in their types, constructed with :*
(for VecT
) or
:+
(for Vec
).
Most of the concepts in normal heterogeneous backprop (for
Numeric.Backprop) should apply here as well, so you can look at any of
the tutorials or examples and repurpose them to work here. Just
remember to convert something like
to Op
'[a, a] b
.Op
N2
a b
As a comparison, this implements something similar in functionality to Numeric.AD and Numeric.AD.Mode.Reverse from the ad package, in that they both offer monomorphic automatic differentiation through back-propagation. This module doesn't allow the computation of jacobians or generalized gradients for \(\mathbb{R}^N \rightarrow \mathbb{R}^M\) functions. This module only computs gradients for (mathbb{R}^N rightarrow mathbb{R})-like functions. This is more of a conscious design decision in the API of this module rather than a fundamental limitation of the implementation.
This module also allows you to build explicit data dependency graphs so
the library can reduce duplication and perform optimizations, which may
or may not provide advantages over Numeric.AD.Mode.Reverse's
unsafePerformIO
-based implicit graph building.
- type BP s n r = BP s (Replicate n r)
- type BPOp s n r a = BP s n r (BVar s n r a)
- type BPOpI s n r a = VecT n (BVar s n r) r -> BVar s n r a
- type BVar s n a = BVar s (Replicate n a)
- type Op n a b = Op (Replicate n a) b
- type OpB s n a b = OpB s (Replicate n a) b
- data VecT k n f a :: forall k. N -> (k -> *) -> k -> * where
- type Vec n = VecT * n I
- newtype I a :: * -> * = I {
- getI :: a
- backprop :: forall n r a. Num r => (forall s. BPOp s n r a) -> Vec n r -> (a, Vec n r)
- evalBPOp :: forall n r a. (forall s. BPOp s n r a) -> Vec n r -> a
- gradBPOp :: forall n r a. Num r => (forall s. BPOp s n r a) -> Vec n r -> Vec n r
- withInps :: Known Nat n => (VecT n (BVar s n r) r -> BP s n r a) -> BP s n r a
- implicitly :: Known Nat n => BPOpI s n r a -> BPOp s n r a
- constVar :: a -> BVar s n r a
- inpVar :: Fin n -> BVar s n r r
- inpVars :: Known Nat n => VecT n (BVar s n r) r
- bpOp :: forall s n r a. (Num r, Known Nat n) => BPOp s n r a -> OpB s n r a
- bindVar :: forall s n r a. Num a => BVar s n r a -> BP s n r (BVar s n r a)
- opVar :: forall s m n r a b. Num b => OpB s m a b -> VecT m (BVar s n r) a -> BP s n r (BVar s n r b)
- (~$) :: forall s m n r a b. Num b => OpB s m a b -> VecT m (BVar s n r) a -> BP s n r (BVar s n r b)
- opVar1 :: forall s n r a b. Num b => OpB s N1 a b -> BVar s n r a -> BP s n r (BVar s n r b)
- opVar2 :: forall s n r a b. Num b => OpB s N2 a b -> BVar s n r a -> BVar s n r a -> BP s n r (BVar s n r b)
- opVar3 :: forall s n r a b. Num b => OpB s N3 a b -> BVar s n r a -> BVar s n r a -> BVar s n r a -> BP s n r (BVar s n r b)
- (-$) :: forall s m n r a b. (Num a, Num b, Known Nat m) => BPOp s m a b -> VecT m (BVar s n r) a -> BP s n r (BVar s n r b)
- liftB :: forall s m n a b r. OpB s m a b -> VecT m (BVar s n r) a -> BVar s n r b
- (.$) :: forall s m n a b r. OpB s m a b -> VecT m (BVar s n r) a -> BVar s n r b
- liftB1 :: OpB s N1 a a -> BVar s n r a -> BVar s n r a
- liftB2 :: OpB s N2 a a -> BVar s n r a -> BVar s n r a -> BVar s n r a
- liftB3 :: OpB s N3 a a -> BVar s n r a -> BVar s n r a -> BVar s n r a -> BVar s n r a
- op1 :: Num a => (forall s. AD s (Forward a) -> AD s (Forward a)) -> Op N1 a a
- op2 :: Num a => (forall s. Reifies s Tape => Reverse s a -> Reverse s a -> Reverse s a) -> Op N2 a a
- op3 :: Num a => (forall s. Reifies s Tape => Reverse s a -> Reverse s a -> Reverse s a -> Reverse s a) -> Op N3 a a
- opN :: (Num a, Known Nat n) => (forall s. Reifies s Tape => Vec n (Reverse s a) -> Reverse s a) -> Op n a a
- composeOp :: forall m n o a b c. (Monad m, Num a, Known Nat n) => VecT o (OpM m n a) b -> OpM m o b c -> OpM m n a c
- composeOp1 :: forall m n a b c. (Monad m, Num a, Known Nat n) => OpM m n a b -> OpM m N1 b c -> OpM m n a c
- (~.) :: forall m n a b c. (Monad m, Num a, Known Nat n) => OpM m N1 b c -> OpM m n a b -> OpM m n a c
- op1' :: (a -> (b, Maybe b -> a)) -> Op N1 a b
- op2' :: (a -> a -> (b, Maybe b -> (a, a))) -> Op N2 a b
- op3' :: (a -> a -> a -> (b, Maybe b -> (a, a, a))) -> Op N3 a b
- pattern (:+) :: forall a n. a -> Vec n a -> Vec (S n) a
- (*:) :: f a -> f a -> VecT k (S (S Z)) f a
- (+:) :: a -> a -> Vec (S (S Z)) a
- head' :: VecT k (S n) f a -> f a
- type N0 = Z
- type N1 = S N0
- type N2 = S N1
- type N3 = S N2
- type N4 = S N3
- type N5 = S N4
- type N6 = S N5
- type N7 = S N6
- type N8 = S N7
- type N9 = S N8
- type N10 = S N9
Types
Backprop types
type BP s n r = BP s (Replicate n r) Source #
A Monad allowing you to explicitly build hetereogeneous data dependency graphs and that the library can perform back-propagation on.
A
is a BP
s n r aBP
action that uses an environment n
values of
type r
, and returns an a
. When "run", it will compute a gradient that
is a vector (Vec
) of n
r
s. (The phantom parameter s
is used to
ensure that any BVar
s aren't leaked out of the monad)
Note that you can only "run" a
that produces a BP
s n rBVar
--
that is, things of the form
BP
s n r (BVar
n r a)
The above is a BP
action that returns a BVar
containing an a
.
When this is run, it'll produce a result of type a
and a gradient of
that is a vector of n
values of type r
. (This form has a type
synonym, BPOp
, for convenience)
For example,
is a monad that represents
a computation with three BP
s N3
DoubleDouble
s as inputs. And, if you ran a
BP
sN3
Double (BVar
N3 Double Int)
Or, using the BPOp
type synonym:
BPOp
sN3
Double Int
with backprop
or gradBPOp
, it'll return a gradient on the inputs (a
vector of three Double
s) and produce a value of type Int
.
Now, one powerful thing about this type is that a BP
is itself an
Op
(or more precisely, an OpM
). So, once you create your fancy BP
computation, you can transform it into an OpM
using bpOp
.
type BPOp s n r a = BP s n r (BVar s n r a) Source #
A handy type synonym representing a BP
action that returns a BVar
.
This is handy because this is the form of BP
actions that
backprop
and gradBPOp
(etc.) expects.
A value of type:
BPOp
s n r a
is an action that takes an input environment of n
values of type r
and produces a BVar
containing a value of type a
. Because it
returns a BVar
, the library can track the data dependencies between
the BVar
and the input environment and perform back-propagation.
See documentation for BP
for an explanation of the phantom type
parameter s
.
type BPOpI s n r a = VecT n (BVar s n r) r -> BVar s n r a Source #
An "implicit" operation on BVar
s that can be backpropagated.
A value of type:
BPOpI
s n r a
takes a vector (Vec
) of n
of BVar
s containg r
s and uses them to (purely)
produce a BVar
containing an a
.
foo :: BPOpI s N2
Double Double
foo (x :* y :* ØV) = x + sqrt y
If you are exclusively doing implicit back-propagation by combining
BVar
s and using BPOpI
s, you are probably better off just importing
Numeric.Backprop.Mono.Implicit, which provides better tools. This
type synonym exists in Numeric.Backprop.Mono just for the implicitly
function, which can convert "implicit" backprop functions like
a
into an "explicit" graph backprop function, a BPOpI
s rs a
.BPOp
s rs a
type BVar s n a = BVar s (Replicate n a) Source #
The basic unit of manipulation inside BP
(or inside an
implicit-graph backprop function). Instead of directly working with
values, you work with BVar
s contating those values. When you work
with a BVar
, the backprop library can keep track of what values
refer to which other values, and so can perform back-propagation to
compute gradients.
A
refers to a value of type BVar
s n r aa
, with an environment
of n
values of type r
. The phantom parameter s
is used to
ensure that stray BVar
s don't leak outside of the backprop process.
(That is, if you're using implicit backprop, it ensures that you interact
with BVar
s in a polymorphic way. And, if you're using explicit
backprop, it ensures that a
never leaves the BVar
s n r a
that it was created in.)BP
s n r
BVar
s have Num
, Fractional
, Floating
, etc. instances, so they
can be manipulated using polymorphic functions and numeric functions in
Haskell. You can add them, subtract them, etc., in "implicit" backprop
style.
(However, note that if you directly manipulate BVar
s using those
instances or using liftB
, it delays evaluation, so every usage site
has to re-compute the result/create a new node. If you want to re-use
a BVar
you created using +
or -
or liftB
, use
bindVar
to force it first. See documentation for
bindVar
for more details.)
type Op n a b = Op (Replicate n a) b Source #
An
describes a differentiable function from Op
n a bn
values of
type a
to a value of type b
.
For example, a value of type
Op
N2
Int Double
is a function that takes two Int
s and returns a Double
.
It can be differentiated to give a gradient of two Int
s, if given
a total derivative for the Double
. Mathematically, it is akin to a:
\[ f : \mathbb{Z}^2 \rightarrow \mathbb{R} \]
See runOp
, gradOp
, and gradOpWith
for examples on how to run it,
and Op
for instructions on creating it.
This type is abstracted over using the pattern synonym with constructor
Op
, so you can create one from scratch with it. However, it's
simplest to create it using op2'
, op1'
, op2'
, and op3'
helper
smart constructors And, if your function is a numeric function, they
can even be created automatically using op1
, op2
, op3
, and opN
with a little help from Numeric.AD from the ad library.
Note that this type is a subset or subtype of OpM
(and also of
OpB
). So, if a function ever expects an
(or a OpM
m as aOpB
), you can always provide an
instead.Op
as a
Many functions in this library will expect an
(or
an OpM
m as a
), and in all of these cases, you can
provide an OpB
s as a
.Op
as a
type OpB s n a b = OpB s (Replicate n a) b Source #
A subclass of OpM
(and superclass of Op
),
representing Op
s that the backprop library uses to perform
backpropation.
An
OpB
s n a b
represents a differentiable function that takes a n
values of type a
produces an a b
, which can be run on
s and also inside
BVar
s
s. For example, an BP
s
takes two OpB
s N2
Double BoolDouble
s
and produces a Bool
, and does it in a differentiable way.
OpB
is a superset of Op
, so, if you see any function that expects
an OpB
(like opVar'
and ~$
, for
example), you can give them an Op
, as well.
You can think of OpB
as a superclass/parent class of Op
in this
sense, and of Op
as a subclass of OpB
.
Vectors
A VecT
is a fixed-length list of a given type. It's basically the
"monomorphic" version of a Prod
(see the mini-tutorial in
Numeric.Backprop).
A
is a list of VecT
n f an
f a
s, and is constructed by consing
them together with :*
(using 'ØV' as nil):
I
"hello":*
I "world" :* I "ok" :* ØV ::VecT
N3
I
String [1,2,3] :* [4,5,6,7] :* ØV ::VecT
N2
[] Int
(I
is the identity functor)
So, in general:
x :: f a y :: f a z :: f a k :: f a x :* y :* z :* k :* ØV ::VecT
fN4
a
Vec
is provided as a convenient type synonym for VecT
I
, and has
a convenient pattern synonym :+
, which can also be used for pattern
matching:
x :: a y :: a z :: a k :: a x::<
y ::< z ::< k ::< ØV ::Vec
N4
a
data VecT k n f a :: forall k. N -> (k -> *) -> k -> * where #
Functor1 l l (VecT l n) | |
Foldable1 l l (VecT l n) | |
Traversable1 l l (VecT l n) | |
Witness ØC (Known N Nat n) (VecT k n f a) | |
(Monad f, Known N Nat n) => Monad (VecT * n f) | |
Functor f => Functor (VecT * n f) | |
(Applicative f, Known N Nat n) => Applicative (VecT * n f) | |
Foldable f => Foldable (VecT * n f) | |
Traversable f => Traversable (VecT * n f) | |
Eq (f a) => Eq (VecT k n f a) | |
(Num (f a), Known N Nat n) => Num (VecT k n f a) | |
Ord (f a) => Ord (VecT k n f a) | |
Show (f a) => Show (VecT k n f a) | |
type WitnessC ØC (Known N Nat n) (VecT k n f a) | |
BP
Backprop
backprop :: forall n r a. Num r => (forall s. BPOp s n r a) -> Vec n r -> (a, Vec n r) Source #
Perform back-propagation on the given BPOp
. Returns the result of
the operation it represents, as well as the gradient of the result with
respect to its inputs. See module header for Numeric.Backprop.Mono
and package documentation for examples and usages.
evalBPOp :: forall n r a. (forall s. BPOp s n r a) -> Vec n r -> a Source #
Simply run the BPOp
on an input vector, getting the result without
bothering with the gradient or with back-propagation.
gradBPOp :: forall n r a. Num r => (forall s. BPOp s n r a) -> Vec n r -> Vec n r Source #
Run the BPOp
on an input vector and return the gradient of the result
with respect to the input vector
Utility combinators
withInps :: Known Nat n => (VecT n (BVar s n r) r -> BP s n r a) -> BP s n r a Source #
Runs a continuation on a Vec
of all of the input BVar
s.
Handy for bringing the environment into scope and doing stuff with it:
foo ::BPOp
N2
Double Int foo =withInps
$ \(x :* y :* ØV) -> do -- do stuff with inputs
Looks kinda like foo (x :* y *+ ØV) = -- ...
, don't it?
Note that the above is the same as
foo ::BPOp
N2
Double Int foo = do caseinpVars
of x :* y :* ØV -> do -- do stuff with inputs
But just a little nicer!
implicitly :: Known Nat n => BPOpI s n r a -> BPOp s n r a Source #
Convert a BPOpI
into a BPOp
. That is, convert a function on
a bundle of BVar
s (generating an implicit graph) into a fully fledged
BPOp
that you can run backprop
on. See BPOpI
for more
information.
If you are going to write exclusively using implicit BVar
operations,
it might be more convenient to use Numeric.Backprop.Mono.Implicit
instead, which is geared around that use case.
Vars
inpVar :: Fin n -> BVar s n r r Source #
Create a BVar
given an index (Fin
) into the input environment. For an
example,
inpVar
FZ
would refer to the first input variable, Bool]@), and
inpVar
(FS
FZ
)
Would refer to the second input variable.
Typically, there shouldn't be any reason to use inpVar
directly. It's
cleaner to get all of your input BVar
s together using withInps
or
inpVars
.
inpVars :: Known Nat n => VecT n (BVar s n r) r Source #
Get a VecT
(vector) of BVar
s for all of the input environment
(the n
r
s) of the BP
s n r
For example, if your BP
has two Double
s inside its input
environment (a
), this would return two BP
s N2
DoubleBVar
s,
pointing to each input Double
.
case (inpVars
::VecT
N2
(BVar
sN2
Double) Double) of x :* y :* ØV -> do -- the first item, x, is a var to the first input x ::BVar
s N2 Double -- the second item, y, is a var to the second input y ::BVar
s N2 Double
bpOp :: forall s n r a. (Num r, Known Nat n) => BPOp s n r a -> OpB s n r a Source #
Turn a BPOp
into an OpB
. Basically converts a BP
taking n
r
s and producing an a
into an Op
taking an n
r
s and returning
an a
, with all of the powers and utility of an Op
, including all of
its gradient-finding glory.
Really just reveals the fact that any
is itself an BPOp
s rs aOp
,
an
, which makes it a differentiable function.OpB
s rs a
Handy because an OpB
can be used with almost all of
the Op
-related functions in this moduel, including opVar
, ~$
, etc.
bindVar :: forall s n r a. Num a => BVar s n r a -> BP s n r (BVar s n r a) Source #
Concretizes a delayed BVar
. If you build up a BVar
using numeric
functions like +
or *
or using liftB
, it'll defer the evaluation,
and all of its usage sites will create a separate graph node.
Use bindVar
if you ever intend to use a BVar
in more than one
location.
-- bad errSquared :: Num a =>BP
s N2 a a errSquared =withInp
$ \(x :* y :* Ø) -> do let err = r - treturn
(err * err) -- err is used twice! -- good errSquared :: Num a =>BP
s N2 a a errSquared =withInp
$ \(x :* y :* Ø) -> do let err = r - t e <-bindVar
err -- force e, so that it's safe to use twice!return
(e * e) -- better errSquared :: Num a =>BP
s N2 a a errSquared =withInp
$ \(x :* y :* Ø) -> do let err = r - t e <-bindVar
errbindVar
(e * e) -- result is forced so user doesn't have to worry
Note the relation to opVar
~$
liftB
/ .$
:
opVar
o xs =bindVar
(liftB
o xs) o~$
xs =bindVar
(o.$
xs)op2
(*)~$
(x :< y :< Ø) =bindVar
(x * y)
So you can avoid bindVar
altogether if you use the explicitly binding
~$
and opVar
etc.
Note that bindVar
on BVar
s that are already forced is a no-op.
From Ops
opVar :: forall s m n r a b. Num b => OpB s m a b -> VecT m (BVar s n r) a -> BP s n r (BVar s n r b) Source #
Apply an OpB
to a VecT
(vector) of BVar
s.
If you had an
, this function will expect a vector of of
three OpB
s N3 a b
s, and the result will be a BVar
s n r a
:BVar
s n r b
myOp ::OpB
s N3 a b x ::BVar
s n r a y ::BVar
s n r a z ::BVar
s n r a x:*
y :* z :* 'ØV' ::VecT
N3 (BVar
s n r) aopVar
myOp (x :* y :* z :* ØV) ::BP
s n r (BVar
s n r b)
Note that OpB
is a superclass of Op
, so you can provide any Op
here, as well (like those created by op1
, op2
, constOp
, op0
etc.)
opVar
has an infix alias, ~$
, so the above example can also be
written as:
myOp~$
(x :* y :* z :* ØV) ::BP
s n r (BVar
s n r b)
to let you pretend that you're applying the myOp
function to three
inputs.
Also note the relation between opVar
and liftB
and bindVar
:
opVar
o xs =bindVar
(liftB
o xs)
(~$) :: forall s m n r a b. Num b => OpB s m a b -> VecT m (BVar s n r) a -> BP s n r (BVar s n r b) infixr 5 Source #
Infix synonym for opVar
, which lets you pretend that you're applying
OpB
s as if they were functions:
myOp ::OpB
s N3 a b x ::BVar
s n r a y ::BVar
s n r a z ::BVar
s n r a x:*
y :* z :* 'ØV' ::VecT
N3 (BVar
s n r) a myOp~$
(x :* y :* z :* ØV) ::BP
s n r (BVar
s n r b)
Note that OpB
is a superclass of Op
, so you can pass in any Op
here, as well (like those created by op1
, op2
, constOp
, op0
etc.)
~$
can also be thought of as a "binding" version of .$
:
o~$
xs =bindVar
(o.$
xs)
opVar1 :: forall s n r a b. Num b => OpB s N1 a b -> BVar s n r a -> BP s n r (BVar s n r b) Source #
Convenient wrapper over opVar
that takes an OpB
with one argument
and a single BVar
argument. Lets you not have to type out the entire
VecT
.
opVar1
o x =opVar
o (x:*
'ØV') myOp ::Op
N2 a b x ::BVar
s n r aopVar1
myOp x ::BP
s n r (BVar
s n r b)
Note that OpB
is a superclass of Op
, so you can pass in an Op
here
(like one made with op1
) as well.
opVar2 :: forall s n r a b. Num b => OpB s N2 a b -> BVar s n r a -> BVar s n r a -> BP s n r (BVar s n r b) Source #
Convenient wrapper over opVar
that takes an OpB
with two arguments
and two BVar
arguments. Lets you not have to type out the entire
VecT
.
opVar2
o x y =opVar
o (x:*
y:*
'ØV') myOp ::Op
N2 a b x ::BVar
s n r a y ::BVar
s n r bopVar2
myOp x y ::BP
s n r (BVar
s n r b)
Note that OpB
is a superclass of Op
, so you can pass in an Op
here
(like one made with op2
) as well.
opVar3 :: forall s n r a b. Num b => OpB s N3 a b -> BVar s n r a -> BVar s n r a -> BVar s n r a -> BP s n r (BVar s n r b) Source #
Convenient wrapper over opVar
that takes an OpB
with three arguments
and three BVar
arguments. Lets you not have to type out the entire
VecT
.
opVar3
o x y z =opVar
o (x:*
y:*
z:*
'ØV') myOp ::Op
N3 a b x ::BVar
s n r a y ::BVar
s n r a z ::BVar
s n r aopVar3
myOp x y z ::BP
s n r (BVar
s n r b)
Note that OpB
is a superclass of Op
, so you can pass in an Op
here
(like one made with op3
) as well.
(-$) :: forall s m n r a b. (Num a, Num b, Known Nat m) => BPOp s m a b -> VecT m (BVar s n r) a -> BP s n r (BVar s n r b) infixr 5 Source #
Lets you treat a
as an BPOp
s n a b
, and "apply"
arguments to it just like you would with an Op
n a bOp
and ~$
/ opVar
.
Basically a convenient wrapper over bpOp
and ~$
:
o-$
xs = bpOp o~$
xs
So for a
, you can "plug in" BPOp
s n a bBVar
s to each a
, and
get a b
as a result.
Useful for running a
that you got from a different function, and
"plugging in" its BPOp
s n a ba
inputs with BVar
s from your current
environment.
Combining
liftB :: forall s m n a b r. OpB s m a b -> VecT m (BVar s n r) a -> BVar s n r b Source #
Apply OpB
over a VecT
of BVar
s, as inputs. Provides "implicit"
back-propagation, with deferred evaluation.
If you had an
, this function will expect a vector of of
three OpB
s N3 a b
s, and the result will be a BVar
s n r a
:BVar
s n r b
myOp ::OpB
s N3 a b x ::BVar
s n r a y ::BVar
s n r a z ::BVar
s n r a x:*
y :* z :* 'ØV' ::VecT
N3 (BVar
s n r) aliftB
myOp (x :* y :* z :* ØV) ::BVar
s n r b
Note that OpB
is a superclass of Op
, so you can provide any Op
here, as well (like those created by op1
, op2
, constOp
, op0
etc.)
liftB
has an infix alias, .$
, so the above example can also be
written as:
myOp.$
(x :* y :* z :* ØV) ::BVar
s n r b
to let you pretend that you're applying the myOp
function to three
inputs.
The result is a new deferred BVar
. This should be fine in most
cases, unless you use the result in more than one location. This will
cause evaluation to be duplicated and multiple redundant graph nodes to
be created. If you need to use it in two locations, you should use
opVar
instead of liftB
, or use bindVar
:
opVar
o xs =bindVar
(liftB
o xs)
liftB
can be thought of as a "deferred evaluation" version of opVar
.
(.$) :: forall s m n a b r. OpB s m a b -> VecT m (BVar s n r) a -> BVar s n r b Source #
Infix synonym for liftB
, which lets you pretend that you're applying
OpB
s as if they were functions:
myOp ::OpB
s N3 a b x ::BVar
s n r a y ::BVar
s n r a z ::BVar
s n r a x:*
y :* z :* 'ØV' ::VecT
N3 (BVar
s n r) a myOp.$
(x :* y :* z :* ØV) ::BVar
s n r b
Note that OpB
is a superclass of Op
, so you can pass in any Op
here, as well (like those created by op1
, op2
, constOp
, op0
etc.)
See the documentation for liftB
for all the caveats of this usage.
.$
can also be thought of as a "deferred evaluation" version of ~$
:
o~$
xs =bindVar
(o.$
xs)
liftB1 :: OpB s N1 a a -> BVar s n r a -> BVar s n r a Source #
Convenient wrapper over liftB
that takes an OpB
with one argument
and a single BVar
argument. Lets you not have to type out the entire
VecT
.
liftB1
o x =liftB
o (x:*
'ØV') myOp ::Op
N2 a b x ::BVar
s n r aliftB1
myOp x ::BVar
s n r b
Note that OpB
is a superclass of Op
, so you can pass in an Op
here
(like one made with op1
) as well.
See the documentation for liftB
for caveats and potential problematic
situations with this.
liftB2 :: OpB s N2 a a -> BVar s n r a -> BVar s n r a -> BVar s n r a Source #
Convenient wrapper over liftB
that takes an OpB
with two arguments
and two BVar
arguments. Lets you not have to type out the entire
VecT
.
liftB2
o x y =liftB
o (x:*
y:*
'ØV') myOp ::Op
N2 a b x ::BVar
s n r a y ::BVar
s n r bliftB2
myOp x y ::BVar
s n r b
Note that OpB
is a superclass of Op
, so you can pass in an Op
here
(like one made with op2
) as well.
See the documentation for liftB
for caveats and potential problematic
situations with this.
liftB3 :: OpB s N3 a a -> BVar s n r a -> BVar s n r a -> BVar s n r a -> BVar s n r a Source #
Convenient wrapper over liftB
that takes an OpB
with three arguments
and three BVar
arguments. Lets you not have to type out the entire
Prod
.
liftB3
o x y z =liftB
o (x:*
y:*
z:*
'ØV') myOp ::Op
N3 a b x ::BVar
s n r a y ::BVar
s n r b z ::BVar
s n r bliftB3
myOp x y z ::BVar
s n r b
Note that OpB
is a superclass of Op
, so you can pass in an Op
here
(like one made with op3
) as well.
See the documentation for liftB
for caveats and potential problematic
situations with this.
Op
op2 :: Num a => (forall s. Reifies s Tape => Reverse s a -> Reverse s a -> Reverse s a) -> Op N2 a a Source #
op3 :: Num a => (forall s. Reifies s Tape => Reverse s a -> Reverse s a -> Reverse s a -> Reverse s a) -> Op N3 a a Source #
opN :: (Num a, Known Nat n) => (forall s. Reifies s Tape => Vec n (Reverse s a) -> Reverse s a) -> Op n a a Source #
composeOp :: forall m n o a b c. (Monad m, Num a, Known Nat n) => VecT o (OpM m n a) b -> OpM m o b c -> OpM m n a c Source #
composeOp1 :: forall m n a b c. (Monad m, Num a, Known Nat n) => OpM m n a b -> OpM m N1 b c -> OpM m n a c Source #
(~.) :: forall m n a b c. (Monad m, Num a, Known Nat n) => OpM m N1 b c -> OpM m n a b -> OpM m n a c infixr 9 Source #
op1' :: (a -> (b, Maybe b -> a)) -> Op N1 a b Source #
Create an Op
of a function taking one input, by giving its explicit
derivative. The function should return a tuple containing the result of
the function, and also a function taking the derivative of the result
and return the derivative of the input.
If we have
\[ \eqalign{ f &: \mathbb{R} \rightarrow \mathbb{R}\cr y &= f(x)\cr z &= g(y) } \]
Then the derivative \( \frac{dz}{dx} \), it would be:
\[ \frac{dz}{dx} = \frac{dz}{dy} \frac{dy}{dx} \]
If our Op
represents \(f\), then the second item in the resulting
tuple should be a function that takes \(\frac{dz}{dy}\) and returns
\(\frac{dz}{dx}\).
If the input is Nothing
, then \(\frac{dz}{dy}\) should be taken to be
\(1\).
As an example, here is an Op
that squares its input:
square :: Num a =>Op
N1
a a square =op1'
$ \x -> (x*x, \case Nothing -> 2 * x Just d -> 2 * d * x )
Remember that, generally, end users shouldn't directly construct Op
s;
they should be provided by libraries or generated automatically.
For numeric functions, single-input Op
s can be generated automatically
using op1
.
op2' :: (a -> a -> (b, Maybe b -> (a, a))) -> Op N2 a b Source #
Create an Op
of a function taking two inputs, by giving its explicit
gradient. The function should return a tuple containing the result of
the function, and also a function taking the derivative of the result
and return the derivative of the input.
If we have
\[ \eqalign{ f &: \mathbb{R}^2 \rightarrow \mathbb{R}\cr z &= f(x, y)\cr k &= g(z) } \]
Then the gradient \( \left< \frac{\partial k}{\partial x}, \frac{\partial k}{\partial y} \right> \) would be:
\[ \left< \frac{\partial k}{\partial x}, \frac{\partial k}{\partial y} \right> = \left< \frac{dk}{dz} \frac{\partial z}{dx}, \frac{dk}{dz} \frac{\partial z}{dy} \right> \]
If our Op
represents \(f\), then the second item in the resulting
tuple should be a function that takes \(\frac{dk}{dz}\) and returns
\( \left< \frac{\partial k}{dx}, \frac{\partial k}{dx} \right> \).
If the input is Nothing
, then \(\frac{dk}{dz}\) should be taken to be
\(1\).
As an example, here is an Op
that multiplies its inputs:
mul :: Num a =>Op
N2
a a mul =op2'
$ \x y -> (x*y, \case Nothing -> (y , x ) Just d -> (d*y, x*d) )
Remember that, generally, end users shouldn't directly construct Op
s;
they should be provided by libraries or generated automatically.
For numeric functions, two-input Op
s can be generated automatically
using op2
.