ےɝ      !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ (c) Justin Le 2017BSD3 justin@jle.im experimental non-portableNone %&+,DORT[ #An # s a# encodes an isomorphism between an s and an a". It basically lets you go from s -> a and back (from a -> s=) while preserving structure. You can basically imagine an # s a to be an (s -> a, a -> s) tuple.*You can get the "forward" direction of an # with :  :: Iso'' s a -> (s -> a) #And the "backwards" direction with &: & :: Iso'' s a -> (a -> s) You can construct an # using %., giving the forward and backwards functions:myIso :: Iso' (Identity a) a$ myIso = iso runIdentity Identityview myIso (Identity "hello")"hello"review myIso "hello"Identity "hello"One powerful thing about #s is that they're  composable using : () :: # c b -> # b a -> # c a GThis is basically provided here so that this package doesn't incurr a lens) dependecy, but if you already depend on lens#, you should use the version from Control.Lens.Iso instead.$A family of isomorphisms. See #.% Construct an $= by giving the "forward" and "backward" direction functions:myIso :: Iso' (Identity a) a$ myIso = iso runIdentity Identityview myIso (Identity "hello")"hello"review myIso "hello"Identity "hello"GThis is basically provided here so that this package doesn't incurr a lens) dependecy, but if you already depend on lens#, you should use the version from Control.Lens.Iso instead.&-Get the "reverse" direction function from an $.GThis is basically provided here so that this package doesn't incurr a lens) dependecy, but if you already depend on lens#, you should use the version from Control.Lens.Review instead.' A useful $8 between two types with the same runtime representation.(An $` between a type that is a product type, and a tuple that contains all of its components. Uses  Generics.SOP and the  typeclass.$import qualified Generics.SOP as SOP+data Foo = A Int Bool deriving Genericinstance SOP.Generic Fooview gTuple (A 10 True)10 ::< True ::< "review gTuple (15 ::< False ::< ) A 15 False)An $A between a sum type whose constructors are products, and a sum () of products ( ). Uses  Generics.SOP and the  typeclass.$import qualified Generics.SOP as SOP'data Bar = A Int Bool | B String Doubleinstance SOP.Generic Bar'view' 'gSOP' (A 10 True)'InL' (10 ::< True ::< )'view' 'gSOP' (B "hello" 3.4)%'InR' ('InL' ("hello" ::< 3.4 ::< )).'review' 'gTuple' ('InL' (15 ::< False ::< )) A 15 False7'review' 'gTuple' ('InR' ('InL' ("bye" ::< 9.8 ::< ))) B "bye" 9.8*An iso between a single-type  and the single type.+/An iso between a single type and a single-type ., Reverse an #q. The forward function becomes the backwards function, and the backwards function becomes the forward function.GThis is basically provided here so that this package doesn't incurr a lens) dependecy, but if you already depend on lens#, you should use the version from Control.Lens.Review instead.#$%&'()*+,#$%&'()*+,$#%,&'()*+#$%&'()*+,None*+,:DOQRT[af-- n a is a list of a s repeated n times.:kind! Replicate N3 Int'[Int, Int, Int]:kind! Replicate N5 Double)'[Double, Double, Double, Double, Double]-     -    -     (c) Justin Le 2017BSD3 justin@jle.im experimental non-portableNone$%&+,9:;DOT[bf=.Helper wrapper used for the implementation of 5..An . as a* describes a differentiable function from as to a.For example, a value of type . '[Int, Bool] Double is a function from an  and a , returning a '. It can be differentiated to give a gradient of an  and a & if given a total derivative for the Double. If we call  2), then, mathematically, it is akin to a:2 f : \mathbb{Z} \times 2 \rightarrow \mathbb{R} See 9, A, and ?% for examples on how to run it, and .! for instructions on creating it.IThis type is abstracted over using the pattern synonym with constructor .Z, so you can create one from scratch with it. However, it's simplest to create it using K, J, K, and Lx helper smart constructors And, if your function is a numeric function, they can even be created automatically using M, N, O, and P with a little help from  Numeric.AD from the ad library.Note that this type is a subset or subtype of / (and also of  &). So, if a function ever expects an / m as a (or a  ), you can always provide an . as a instead..Many functions in this library will expect an / m as a (or an   s as a2), and in all of these cases, you can provide an . as a./An / m as a represents a differentiable (monadic) function from as to a, in the context of a  m.For example, an / IO '[Int, Bool] Double "would be a function that takes an  and a  and returns a  (in '). It can be differentiated to give a gradient of an  and a  (also in )) if given the total derivative for the Double. Note that an / is a  superclass of .#, so any function that expects an / m as a can also accept an . as a.See ;, B, and @ for examples on how to run it.0 Construct an / by giving a (monadic) function creating the result, and also a continuation on how to create the gradient, given the total derivative of a.!See the module documentation for Numeric.Backprop.Op= for more details on the function that this constructor and . expect.1 Construct an . by giving a function creating the result, and also a continuation on how to create the gradient, given the total derivative of a.!See the module documentation for Numeric.Backprop.Op= for more details on the function that this constructor and / expect.2A combination of ; and > . Given an /( and inputs, returns the result of the /- and a continuation that gives its gradient.[The continuation takes the total derivative of the result as input. See documenation for > and module documentation for Numeric.Backprop.Op for more information.3A combination of 9 and = . Given an .( and inputs, returns the result of the .- and a continuation that gives its gradient.[The continuation takes the total derivative of the result as input. See documenation for = and module documentation for Numeric.Backprop.Op for more information.4 A version of 5 taking explicit <, indicating the number of inputs expected and their types.Requiring an explicit  is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the the expected input tuple. If you ever actually explicitly write down as5 as a list of types, you should be able to just use 5.5Compose /s together, similar to . But, because all /s are #\mathbb{R}^N \rightarrow \mathbb{R}, this is more like  for functions, or liftAN.That is, given an / m as b1, an / m as b2 , and an / m as b3, it can compose them with an / m '[b1,b2,b3] c to create an / m as c.6 A version of 7 taking explicit <, indicating the number of inputs expected and their types.Requiring an explicit  is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the the expected input tuple. If you ever actually explicitly write down as5 as a list of types, you should be able to just use 7.7Convenient wrapper over 5J for the case where the second function only takes one input, so the two /,s can be directly piped together, like for .8'Convenient infix synonym for (flipped) 7. Meant to be used just like : M negate :: . '[a] a N. (+) :: Op '[a,a] a op1 negate 8 op2 (+) :: Op '[a, a] a 9Run the function that an . encodes, to get the result.runOp (op2 (*)) (3 ::< 5 ::< )15:Run the function that an .X encodes, to get the resulting output and also its gradient with respect to the inputs.AgradOpM' (op2 (*)) (3 ::< 5 ::< ) :: IO (Int, Tuple '[Int, Int])(15, 5 ::< 3 ::< );The monadic version of 9, for /s.*runOpM (op2 (*)) (3 ::< 5 ::< ) :: IO Int15<The monadic version of :, for /s.=A combination of A and ?N. The third argument is (optionally) the total derivative the result. Give w and it is assumed that the result is the final result (and the total derivative is 1), and this behaves the same as A. Give  d and it uses the d? as the total derivative of the result, and this behaves like ?.See A" and the module documentaiton for Numeric.Backprop.Op for more information.>The monadic version of =, for /s.?Run the function that an . encodes, and get the gradient of a "final result" with respect to the inputs, given the total derivative of the output with the final result.See A" and the module documentaiton for Numeric.Backprop.Op for more information.@The monadic version of ?, for /s.ARun the function that an .I encodes, and get the gradient of the output with respect to the inputs. gradOp (op2 (*)) (3 ::< 5 ::< ) 5 ::< 3 ::< -- the gradient of x*y is (y, x)BThe monadic version of A, for /s.CAn . that coerces an item into another item whose type has the same runtime representation. Requires the input to be an instance of .4gradOp' opCoerce (Identity 5) :: (Int, Identity Int)(5, Identity 1) C = F ' D A version of E taking explicit <, indicating the number of inputs expected and their types.Requiring an explicit  is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the the expected input tuple. If you ever actually explicitly write down as5 as a list of types, you should be able to just use E.EAn . that takes as% and returns exactly the input tuple.#gradOp' opTup (1 ::< 2 ::< 3 ::< )*(1 ::< 2 ::< 3 ::< , 1 ::< 1 ::< 1 ::< )FAn .C that runs the input value through the isomorphism encoded in the $+. Requires the input to be an instance of .yWarning: This is unsafe! It assumes that the isomorphisms themselves have derivative 1, so will break for things like  =. Basically, don't use this for any "numeric" isomorphisms.G A version of H taking explicit 3, indicating the number of inputs and their types.Requiring an explicit  is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the the expected input tuple. If you ever actually explicitly write down as5 as a list of types, you should be able to just use H.HAn .D that ignores all of its inputs and returns a given constant value.*gradOp' (opConst 10) (1 ::< 2 ::< 3 ::< )(10, 0 ::< 0 ::< 0 ::< )I Create an .: that takes no inputs and always returns the given value.'There is no gradient, of course (using AQ will give you an empty tuple), because there is no input to have a gradient of.gradOp' (op0 10) (10, )For a constant .& that takes input and ignores it, see H and G."Note that because this returns an .4, it can be used with any function that expects an / or   , as well.J Create an . of a function taking one input, by giving its explicit derivative. The function should return a tuple containing the result of the function, and also a function taking the derivative of the result and return the derivative of the input. If we haveT \eqalign{ f &: \mathbb{R} \rightarrow \mathbb{R}\cr y &= f(x)\cr z &= g(y) } Then the derivative  \frac{dz}{dx} , it would be:/ \frac{dz}{dx} = \frac{dz}{dy} \frac{dy}{dx} If our . represents fO, then the second item in the resulting tuple should be a function that takes  \frac{dz}{dy} and returns  \frac{dz}{dx}.If the input is , then  \frac{dz}{dy} should be taken to be 1.As an example, here is an . that squares its input: square :: Num a => . '[a] a square = Ju $ \x -> (x*x, \case Nothing -> 2 * x Just d -> 2 * d * x ) ARemember that, generally, end users shouldn't directly construct .Ds; they should be provided by libraries or generated automatically.$For numeric functions, single-input .(s can be generated automatically using M.K Create an . of a function taking two inputs, by giving its explicit gradient. The function should return a tuple containing the result of the function, and also a function taking the derivative of the result and return the derivative of the input. If we haveY \eqalign{ f &: \mathbb{R}^2 \rightarrow \mathbb{R}\cr z &= f(x, y)\cr k &= g(z) } Then the gradient M \left< \frac{\partial k}{\partial x}, \frac{\partial k}{\partial y} \right>  would be: \left< \frac{\partial k}{\partial x}, \frac{\partial k}{\partial y} \right> = \left< \frac{dk}{dz} \frac{\partial z}{dx}, \frac{dk}{dz} \frac{\partial z}{dy} \right> If our . represents fO, then the second item in the resulting tuple should be a function that takes  \frac{dk}{dz} and returns = \left< \frac{\partial k}{dx}, \frac{\partial k}{dx} \right> .If the input is , then  \frac{dk}{dz} should be taken to be 1.As an example, here is an . that multiplies its inputs: mul :: Num a => . '[a, a] a mul = K{ $ \x y -> (x*y, \case Nothing -> (y , x ) Just d -> (d*y, x*d) ) ARemember that, generally, end users shouldn't directly construct .Ds; they should be provided by libraries or generated automatically.!For numeric functions, two-input .(s can be generated automatically using N.L Create an .] of a function taking three inputs, by giving its explicit gradient. See documentation for K for more details.MAutomatically create an .5 of a numerical function taking one argument. Uses V, and so can take any numerical function polymorphic over the standard numeric types.(gradOp' (op1 (recip . negate)) (5 ::< )(-0.2, 0.04 ::< )NAutomatically create an .6 of a numerical function taking two arguments. Uses V, and so can take any numerical function polymorphic over the standard numeric types.2gradOp' (op2 (\x y -> x * sqrt y)) (3 ::< 4 ::< )(6.0, 2.0 ::< 0.75 ::< )OAutomatically create an .8 of a numerical function taking three arguments. Uses V, and so can take any numerical function polymorphic over the standard numeric types.?gradOp' (op3 (\x y z -> (x * sqrt y)**z)) (3 ::< 4 ::< 2 ::< )%(36.0, 24.0 ::< 9.0 ::< 64.503 ::< )PAutomatically create an .; of a numerical function taking multiple arguments. Uses V, and so can take any numerical function polymorphic over the standard numeric types.<gradOp' (opN (\(x :+ y :+ ) -> x * sqrt y)) (3 ::< 4 ::< )(6.0, 2.0 ::< 0.75 ::< )QOptimized version of M (Q).ROptimized version of M (R).SOptimized version of M (S).TOptimized version of M (T).UOptimized version of M (U).VOptimized version of M V.WOptimized version of M W.XOptimized version of M X.YOptimized version of M Y.ZOptimized version of M Z.[Optimized version of M [.\Optimized version of M \.]Optimized version of N ].^Optimized version of M ^._Optimized version of M _.`Optimized version of M `.aOptimized version of M a.bOptimized version of M b.cOptimized version of M c.dOptimized version of M d.eOptimized version of M e.fOptimized version of M f.gOptimized version of M g.hOptimized version of M h.iOptimized version of M i.B./012/ to runInputs1Result, and continuation to get the gradient3. to runInputs1Result, and continuation to get the gradient4 of / s taking as and returning different b in bs/ taking eac of the bs from the input . Composed /5 of / s taking as and returning different b in bs/ taking eac of the bs from the input . Composed /6789:;<=. to runInputs to run it withIf 8, taken as the total derivative of the result. If 3, assumes that the result is the final result. The gradient>/ to runInputs to run it withIf 8, taken as the total derivative of the result. If 3, assumes that the result is the final result. The gradient?. to runInputs to run it with"The total derivative of the result The gradient@/ to runInputs to run it with"The total derivative of the result the gradientABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklI -./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghiI.1/09A:?=3;B<@>257846IHGMNOP-JKLCEFD QRSVXWTYZ[\U]^_`abcdefghi?./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijkl89  (c) Justin Le 2017BSD3 justin@jle.im experimental non-portableNone%&+,29:;<=DIOQRT[fEssentially a "single-usage"  . It's a stateful node, but only ever has a single consumer (and so its total derivative comes from a single partial derivative). Used when keeping track of !s. BA (stateful) node in the graph of operations/data dependencies in n that the library uses. m4s can refer to these to get results from them, and ";s can refer to these to get partial derivatives from them."Used exclusively by #` to specify "where" and "how" to look for partial derivatives at usage sites of a given entity.$The entity is used in a  , and as an Nth input%The entity is used in a , and as an Nth input&7The entity is used somehow in the terminal result of a n@, and so therefore has a fixed partial derivative contribution.m&The basic unit of manipulation inside nl (or inside an implicit-graph backprop function). Instead of directly working with values, you work with m1s contating those values. When you work with a m, the backprop library can keep track of what values refer to which other values, and so can perform back-propagation to compute gradients.A m s rs a refers to a value of type a., with an environment of values of the types rs. The phantom parameter s is used to ensure that stray m-s don't leak outside of the backprop process.P(That is, if you're using implicit backprop, it ensures that you interact with mTs in a polymorphic way. And, if you're using explicit backprop, it ensures that a m s rs a never leaves the n s rs that it was created in.)ms have , ', (, etc. instances, so they can be manipulated using polymorphic functions and numeric functions in Haskell. You can add them, subtract them, etc., in "implicit" backprop style./(However, note that if you directly manipulate m"s using those instances or using  w, it delays evaluation, so every usage site has to re-compute the result/create a new node. If you want to re-use a m you created using ) or * or  , use , to force it first. See documentation for  for more details.)+A BVar referring to a  ,1A BVar referring to an environment input variable-7A constant BVar that refers to a specific Haskell value!?A BVar that combines several other BVars using a function (an .#). Essentially a branch of a tree.nA Monad allowing you to explicitly build hetereogeneous data dependency graphs and that the library can perform back-propagation on.A n s rs a is a n$ action that uses an environment of rs returning a a>. When "run", it will compute a gradient that is a tuple of rs. (The phantom parameter s is used to ensure that any m"s aren't leaked out of the monad)Note that you can only "run" a n s rs that produces a m -- that is, things of the form n s rs (m s rs a) The above is a n action that returns a m containing an a4. When this is run, it'll produce a result of type a' and a gradient of that is a tuple of rs#. (This form has a type synonym, , for convenience)For example, a n s '[ Int, Double, Double ]3 is a monad that represents a computation with an , , and  as inputs. And, if you ran a n s '[ Int, Double, Double ] (m$ s '[ Int, Double, Double ] Double) Or, using the BPOp type synonym: # s '[ Int, Double, Double ] Double with  or *, it'll return a gradient on the inputs (, , and ) and produce a value of type .2Now, one powerful thing about this type is that a n is itself an . (or more precisely, an  , which is a subtype of /#). So, once you create your fancy n, computation, you can transform it into an / using ..The "state" of a nN action, which keeps track of what nodes, if any, refer to any of the inputs.# Reference to  usage sites? for a given entity, used to get partial or total derivatives./ A list of "Js pointing to places that use the entity, to provide partial derivatives.0MThe entity is the terminal result of a BP, so its total derivative is fixed.oA subclass of / (and superclass of .), representing . s that the backprop' library uses to perform backpropation.An o s rs a ;represents a differentiable function that takes a tuple of rs and produces an a a, which can be run on m ss and also inside n ss. For example, an o s '[ Int, Double ] Bool takes an  and a  and produces a &, and does it in a differentiable way.o is a superset of ./, so, if you see any function that expects an o (like  and %, for example), you can give them an . , as well.You can think of o! as a superclass/parent class of . in this sense, and of . as a subclass of o.1 Combines two / lists. If either input is an 0o, then throws away the other result and keeps the new terminal forced total derivative. (Biases to the left)2ETraversal (fake prism) to refer to the list of internal refs if the  ForwardRef) isn't associated with a terminal entity.3 See note for  instance.4 See note for  instance.5Note that if you use the  instance to create ms, the resulting m is deferred/delayed. At every location you use it, it will be recomputed, and a separate graph node will be created. If you are using a m you made with the & instance in multiple locations, use . first to force it and prevent recomputation.-6789: ;<=>?"$%&m+,-!n@A.BC#/0o1DEFGHIJKL2345)6789: ;<=>?"$%&m+,-!n@A.BC#/0oDEFGHIJKL26789: ;<=>?"$%&m+,-!n@A.BC#/0o1DEFGHIJKL2345(c) Justin Le 2017BSD3 justin@jle.im experimental non-portableNone!"%&+,:DOQRT[bf%pAn "implicit" operation on m/s that can be backpropagated. A value of type: p s rs a takes a bunch of m s containg rs& and uses them to (purely) produce a m containing an a. Ifoo :: BPOpI s '[ Double, Double ] Double foo (x :< y :< ) = x + sqrt y EIf you are exclusively doing implicit back-propagation by combining m s and using p/s, you are probably better off just importing Numeric.Backprop.Implicit=, which provides better tools. This type synonym exists in Numeric.Backprop just for the C function, which can convert "implicit" backprop functions like a p s rs a0 into an "explicit" graph backprop function, a q s rs a.q$A handy type synonym representing a n action that returns a m-. This is handy because this is the form of n actions that  and  (etc.) expects.A value of type: q s rs a 0is an action that takes an input environment of rs and produces a m containing a value of type a. Because it returns a m;, the library can track the data dependencies between the m9 and the input environment and perform back-propagation.See documentation for n3 for an explanation of the phantom type parameter s.r Apply an o to a  (tupling) of ms.If you had an o s '[a, b, c] d+, this function will expect a 3-Prod of a m s rs a, a m s rs b, and a m s rs c, and the result will be a m s rs d: myOp :: o s '[a, b, c] d x :: m s rs a y :: m s rs b z :: m* s rs c x :< y :< z :< ::  (m s rs) '[a, b, c] r myOp (x :< y :< z :< ) :: n s rs (m s rs d)  Note that o is a superclass of ., so you can provide any .' here, as well (like those created by M, N, constOp, I etc.)r has an infix alias, }/, so the above example can also be written as: myOp } (x :< y :< z :< ) :: n s rs (m s rs d) ,to let you pretend that you're applying the myOp function to three inputs.Also note the relation between r and  and : r o xs =  ( o xs) r- can be thought of as a "binding" version of .s Split out a m of a tuple into a tuple () of ms. <-- the environment is a single Int-Bool tuple, tup stuff :: n% s '[ Tuple '[Int, Bool] ] a stuff = ( $ \(tup :< ) -> do i :< b :< <- s tup -- now, i is a m pointing to the  inside tup -- and b is a m pointing to the : inside tup -- you can do stuff with the i and b here  Note that s = t M tUse an $ (or compatible ^ from the lens library) to "pull out" the parts of a data type and work with each part as a m.%If there is an isomorphism between a b and a  as (that is, if an a$ is just a container for a bunch of as#), then it lets you break out the as inside and work with those. !data Foo = F Int Bool fooIso :: ## Foo (Tuple '[Int, Bool]) fooIso = %W (\(F i b) -> i ::< b ::< ) (\(i ::< b ::< ) -> F i b ) t fooIso :: m rs Foo -> n s rs ( (m s rs) '[Int, Bool]) stuff :: n s '[Foo] a stuff = O $ \(foo :< ) -> do i :< b :< <- partsVar fooIso foo -- now, i is a m pointing to the  inside foo -- and b is a m pointing to the : inside foo -- you can do stuff with the i and b here BYou can use this to pass in product types as the environment to a n=, and then break out the type into its constituent products.Note that for a type like Foo, fooIso& can be generated automatically with  from  GHC.Generics and  from  Generics.SOP and  generics-sop , using the ( iso. See w for more information.2Also, if you are literally passing a tuple (like n s '[Tuple '[Int, Bool]2) then you can give in the identity isomorphism (M ) or use s.uA useful infix alias for t.Building on the example from t: !data Foo = F Int Bool fooIso :: ## Foo (Tuple '[Int, Bool]) fooIso = %` (\(F i b) -> i ::< b ::< ) (\(i ::< b ::< ) -> F i b ) stuff :: n s '[Foo] a stuff = / $ \(foo :< ) -> do i :< b :< <- fooIso u foo -- now, i is a m pointing to the  inside foo -- and b is a m pointing to the : inside foo -- you can do stuff with the i and b here See wG for an example usage of splitting up an arbitrary product type (like Foo) using  GHC.Geneics and  Generics.SOP.v A continuation-based version of t. Instead of binding the parts and using it in the rest of the block, provide a continuation to handle do stuff with the parts inside.Building on the example from t: !data Foo = F Int Bool fooIso :: ## Foo (Tuple '[Int, Bool]) fooIso = %` (\(F i b) -> i ::< b ::< ) (\(i ::< b ::< ) -> F i b ) stuff :: n s '[Foo] a stuff =  $ \(foo :< ) -> do v8 fooIso foo $ \(i :< b :< ) -> do -- now, i is a m pointing to the  inside foo -- and b is a m pointing to the < inside foo -- you can do stuff with the i and b here Useful so that you can work with the internal parts of the data type in a closure, so the parts don't leak out to the rest of your n'. But, mostly just a stylistic choice.wUsing  from  GHC.Generics and  from  Generics.SOP, split a m* containing a product type into a tuple () of m!s pointing to each value inside.Building on the example from t: jimport qualified Generics.SOP as SOP data Foo = F Int Bool deriving Generic instance SOP.Generic Foo w :: m rs Foo -> n s rs ( (m s rs) '[Int, Bool]) stuff :: n s '[Foo] a stuff = ( $ \(foo :< ) -> do i :< b :< <- w foo -- now, i is a m pointing to the  inside foo -- and b is a m pointing to the : inside foo -- you can do stuff with the i and b here Because Foo is a straight up product type, w can use  GHC.Generics and take out the items inside.Note that because w = s ( Then, you can also use ( with u:  stuff :: n s '[Foo] a stuff = ( $ \(foo :< ) -> do i :< b :< <- ( u foo -- now, i is a m pointing to the  inside foo -- and b is a m pointing to the : inside foo -- you can do stuff with the i and b here xUse an $ (or compatible n from the lens library) to "pull out" the different constructors of a sum type and return a (choice) sum of m s that you can pattern match on.%If there is an isomorphism between a b and a   as (that is, if an a& is just a sum type for every type in as), then it lets you branch) on which constructor is used inside the b.+Essentially implements pattern matching on m values. 0data Bar = A Int | B Bool | C String barIso :: # Bar (" I '[Int, Bool, String]) barIso = % (\case A i -> $ (I i) B b ->  (% (I b)) C s ->  ( (/ (I s)) ) (\case . (I i) -> A i  (+ (I b)) -> B b  ( ( (I s))) -> C s ) choicesVar barIso :: BVar rs Bar -> BP s rs (Sum I (BVar s rs) '[Int, Bool, String]) stuff :: n s '[Bar] a stuff =  $ \(bar :< ) -> do c <- x barIso bar case c of r i -> do -- in this branch, bar was made with the A constructor -- i is the Int inside it  (t b) -> do -- in this branch, bar was made with the B constructor -- b is the Bool inside it  ( (q s)) -> do -- in this branch, bar was made with the B constructor -- s is the String inside it >You can use this to pass in sum types as the environment to a n@, and then branch on which constructor the value was made with.See Numeric.Backprop#sum for a mini-tutorial on .y A continuation-based version of x. Instead of binding the parts and using it in the rest of the block, provide a continuation that will handle every possible constructor/case of the type of the value the m points to.Building on the example from x: 0data Bar = A Int | B Bool | C String barIso :: # Bar (" I '[Int, Bool, String]) barIso = % (\case A i -> $ (I i) B b ->  (% (I b)) C s ->  ( (/ (I s)) ) (\case . (I i) -> A i  (+ (I b)) -> B b  ( (" (I s))) -> C s ) xU barIso :: BVar rs Bar -> BP s rs (Sum I (BVar s rs) '[Int, Bool, String]) stuff :: n s '[Bar] a stuff =  $ \(bar :< ) -> do y barIso bar $ case r i -> do -- in this branch, bar was made with the A constructor -- i is the Int inside it  (t b) -> do -- in this branch, bar was made with the B constructor -- b is the Bool inside it  ( (q s)) -> do -- in this branch, bar was made with the B constructor -- s is the String inside it  Nicer than x directly, because you don't have to give the result a superfluous name before pattern matching on it. You can just directly pattern match in the lambda, so there's a lot less syntactical noise.zA useful infix alias for x.Building on the example from x: 0data Bar = A Int | B Bool | C String barIso :: # Bar (" I '[Int, Bool, String]) barIso = % (\case A i -> $ (I i) B b ->  (% (I b)) C s ->  ( (/ (I s)) ) (\case . (I i) -> A i  (+ (I b)) -> B b  ( (+ (I s))) -> C s ) stuff :: n s '[Bar] a stuff = % $ \(bar :< ) -> do c <- barIso z bar case c of r i -> do -- in this branch, bar was made with the A constructor -- i is the Int inside it  (t b) -> do -- in this branch, bar was made with the B constructor -- b is the Bool inside it  ( (q s)) -> do -- in this branch, bar was made with the B constructor -- s is the String inside it {A combination of t and x@, that lets you split a type into a sum of products. Using an $ (or compatible f from the lens library), you can pull out a type that is a sum of products into a sum of products of ms.;Implements branching on the constructors of a value that a mP contains, and also splitting out the different items inside each constructor. @data Baz = A Int Bool | B String Double bazIso :: # Baz ( 0 '[ '[Int, Bool], '[String, Double] ]) bazIso = % (\case A i b -> 4 (I (i ::< b ::< )) B s d ->  (= (I (s ::< d ::< ))) ) (\case  (I (i :: b ::< )) - A i b  ( (I (s :: d ::< ))) - B s d ) { bazIso :: m rs Baz -> n s rs ( ( (m8 s rs)) '[ '[Int, Bool], '[String, Double] ]) stuff :: n s '[Baz] a stuff =  $ \(baz :< ) -> do c <- { barIso baz case c of  (i : b:< ) -} do -- in this branch, baz was made with the A constructor -- i and b are the Int and Bool inside it  ( (s : d:< )) -| do -- in this branch, baz was made with the B constructor -- s and d are the String and Double inside it oEssentially exists to implement "pattern matching" on multiple constructors and fields for the value inside a m.Note that for a type like Baz, bazIso& can be generated automatically with  from  GHC.Generics and  from  Generics.SOP and  generics-sop, with ). See | for more information.See Numeric.Backprop#sum for a mini-tutorial on .|Using  from  GHC.Generics and  from  Generics.SOP, split a mC containing a sum of products (any simple ADT, essentialy) into a 0 of each constructor, each containing a tuple () of m!s pointing to each value inside.Building on the example from {: import qualified Generics.SOP as SOP data Baz = A Int Bool | B String Double deriving Generic instance SOP.Generic Baz | :: m rs Baz -> n s rs ( ( (m8 s rs)) '[ '[Int, Bool], '[String, Double] ]) stuff :: n s '[Baz] a stuff = > $ \(baz :< ) -> do c <- gSplits baz case c of  (i : b:< ) -} do -- in this branch, baz was made with the A constructor -- i and b are the Int and Bool inside it  ( (s : d:< )) -| do -- in this branch, baz was made with the B constructor -- s and d are the String and Double inside it Because Foo( is a straight up sum-of-products type, | can use  GHC.Generics and take out the items inside.Note: w = s ) See Numeric.Backprop#sum for a mini-tutorial on .}Infix synonym for r/, which lets you pretend that you're applying os as if they were functions: myOp :: o s '[a, b, c] d x :: m s rs a y :: m s rs b z :: m' s rs c x :< y :< z :< ::  (m s rs) '[a, b, c] myOp } (x :< y :< z :< ) :: n s rs (m s rs d)  Note that o is a superclass of ., so you can pass in any .' here, as well (like those created by M, N, constOp, I etc.)}2 can also be thought of as a "binding" version of : o } xs =  (o  xs) ~Lets you treat a q s as b as an . as b;, and "apply" arguments to it just like you would with an . and } / r.$Basically a convenient wrapper over  and }: o ~ xs = bpOp o } xs  So for a q s as b, you can "plug in" ms to as , and get a b as a result.Useful for running a q s as b@ that you got from a different function, and "plugging in" its as inputs with m!s from your current environment. Create a mJ that represents just a specific value, that doesn't depend on any other ms.Convenient wrapper over r that takes an o! with one argument and a single m6 argument. Lets you not have to type out the entire .  o x = r o (x  '') myOp :: . '[a] b x :: m s rs a  myOp x :: n s rs (m s rs b)  Note that o is a superclass of ., so you can pass in an . here (like one made with M ) as well.Convenient wrapper over r that takes an o with two arguments and two m7 arguments. Lets you not have to type out the entire .  o x y = r o (x  y  '') myOp :: . '[a, b] c x :: m s rs a y :: m s rs b  myOp x y :: n s rs (m s rs c)  Note that o is a superclass of ., so you can pass in an . here (like one made with N ) as well.Convenient wrapper over r that takes an o! with three arguments and three m7 arguments. Lets you not have to type out the entire .  o x y z = r o (x  y  z  '') myOp :: . '[a, b, c] d x :: m s rs a y :: m s rs b z :: m s rs c  myOp x y z :: n s rs (m s rs d)  Note that o is a superclass of ., so you can pass in an . here (like one made with O ) as well.Concretizes a delayed m. If you build up a m using numeric functions like ) or N or using \, it'll defer the evaluation, and all of its usage sites will create a separate graph node.Use  if you ever intend to use a m in more than one location. -- bad errSquared :: Num a => n s '[a, a] a errSquared = withInp0 $ \(r :< t :< ) -> do let err = r - t OE (err * err) -- err is used twice! -- good errSquared :: Num a => n s '[a, a] a errSquared = 5 $ \(r :< t :< ) -> do let err = r - t e <- 9 err -- force e, so that it's safe to use twice! O+ (e * e) -- better errSquared :: Num a => n s '[a, a] a errSquared = 5 $ \(r :< t :< ) -> do let err = r - t e <-  err @ (e * e) -- result is forced so user doesn't have to worry Note the relation to r  }   / : r o xs =  ( o xs) o } xs =  (o  xs) N (*) } (x :< y :< ) =  (x * y) So you can avoid / altogether if you use the explicitly binding } and r etc. Note that  on m%s that are already forced is a no-op.&Perform back-propagation on the given q. Returns the result of the operation it represents, as well as the gradient of the result with respect to its inputs. See module header for Numeric.Backprop4 and package documentation for examples and usages.Turn a q into an o. Basically converts a n taking an rs and producing an a into an . taking an rs and returning an a,, with all of the powers and utility of an ./, including all of its gradient-finding glory.&Really just reveals the fact that any q s rs a is itself an ., an o s rs a+, which makes it a differentiable function.Handy because an o% can be used with almost all of the .--related functions in this moduel, including r, }, etc.Simply run the qe on an input tuple, getting the result without bothering with the gradient or with back-propagation.Run the qZ on an input tuple and return the gradient of the result with respect to the input tuple. A version of  taking explicit <, indicating the number of inputs required and their types.Requiring an explicit  is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the list of inputs. If you ever actually explicitly write down rs5 as a list of types, you should be able to just use . Convert a p into a q/. That is, convert a function on a bundle of m7s (generating an implicit graph) into a fully fledged q that you can run  on. See p for more information.5If you are going to write exclusively using implicit m1 operations, it might be more convenient to use Numeric.Backprop.Implicit0 instead, which is geared around that use case. Create a m= given an index into the input environment. For an example,  P would refer to the first input variable (the  in a n s '[Int, Bool]), and  (Q P) Would refer to the second input variable (the  in a n s '[Int, Bool])0Typically, there shouldn't be any reason to use 3 directly. It's cleaner to get all of your input ms together using  or .Get a  (tupling) of m%s for all of the input environment (rs ) of the n s rsFor example, if your n has an  and  in its input environment (a n s '[Int, Double]), this would return a m pointing to the  and a m pointing to the . case ( ::  (mj s '[Int, Double]) '[Int, Double]) of x :< y :< -> do -- the first item, x, is a var to the input  -- x :: mG s '[Int, Double] Int -- the second item, y, is a var to the input  -- y :: m s '[Int, Double] Double  A version of  taking explicit <, indicating the number of inputs required and their types.Mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the list of inputs. If you ever actually explicitly write down rs5 as a list of types, you should be able to just use . A version of  taking explicit <, indicating the number of inputs required and their types.Mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the list of inputs. If you ever actually explicitly write down rs5 as a list of types, you should be able to just use .Runs a continuation on a  of all of the input ms.FHandy for bringing the environment into scope and doing stuff with it: foo :: q '[Double, Int] a foo = 4 $ \(x :< y :< ) -> do -- do stuff with inputs Looks kinda like foo (x :< y :< ) = -- ... , don't it?"Note that the above is the same as foo :: q$ '[Double, Int] a foo = do case  of x : y:< -$ do -- do stuff with inputs But just a little nicer!Apply o over a  of mTs, as inputs. Provides "implicit-graph" back-propagation, with deferred evaluation.If you had an o s '[a, b, c] d+, this function will expect a 3-Prod of a m s rs a, a m s rs b, and a m s rs c, and the result will be a m s rs d: myOp :: o s '[a, b, c] d x :: m s rs a y :: m s rs b z :: m* s rs c x :< y :< z :< ::  (m s rs) '[a, b, c]  myOp (x :< y :< z :< ) :: m s rs d  Note that o is a superclass of ., so you can provide any .' here, as well (like those created by M, N, constOp, I etc.) has an infix alias, /, so the above example can also be written as: myOp  (x :< y :< z :< ) :: m s rs d ,to let you pretend that you're applying the myOp function to three inputs.The result is a new deferred m. This should be fine in most cases, unless you use the result in more than one location. This will cause evaluation to be duplicated and multiple redundant graph nodes to be created. If you need to use it in two locations, you should use r instead of  , or use : r o xs =  ( o xs) 9 can be thought of as a "deferred evaluation" version of r.Infix synonym for /, which lets you pretend that you're applying os as if they were functions: myOp :: o s '[a, b, c] d x :: m s rs a y :: m s rs b z :: m' s rs c x :< y :< z :< ::  (m s rs) '[a, b, c] myOp  (x :< y :< z :< ) :: m s rs d  Note that o is a superclass of ., so you can pass in any .' here, as well (like those created by M, N, constOp, I etc.)See the documentation for # for all the caveats of this usage.> can also be thought of as a "deferred evaluation" version of }: o } xs =  (o  xs) Convenient wrapper over  that takes an o! with one argument and a single m6 argument. Lets you not have to type out the entire .  o x =  o (x  '') myOp :: . '[a] b x :: m s rs a  myOp x :: m s rs b  Note that o is a superclass of ., so you can pass in an . here (like one made with M ) as well.See the documentation for = for caveats and potential problematic situations with this.Convenient wrapper over  that takes an o with two arguments and two m7 arguments. Lets you not have to type out the entire .  o x y =  o (x  y  '') myOp :: . '[a, b] c x :: m s rs a y :: m s rs b  myOp x y :: m s rs c  Note that o is a superclass of ., so you can pass in an . here (like one made with N ) as well.See the documentation for = for caveats and potential problematic situations with this.Convenient wrapper over  that takes an o! with three arguments and three m7 arguments. Lets you not have to type out the entire .  o x y z =  o (x  y  z  '') myOp :: . '[a, b, c] d x :: m s rs a y :: m s rs b z :: m s rs c  myOp x y z :: m s rs d  Note that o is a superclass of ., so you can pass in an . here (like one made with O ) as well.See the documentation for = for caveats and potential problematic situations with this.RApply a function to the contents of an STRef, and cache the results using the given lens. If already calculated, simply returned the cached result.*pqrstuvwxyz{|ST}~Uq to runinputoutputq to differentiate'inputgradientVWR\ ().578JKLMNOPQRSTUVWXYZ[\]^_`abcdefghimnopqrstuvwxyz{|}~\nqpm.or}~tuvsw(xzy{|)MNOP578JKL QRSVXWTYZ[\U]^_`abcdefghi*pqrstuvwxyz{|ST}~UVWRu1z1}5~55(c) Justin Le 2017BSD3 justin@jle.im experimental non-portableNone ,:DOQRTbf An operation on m.s that can be backpropagated. A value of type:  rs a takes a bunch of m s containg rs& and uses them to (purely) produce a m containing an a. foo :: # '[ Double, Double ] Double foo (x  y  '') = x + sqrt y  here is related to 1 from the normal explicit-graph backprop module Numeric.Backprop.Run back-propagation on a ^ function, getting both the result and the gradient of the result with respect to the inputs. foo :: T '[Double, Double] Double foo (x :< y :< ) = let z = x * sqrt y in z + x ** y 'backprop' foo (2 ::< 3 ::< )(11.46, 13.73 ::< 6.12 ::< )Run the Z on an input tuple and return the gradient of the result with respect to the input tuple. foo :: T '[Double, Double] Double foo (x :< y :< ) = let z = x * sqrt y in z + x ** y grad foo (2 ::< 3 ::< )13.73 ::< 6.12 ::< Simply run the e on an input tuple, getting the result without bothering with the gradient or with back-propagation. foo :: T '[Double, Double] Double foo (x :< y :< ) = let z = x * sqrt y in z + x ** y eval foo (2 ::< 3 ::< )11.46 A version of  taking explicit E, indicating the number of items in the input tuple and their types.Requiring an explicit  is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the internal tuple. If you ever actually explicitly write down bs5 as a list of types, you should be able to just use .Use an $ (or compatible ^ from the lens library) to "pull out" the parts of a data type and work with each part as a m.%If there is an isomorphism between a b and a  as (that is, if an a$ is just a container for a bunch of as#), then it lets you break out the as inside and work with those. !data Foo = F Int Bool fooIso :: ## Foo (Tuple '[Int, Bool]) fooIso = %W (\(F i b) -> i ::< b ::< ) (\(i ::< b ::< ) -> F i b )  fooIso :: m rs Foo ->  (m s rs) '[Int, Bool] stuff :: ( s '[Foo] a stuff (foo :< ) = case  fooIso foo of i : b:< - -- now, i is a m pointing to the " inside foo -- and b is a m pointing to the > inside foo -- you can do stuff with the i and b here BYou can use this to pass in product types as the environment to a n=, and then break out the type into its constituent products.Note that for a type like Foo, fooIso& can be generated automatically with  from  GHC.Generics and  from  Generics.SOP and  generics-sop , using the ( iso. See  for more information.2Also, if you are literally passing a tuple (like n s '[Tuple '[Int, Bool]2) then you can give in the identity isomorphism (M ) or use .At the moment, this implicit & is less efficient than the explicit ', but this might change in the future. A version of  taking explicit ;, indicating the number of internal items and their types.Requiring an explicit  is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the internal tuple. If you ever actually explicitly write down bs5 as a list of types, you should be able to just use . A continuation-based version of . Instead of binding the parts and using it in the rest of the block, provide a continuation to handle do stuff with the parts inside.Building on the example from : !data Foo = F Int Bool fooIso :: ## Foo (Tuple '[Int, Bool]) fooIso = %` (\(F i b) -> i ::< b ::< ) (\(i ::< b ::< ) -> F i b ) stuff ::  s '[Foo] a stuff (foo :< ) = < fooIso foo $ \case i :< b :< -> -- now, i is a m pointing to the  inside foo -- and b is a m pointing to the < inside foo -- you can do stuff with the i and b here 'Mostly just a stylistic alternative to . A version of  taking explicit ;, indicating the number of internal items and their types.Requiring an explicit  is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the internal tuple. If you ever actually explicitly write down as5 as a list of types, you should be able to just use . Split out a m of a tuple into a tuple () of ms. <-- the environment is a single Int-Bool tuple, tup stuff :: 7 s '[ Tuple '[Int, Bool] ] a stuff (tup :< ) = case splitVar tup of i :< b :< <-  tup -- now, i is a m pointing to the  inside tup -- and b is a m pointing to the : inside tup -- you can do stuff with the i and b here  Note that  =  M  A version of  taking explicit ;, indicating the number of internal items and their types.Requiring an explicit  is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the internal tuple. If you ever actually explicitly write down as5 as a list of types, you should be able to just use .Using  from  GHC.Generics and  from  Generics.SOP, split a m* containing a product type into a tuple () of m!s pointing to each value inside.Building on the example from : jimport qualified Generics.SOP as SOP data Foo = F Int Bool deriving Generic instance SOP.Generic Foo  :: m rs Foo ->  (m s rs) '[Int, Bool] stuff :: ( s '[Foo] a stuff (foo :< ) = case  foo of i : b:< - -- now, i is a m pointing to the " inside foo -- and b is a m pointing to the > inside foo -- you can do stuff with the i and b here Because Foo is a straight up product type,  can use  GHC.Generics and take out the items inside. Note that  =  ( B (.JKLMNOPQRSTUVWXYZ[\]^_`abcdefghimoBm.o(MNOPJKL QRSVXWTYZ[\U]^_`abcdefghi (c) Justin Le 2017BSD3 justin@jle.im experimental non-portableNone $%&,:OT[bf8An  m n a b6 represents a differentiable (monadic) function from n values of type a to a value of type b.For example, an  IO  Int Double #would be a function that takes two s and returns a  (in '). It can be differentiated to give a gradient of the two input  s (also in $) if given the total derivative for a. Note that an  is a  superclass of #, so any function that expects an  m as a can also accept an  as a.See , , and  for examples on how to run it.An  n a b* describes a differentiable function from n values of type a to a value of type b.For example, a value of type   Int Double is a function that takes two s and returns a &. It can be differentiated to give a gradient of two (s, if given a total derivative for the #. Mathematically, it is akin to a:+ f : \mathbb{Z}^2 \rightarrow \mathbb{R} See , , and % for examples on how to run it, and ! for instructions on creating it.IThis type is abstracted over using the pattern synonym with constructor Z, so you can create one from scratch with it. However, it's simplest to create it using , , , and x helper smart constructors And, if your function is a numeric function, they can even be created automatically using , , , and  with a little help from  Numeric.AD from the ad library.Note that this type is a subset or subtype of  (and also of  &). So, if a function ever expects an  m as a (or a  ), you can always provide an  as a instead..Many functions in this library will expect an  m as a (or an   s as a2), and in all of these cases, you can provide an  as a. Construct an  by giving a (monadic) function creating the result, and also a continuation on how to create the gradient, given the total derivative of a.!See the module documentation for Numeric.Backprop.Op= for more details on the function that this constructor and  expect. Construct an  by giving a function creating the result, and also a continuation on how to create the gradient, given the total derivative of a.!See the module documentation for Numeric.Backprop.Op= for more details on the function that this constructor and  expect. Create an : that takes no inputs and always returns the given value.'There is no gradient, of course (using R will give you an empty vector), because there is no input to have a gradient of.gradOp' (op0 10) V(10, V)For a constant & that takes input and ignores it, see ."Note that because this returns an 4, it can be used with any function that expects an  or   , as well. A version of  taking explicit X,, indicating the number of inputs required.Requiring an explicit X is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the length of the the expected input vector. If you ever actually explicitly write down the size n!, you should be able to just use .An D that ignores all of its inputs and returns a given constant value.(gradOp' (opConst 10) (1 :+ 2 :+ 3 :+ V)(10, 0 :+ 0 :+ 0 :+ V)Automatically create an 5 of a numerical function taking one argument. Uses V, and so can take any numerical function polymorphic over the standard numeric types.(gradOp' (op1 (recip . negate)) (5 :+ V)(-0.2, 0.04 :+ V)Automatically create an 6 of a numerical function taking two arguments. Uses V, and so can take any numerical function polymorphic over the standard numeric types.1gradOp' (op2 (\x y -> x * sqrt y)) (3 :+ 4 :+ V)(6.0, 2.0 :+ 0.75 :+ V)Automatically create an 8 of a numerical function taking three arguments. Uses V, and so can take any numerical function polymorphic over the standard numeric types.=gradOp' (op3 (\x y z -> (x * sqrt y)**z)) (3 :+ 4 :+ 2 :+ V)#(36.0, 24.0 :+ 9.0 :+ 64.503 :+ V)Automatically create an ; of a numerical function taking multiple arguments. Uses V, and so can take any numerical function polymorphic over the standard numeric types.;gradOp' (opN (\(x :+ y :+ ) -> x * sqrt y)) (3 :+ 4 :+ V)(6.0, 2.0 :+ 0.75 :+ V) Create an  of a function taking one input, by giving its explicit derivative. The function should return a tuple containing the result of the function, and also a function taking the derivative of the result and return the derivative of the input. If we haveT \eqalign{ f &: \mathbb{R} \rightarrow \mathbb{R}\cr y &= f(x)\cr z &= g(y) } Then the derivative  \frac{dz}{dx} , it would be:/ \frac{dz}{dx} = \frac{dz}{dy} \frac{dy}{dx} If our  represents fO, then the second item in the resulting tuple should be a function that takes  \frac{dz}{dy} and returns  \frac{dz}{dx}.If the input is , then  \frac{dz}{dy} should be taken to be 1.As an example, here is an  that squares its input: square :: Num a =>   a a square = u $ \x -> (x*x, \case Nothing -> 2 * x Just d -> 2 * d * x ) ARemember that, generally, end users shouldn't directly construct Ds; they should be provided by libraries or generated automatically.$For numeric functions, single-input (s can be generated automatically using . Create an  of a function taking two inputs, by giving its explicit gradient. The function should return a tuple containing the result of the function, and also a function taking the derivative of the result and return the derivative of the input. If we haveY \eqalign{ f &: \mathbb{R}^2 \rightarrow \mathbb{R}\cr z &= f(x, y)\cr k &= g(z) } Then the gradient M \left< \frac{\partial k}{\partial x}, \frac{\partial k}{\partial y} \right>  would be: \left< \frac{\partial k}{\partial x}, \frac{\partial k}{\partial y} \right> = \left< \frac{dk}{dz} \frac{\partial z}{dx}, \frac{dk}{dz} \frac{\partial z}{dy} \right> If our  represents fO, then the second item in the resulting tuple should be a function that takes  \frac{dk}{dz} and returns = \left< \frac{\partial k}{dx}, \frac{\partial k}{dx} \right> .If the input is , then  \frac{dk}{dz} should be taken to be 1.As an example, here is an  that multiplies its inputs: mul :: Num a =>   a a mul = { $ \x y -> (x*y, \case Nothing -> (y , x ) Just d -> (d*y, x*d) ) ARemember that, generally, end users shouldn't directly construct Ds; they should be provided by libraries or generated automatically.!For numeric functions, two-input (s can be generated automatically using . Create an ] of a function taking three inputs, by giving its explicit gradient. See documentation for  for more details.A combination of  and  . Given an ( and inputs, returns the result of the - and a continuation that gives its gradient.[The continuation takes the total derivative of the result as input. See documenation for  and module documentation for Numeric.Backprop.Op for more information.Run the function that an  encodes, to get the result.runOp (op2 (*)) (3 :+ 5 :+ )15A combination of  and N. The third argument is (optionally) the total derivative the result. Give w and it is assumed that the result is the final result (and the total derivative is 1), and this behaves the same as . Give  d and it uses the d? as the total derivative of the result, and this behaves like .See " and the module documentaiton for Numeric.Backprop.Op for more information.Run the function that an  encodes, and get the gradient of a "final result" with respect to the inputs, given the total derivative of the output with the final result.See " and the module documentaiton for Numeric.Backprop.Op for more information.Run the function that an I encodes, and get the gradient of the output with respect to the inputs.gradOp (op2 (*)) (3 :+ 5 :+ V) 5 :+ 3 :+ V -- the gradient of x*y is (y, x)Run the function that an X encodes, to get the resulting output and also its gradient with respect to the inputs.9gradOpM' (op2 (*)) (3 :+ 5 :+ V) :: IO (Int, Vec N2 Int)(15, 5 :+ 3 :+ V)The monadic version of , for s.)runOpM (op2 (*)) (3 :+ 5 :+ V) :: IO Int15The monadic version of , for s.)runOpM (op2 (*)) (3 :+ 5 :+ V) :: IO Int15The monadic version of , for s.The monadic version of , for s.The monadic version of , for s.The monadic version of , for s. A version of  taking explicit X9, indicating the number of inputs expected in the first sRequiring an explicit X is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the length of the the expected input vector. If you ever actually explicitly write down the size n!, you should be able to just use .Compose s together, similar to . But, because all s are #\mathbb{R}^N \rightarrow \mathbb{R}, this is more like  for functions, or liftAN.That is, given an o of  m n a b s, it can compose them with an  m o b c to create an  m o a c.Convenient wrappver over J for the case where the second function only takes one input, so the two ,s can be directly piped together, like for .Convenient wrappver over J for the case where the second function only takes one input, so the two ,s can be directly piped together, like for .'Convenient infix synonym for (flipped) . Meant to be used just like :  negate ::  '[a] a . (+) :: Op '[a,a] a op1 negate  op2 (+) :: Op '[a, a] a Optimized version of  (Y).Optimized version of  (*).Optimized version of  (Z).Optimized version of  ([).Optimized version of  (\).Optimized version of  ].Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .Optimized version of  .8O !"-O- !"89 (c) Justin Le 2017BSD3 justin@jle.im experimental non-portableNone%&+,:DOQRT[bfA subclass of  (and superclass of ), representing  s that the backprop( library uses to perform backpropation.An  s n a b 2represents a differentiable function that takes a n values of type a produces an a b, which can be run on  ss and also inside  ss. For example, an  s  Double Bool takes two s and produces a &, and does it in a differentiable way. is a superset of /, so, if you see any function that expects an  (like  and &, for example), you can give them an  , as well.You can think of ! as a superclass/parent class of  in this sense, and of  as a subclass of .An "implicit" operation on /s that can be backpropagated. A value of type:  s n r a takes a vector () of n of  s containg r's and uses them to (purely) produce a  containing an a. foo :: BPOpI s / Double Double foo (x :* y :* V) = x + sqrt y EIf you are exclusively doing implicit back-propagation by combining  s and using /s, you are probably better off just importing Numeric.Backprop.Mono.Implicit=, which provides better tools. This type synonym exists in Numeric.Backprop.Mono just for the D function, which can convert "implicit" backprop functions like a  s rs a/ into an "explicit" graph backprop function, a  s rs a.$A handy type synonym representing a  action that returns a -. This is handy because this is the form of  actions that  and  (etc.) expects.A value of type:  s n r a 0is an action that takes an input environment of n values of type r and produces a  containing a value of type a. Because it returns a ;, the library can track the data dependencies between the 8 and the input environment and perform back-propagation.See documentation for 3 for an explanation of the phantom type parameter s.&The basic unit of manipulation inside l (or inside an implicit-graph backprop function). Instead of directly working with values, you work with 1s contating those values. When you work with a , the backprop library can keep track of what values refer to which other values, and so can perform back-propagation to compute gradients.A  s n r a refers to a value of type a, with an environment of n values of type r. The phantom parameter s is used to ensure that stray -s don't leak outside of the backprop process.P(That is, if you're using implicit backprop, it ensures that you interact with Ts in a polymorphic way. And, if you're using explicit backprop, it ensures that a  s n r a never leaves the  s n r that it was created in.)s have , ', (, etc. instances, so they can be manipulated using polymorphic functions and numeric functions in Haskell. You can add them, subtract them, etc., in "implicit" backprop style./(However, note that if you directly manipulate "s using those instances or using w, it delays evaluation, so every usage site has to re-compute the result/create a new node. If you want to re-use a  you created using ) or * or , use , to force it first. See documentation for  for more details.)A Monad allowing you to explicitly build hetereogeneous data dependency graphs and that the library can perform back-propagation on.A  s n r a is a ! action that uses an environment n values of type r, and returns an a<. When "run", it will compute a gradient that is a vector () of n rs. (The phantom parameter s is used to ensure that any !s aren't leaked out of the monad)Note that you can only "run" a  s n r that produces a  -- that is, things of the form  s n r ( n r a) The above is a  action that returns a  containing an a4. When this is run, it'll produce a result of type a( and a gradient of that is a vector of n values of type r#. (This form has a type synonym, , for convenience) For example,  s  Double6 is a monad that represents a computation with three s as inputs. And, if you ran a  s  Double ( N3 Double Int) Or, using the  type synonym:  s  Double Int with  or <, it'll return a gradient on the inputs (a vector of three s) and produce a value of type .2Now, one powerful thing about this type is that a  is itself an  (or more precisely, an #). So, once you create your fancy , computation, you can transform it into an  using . Apply an  to a  (vector) of s.If you had an  s N3 a b2, this function will expect a vector of of three  s n r as, and the result will be a  s n r b: myOp ::  s N3 a b x ::  s n r a y ::  s n r a z ::  s n r a x  y :* z :* 'V' ::  N3 ( s n r) a  myOp (x :* y :* z :* V) ::  s n r ( s n r b)  Note that  is a superclass of , so you can provide any ' here, as well (like those created by , , constOp,  etc.) has an infix alias, /, so the above example can also be written as: myOp  (x :* y :* z :* V) ::  s n r ( s n r b) ,to let you pretend that you're applying the myOp function to three inputs.Also note the relation between  and  and :  o xs =  ( o xs) - can be thought of as a "binding" version of .Infix synonym for /, which lets you pretend that you're applying s as if they were functions: myOp ::  s N3 a b x ::  s n r a y ::  s n r a z ::  s n r a x  y :* z :* 'V' ::  N3 ( s n r) a myOp  (x :* y :* z :* V) ::  s n r ( s n r b)  Note that  is a superclass of , so you can pass in any ' here, as well (like those created by , , constOp,  etc.)2 can also be thought of as a "binding" version of : o  xs =  (o  xs) Lets you treat a  s n a b as an  n a b;, and "apply" arguments to it just like you would with an  and  / .$Basically a convenient wrapper over  and : o  xs = bpOp o  xs  So for a  s n a b, you can "plug in"  s to each a , and get a b as a result.Useful for running a  s n a b@ that you got from a different function, and "plugging in" its a inputs with !s from your current environment. Create a J that represents just a specific value, that doesn't depend on any other s.Convenient wrapper over  that takes an ! with one argument and a single 6 argument. Lets you not have to type out the entire .  o x =  o (x  'V') myOp ::  N2 a b x ::  s n r a  myOp x ::  s n r ( s n r b)  Note that  is a superclass of , so you can pass in an  here (like one made with  ) as well.Convenient wrapper over  that takes an  with two arguments and two 7 arguments. Lets you not have to type out the entire .  o x y =  o (x  y  'V') myOp ::  N2 a b x ::  s n r a y ::  s n r b  myOp x y ::  s n r ( s n r b)  Note that  is a superclass of , so you can pass in an  here (like one made with  ) as well.Convenient wrapper over  that takes an ! with three arguments and three 7 arguments. Lets you not have to type out the entire .  o x y z =  o (x  y  z  'V') myOp ::  N3 a b x ::  s n r a y ::  s n r a z ::  s n r a  myOp x y z ::  s n r ( s n r b)  Note that  is a superclass of , so you can pass in an  here (like one made with  ) as well.Concretizes a delayed . If you build up a  using numeric functions like ) or N or using \, it'll defer the evaluation, and all of its usage sites will create a separate graph node.Use  if you ever intend to use a  in more than one location. -- bad errSquared :: Num a =>  s N2 a a errSquared = withInp0 $ \(x :* y :* ) -> do let err = r - t OE (err * err) -- err is used twice! -- good errSquared :: Num a =>  s N2 a a errSquared = withInp5 $ \(x :* y :* ) -> do let err = r - t e <- 9 err -- force e, so that it's safe to use twice! O+ (e * e) -- better errSquared :: Num a =>  s N2 a a errSquared = withInp5 $ \(x :* y :* ) -> do let err = r - t e <-  err @ (e * e) -- result is forced so user doesn't have to worry Note the relation to      / :  o xs =  ( o xs) o  xs =  (o  xs)  (*)  (x :< y :< ) =  (x * y) So you can avoid / altogether if you use the explicitly binding  and  etc. Note that  on %s that are already forced is a no-op.&Perform back-propagation on the given . Returns the result of the operation it represents, as well as the gradient of the result with respect to its inputs. See module header for Numeric.Backprop.Mono4 and package documentation for examples and usages.Simply run the f on an input vector, getting the result without bothering with the gradient or with back-propagation.Run the [ on an input vector and return the gradient of the result with respect to the input vector A version of  taking explicit X$, indicating the size of the input .Requiring an explicit X is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the length of the the expected input vector. If you ever actually explicitly write down the size n!, you should be able to just use .Turn a  into an . Basically converts a  taking n rs and producing an a into an  taking an n rs and returning an a+, with all of the powers and utility of an /, including all of its gradient-finding glory.&Really just reveals the fact that any  s rs a is itself an , an  s rs a+, which makes it a differentiable function.Handy because an % can be used with almost all of the --related functions in this moduel, including , , etc. Create a  given an index (^/) into the input environment. For an example,  _ would refer to the first input variable, Bool]@), and  (` _) Would refer to the second input variable.0Typically, there shouldn't be any reason to use 3 directly. It's cleaner to get all of your input s together using  or .Get a  (vector) of )s for all of the input environment (the n r s) of the  s n rFor example, if your  has two #s inside its input environment (a  s  Double), this would return two s, pointing to each input . case ( ::   ( s h Double) Double) of x :* y :* V -> do -- the first item, x, is a var to the first input x :: N s N2 Double -- the second item, y, is a var to the second input y ::  s N2 Double Runs a continuation on a  of all of the input s.FHandy for bringing the environment into scope and doing stuff with it: foo ::   Double Int foo = 5 $ \(x :* y :* V) -> do -- do stuff with inputs Looks kinda like foo (x :* y *+ V) = -- ... , don't it?"Note that the above is the same as foo ::   Double Int foo = do case = of x :* y :* V -> do -- do stuff with inputs But just a little nicer! Convert a  into a /. That is, convert a function on a bundle of 7s (generating an implicit graph) into a fully fledged  that you can run  on. See  for more information.5If you are going to write exclusively using implicit 1 operations, it might be more convenient to use Numeric.Backprop.Mono.Implicit0 instead, which is geared around that use case.Apply  over a  of Ns, as inputs. Provides "implicit" back-propagation, with deferred evaluation.If you had an  s N3 a b2, this function will expect a vector of of three  s n r as, and the result will be a  s n r b: myOp ::  s N3 a b x ::  s n r a y ::  s n r a z ::  s n r a x  y :* z :* 'V' ::  N3 ( s n r) a  myOp (x :* y :* z :* V) ::  s n r b  Note that  is a superclass of , so you can provide any ' here, as well (like those created by , , constOp,  etc.) has an infix alias, /, so the above example can also be written as: myOp  (x :* y :* z :* V) ::  s n r b ,to let you pretend that you're applying the myOp function to three inputs.The result is a new deferred . This should be fine in most cases, unless you use the result in more than one location. This will cause evaluation to be duplicated and multiple redundant graph nodes to be created. If you need to use it in two locations, you should use  instead of  , or use :  o xs =  ( o xs) 9 can be thought of as a "deferred evaluation" version of .Infix synonym for /, which lets you pretend that you're applying s as if they were functions: myOp ::  s N3 a b x ::  s n r a y ::  s n r a z ::  s n r a x  y :* z :* 'V' ::  N3 ( s n r) a myOp  (x :* y :* z :* V) ::  s n r b  Note that  is a superclass of , so you can pass in any ' here, as well (like those created by , , constOp,  etc.)See the documentation for # for all the caveats of this usage.> can also be thought of as a "deferred evaluation" version of : o  xs =  (o  xs) Convenient wrapper over  that takes an ! with one argument and a single 6 argument. Lets you not have to type out the entire .  o x =  o (x  'V') myOp ::  N2 a b x ::  s n r a  myOp x ::  s n r b  Note that  is a superclass of , so you can pass in an  here (like one made with  ) as well.See the documentation for = for caveats and potential problematic situations with this.Convenient wrapper over  that takes an  with two arguments and two 7 arguments. Lets you not have to type out the entire .  o x y =  o (x  y  'V') myOp ::  N2 a b x ::  s n r a y ::  s n r b  myOp x y ::  s n r b  Note that  is a superclass of , so you can pass in an  here (like one made with  ) as well.See the documentation for = for caveats and potential problematic situations with this.Convenient wrapper over  that takes an ! with three arguments and three 7 arguments. Lets you not have to type out the entire Prod.  o x y z =  o (x  y  z  'V') myOp ::  N3 a b x ::  s n r a y ::  s n r b z ::  s n r b  myOp x y z ::  s n r b  Note that  is a superclass of , so you can pass in an  here (like one made with  ) as well.See the documentation for = for caveats and potential problematic situations with this.U !"U !"55(c) Justin Le 2017BSD3 justin@jle.im experimental non-portableNone:OTbAn operation on .s that can be backpropagated. A value of type:  n r a takes a vector () of  s containg n r's and uses them to (purely) produce a  containing an a. foo ::   Double Double foo (x  y  'V') = x + sqrt y  here is related to 1 from the normal explicit-graph backprop module Numeric.Backprop.Mono.Run back-propagation on a ^ function, getting both the result and the gradient of the result with respect to the inputs. foo ::  J Double Double foo (x :* y :* V) = let z = x * sqrt y in z + x ** y 'backprop' foo (2 :+ 3 :+ V)(11.46, 13.73 :+ 6.12 :+ V)Run the Z on an input tuple and return the gradient of the result with respect to the input tuple. foo ::  J Double Double foo (x :* y :* V) = let z = x * sqrt y in z + x ** y 'grad' foo (2 :+ 3 :+ V)13.73 :+ 6.12 :+ VSimply run the e on an input tuple, getting the result without bothering with the gradient or with back-propagation. foo ::  J Double Double foo (x :* y :* V) = let z = x * sqrt y in z + x ** y 'eval' foo (2 :+ 3 :+ V)11.46@ !"@ !"a !"#$"#%"#&"#'"#("#)"#*"#+",-",$",.",/",0",1",2",3",4"56"57"57"89"8:"8;"<="<>"<?"<@"<A"<B"<C"<D"<E"<F"<GHIJKLMNOPQRSSRTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~     SRSRkijopqrlmnU[_ac\T]d^`bVWXYZstuvwxyz{|}~  "                                  """"<"< !""#$"#%"#&''backprop-0.0.2.0-4cb7W0fLZ9GGKeB8OXvOWJNumeric.Backprop.IsoNumeric.Backprop.Op.MonoNumeric.Backprop.OpNumeric.BackpropNumeric.Backprop.ImplicitNumeric.Backprop.MonoNumeric.Backprop.Mono.ImplicitData.Type.UtilOpB Numeric.LensexponentiatingNumeric.Backprop.InternalliftBbindVarBPOpbackpropgradBPOpbpOpopVar'~$Control.Lens.IsoIso GHC.GenericsGeneric Generics.SOPBPOpIpartsVar Numeric.ADdiffgrad(microlens-0.4.8.0-9sB7IE1nvPRFUzu1zwDBDSLens.Micro.Extrasview/type-combinators-0.2.4.3-EyU0Q5tzlEK5bb9byDWJrtData.Type.Vectorhead'+:*::+:*ØVVecTVecData.Type.Productonly_only:>::<:<ØProdTupleData.Type.CombinatorgetII Data.Type.SumInRInLSumType.Family.NatN0N1N2N3N4N5N6N7N8N9N10Iso'isoreviewcoercedgTuplegSOPsum1resum1from ReplicateOpOpMrunOpM'runOp' composeOp' composeOp composeOp1' composeOp1~.runOpgradOp'runOpMgradOpM' gradOpWith' gradOpWithM' gradOpWith gradOpWithMgradOpgradOpMopCoerceopTup'opTupopIsoopConst'opConstop0op1'op2'op3'op1op2op3opN+.-.*./.**.negateOpsignumOpabsOprecipOpexpOplogOpsqrtOp logBaseOpsinOpcosOptanOpasinOpacosOpatanOpsinhOpcoshOptanhOpasinhOpacoshOpatanhOp $fFloatingOpM$fFractionalOpM$fNumOpMBVarBPopVar splitVars#<~ withPartsgSplit choicesVar withChoices?<~sopVargSplits-$constVaropVar1opVar2opVar3evalBPOp implicitly' implicitlyinpVarinpVarsinpVars' withInps'withInps.$liftB1liftB2liftB3eval partsVar' withParts' splitVars'gSplit'bpOp'baseGHC.Base.+generics-sop-0.2.4.0-8q0y0oRZNln24Ov1OYtww6Generics.SOP.UniversesopsopTCnpProdnsSum vecToProd prodToVec' prodAlongfinIndex traverse1_ itraverse1_for1for1_ifor1ifor1_zipPunzipPindexP prodLength vecLengthtagSumreplWitreplLen lengthProdOpContghc-prim GHC.TypesIntBoolDoubleMonadIOData.Type.LengthLengthData.TraversablesequenceNothingJustGHC.NumNum!ad-4.3.2.1-5rtdMRQUMHiK2myB21WWWxNumeric.AD.Mode.ForwardNumeric.AD.Mode.ReverseOC runOpContBPPipeBPNodeBVOpBPInpRef ForwardRefsIRNodeIRPipeIRConstGHC.Real Fractional GHC.FloatFloating+-BVNodeBVInpBVConstBPState FRInternal FRTerminal$fMonoidForwardRefs _FRInternal$fFloatingBVar$fFractionalBVar $fNumBVarBPP_bppOut_bppRes _bppGradFunc _bppGradCacheBPN_bpnOut_bpnRes _bpnGradFunc _bpnGradCachebpSTBPS _bpsSources bpsSources bpnGradCache bpnGradFuncbpnOutbpnRes bppGradCache bppGradFuncbppOutbppResid*returnData.Type.IndexIZIScaching resolveVar registerVar backwardPasscloseOff backpropWith Data.Type.NatNat/**negate Data.Type.FinFinFZFS