` TC      !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~None ,-FQSTV] A n a is a list of a s repeated n times.:kind! Replicate N3 Int'[Int, Int, Int]:kind! Replicate N5 Double)'[Double, Double, Double, Double, Double] (c) Justin Le 2018BSD3 justin@jle.im experimental non-portableNone &'-<QV]dh2.Helper wrapper used for the implementation of .An  as a* describes a differentiable function from as to a.For example, a value of type  '[Int, Bool] Double is a function from an  and a , returning a '. It can be differentiated to give a gradient of an  and a & if given a total derivative for the Double. If we call  2), then, mathematically, it is akin to a:2 f : \mathbb{Z} \times 2 \rightarrow \mathbb{R} See , , and % for examples on how to run it, and ! for instructions on creating it.NIt is simpler to not use this type constructor directly, and instead use the $, #, $, and % helper smart constructors.See Numeric.Backprop.Op#prod for a mini-tutorial on using  and  . Construct an  by giving a function creating the result, and also a continuation on how to create the gradient, given the total derivative of a.!See the module documentation for Numeric.Backprop.Op= for more details on the function that this constructor and  expect.Run the function that the \ encodes, returning a continuation to compute the gradient, given the total derivative of a. See documentation for Numeric.Backprop.Op for more information. A version of  taking explicit <, indicating the number of inputs expected and their types.Requiring an explicit  is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the the expected input tuple. If you ever actually explicitly write down as5 as a list of types, you should be able to just use .Compose s together, like  for functions, or liftAN.That is, given an  as b1, an  as b2 , and an  as b3, it can compose them with an  '[b1,b2,b3] c to create an  as c. A version of  taking explicit <, indicating the number of inputs expected and their types.Requiring an explicit  is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the the expected input tuple. If you ever actually explicitly write down as5 as a list of types, you should be able to just use .Convenient wrapper over J for the case where the second function only takes one input, so the two ,s can be directly piped together, like for .'Convenient infix synonym for (flipped) . Meant to be used just like : f ::  '[b] c g ::  '[a,a] b f  g :: Op '[a, a] c Run the function that an  encodes, to get the result.runOp (op2 (*)) (3 ::< 5 ::< )15Run the function that an X encodes, to get the resulting output and also its gradient with respect to the inputs.!gradOp' (op2 (*)) (3 ::< 5 ::< )(15, 5 ::< 3 ::< )"Get the gradient function that an N encodes, with a third argument expecting the total derivative of the result.!See the module documentaiton for Numeric.Backprop.Op for more information.Run the function that an I encodes, and get the gradient of the output with respect to the inputs. gradOp (op2 (*)) (3 ::< 5 ::< ) 5 ::< 3 ::< -- the gradient of x*y is (y, x)  o xs =  o xs 1 An X that coerces an item into another item whose type has the same runtime representation.4gradOp' opCoerce (Identity 5) :: (Int, Identity Int)(5, Identity 1)  =  coerced  An A that just returns whatever it receives. The identity function.  =    An  that takes as% and returns exactly the input tuple.#gradOp' opTup (1 ::< 2 ::< 3 ::< )*(1 ::< 2 ::< 3 ::< , 1 ::< 1 ::< 1 ::< )An 2 that runs the input value through an isomorphism.yWarning: This is unsafe! It assumes that the isomorphisms themselves have derivative 1, so will break for things like =. Basically, don't use this for any "numeric" isomorphisms.An Y that runs the input value through an isomorphism between a tuple of values and a value.yWarning: This is unsafe! It assumes that the isomorphisms themselves have derivative 1, so will break for things like =. Basically, don't use this for any "numeric" isomorphisms.An 3 that extracts a value from an input value using a .Warning: This is unsafe! It assumes that it extracts a specific value unchanged, with derivative 1, so will break for things that numerically manipulate things before returning them.  A version of ! taking explicit 3, indicating the number of inputs and their types.Requiring an explicit  is mostly useful for rare "extremely polymorphic" situations, where GHC can't infer the type and length of the the expected input tuple. If you ever actually explicitly write down as5 as a list of types, you should be able to just use !.!An D that ignores all of its inputs and returns a given constant value.*gradOp' (opConst 10) (1 ::< 2 ::< 3 ::< )(10, 0 ::< 0 ::< 0 ::< )" Create an : that takes no inputs and always returns the given value.'There is no gradient, of course (using Q will give you an empty tuple), because there is no input to have a gradient of.runOp (op0 10) (10, )For a constant & that takes input and ignores it, see ! and  .# Create an  of a function taking one input, by giving its explicit derivative. The function should return a tuple containing the result of the function, and also a function taking the derivative of the result and return the derivative of the input. If we haveT \eqalign{ f &: \mathbb{R} \rightarrow \mathbb{R}\cr y &= f(x)\cr z &= g(y) } Then the derivative  \frac{dz}{dx} , it would be:/ \frac{dz}{dx} = \frac{dz}{dy} \frac{dy}{dx} If our  represents fO, then the second item in the resulting tuple should be a function that takes  \frac{dz}{dy} and returns  \frac{dz}{dx}.As an example, here is an  that squares its input: square :: Num a =>  '[a] a square = #6 $ \x -> (x*x, \d -> 2 * d * x ) ARemember that, generally, end users shouldn't directly construct Ds; they should be provided by libraries or generated automatically.$ Create an  of a function taking two inputs, by giving its explicit gradient. The function should return a tuple containing the result of the function, and also a function taking the derivative of the result and return the derivative of the input. If we haveY \eqalign{ f &: \mathbb{R}^2 \rightarrow \mathbb{R}\cr z &= f(x, y)\cr k &= g(z) } Then the gradient M \left< \frac{\partial k}{\partial x}, \frac{\partial k}{\partial y} \right>  would be: \left< \frac{\partial k}{\partial x}, \frac{\partial k}{\partial y} \right> = \left< \frac{dk}{dz} \frac{\partial z}{dx}, \frac{dk}{dz} \frac{\partial z}{dy} \right> If our  represents fO, then the second item in the resulting tuple should be a function that takes  \frac{dk}{dz} and returns = \left< \frac{\partial k}{dx}, \frac{\partial k}{dx} \right> .As an example, here is an  that multiplies its inputs: mul :: Num a =>  '[a, a] a mul = op2'9 $ \x y -> (x*y, \d -> (d*y, x*d) ) ARemember that, generally, end users shouldn't directly construct Ds; they should be provided by libraries or generated automatically.% Create an ] of a function taking three inputs, by giving its explicit gradient. See documentation for $ for more details.& for addition' for subtraction( for multiplication) for division* for exponentiation+ for negation, for - for absolute value. for multiplicative inverse/ for 0 for the natural logarithm1 for square root2 for 3 for sine4 for cosine5 for tangent6 for arcsine7 for arccosine8 for arctangent9 for hyperbolic sine: for hyperbolic cosine; for hyperbolic tangent< for hyperbolic arcsine= for hyperbolic arccosine> for hyperbolic arctangent of  s taking as and returning different b in bsOpM taking eac of the bs from the input . Composed  of  s taking as and returning different b in bs taking eac of the bs from the input . Composed  to runInputs to run it with#The total derivative of the result. The gradient=  !"#$%&'()*+,-./0123456789:;<=>= "! #$%&'(+-,)./01*23456789:;<=>9 (c) Justin Le 2018BSD3 justin@jle.im experimental non-portableNone"#%&',-6<FNQVh,BqAn ephemeral Wengert Tape in the environment. Used internally to track of the computational graph of variables.'For the end user, one can just imagine  s B as a required constraint on s% that allows backpropagation to work.CA C s a is a value of type a that can be "backpropagated".Functions referring to Cis are tracked by the library and can be automatically differentiated to get their gradients and results.+For simple numeric values, you can use its , , and J instances to manipulate them as if they were the numbers they represent.If aJ contains items, the items can be accessed and extracted using lenses. A  b a can be used to access an a inside a b , using ^^. (I): () :: a ->  a b -> b (^^.) :: C s a ->  a b -> C s b There is also ^^? (M ), to use a Prism' or ^ to extract a target that may or may not be present (which can implement pattern matching), ^^.. (N ) to use a  to extract all targets inside a C, and .~~ (J%) to set and update values inside a C.@For more complex operations, libraries can provide functions on C s using E and related functions. This is how you can create primitive functions that users can use to manipulate your library's values.For example, the hmatrix7 library has a matrix-vector multiplication function, #> :: L m n -> R n -> L m.+A library could instead provide a function #> :: C% (L m n) -> BVar (R n) -> BVar (R m)3, which the user can then use to manipulate their Cs of L m ns and R ns, etc.See Numeric.Backprop#liftops and documentation for E for more information.$Project out a constant value if the C refers to one.Debugging string for an .Debugging string for a  SomeTapeMode.DLift a value into a C representing a constant value.ZThis value will not be considered an input, and its gradients will not be backpropagated.ELift an P with an arbitrary number of inputs to a function on the appropriate number of Cs.AShould preferably be used only by libraries to provide primitive C& functions for their types for users.See Numeric.Backprop#liftops and documentation for E for more information, and Numeric.Backprop.Op#prod for a mini-tutorial on using  and  .FLift an 2 with a single input to be a function on a single C.AShould preferably be used only by libraries to provide primitive C& functions for their types for users.See Numeric.Backprop#liftops and documentation for E for more information.GLift an + with two inputs to be a function on a two Cs.AShould preferably be used only by libraries to provide primitive C& functions for their types for users.See Numeric.Backprop#liftops and documentation for E for more information.HLift an / with three inputs to be a function on a three Cs.AShould preferably be used only by libraries to provide primitive C& functions for their types for users.See Numeric.Backprop#liftops and documentation for E for more information.IUsing a , extract a value inside a C . Meant to evoke parallels to view from lens.See documentation for ^^. for more information.JUsing a , set a value inside a C0. Meant to evoke parallels to "set" from lens.See documentation for .~~ for more information.KExtract all of the C s out of a  container of Cs.LCollect all of the Cs in a container into a C of that container's contents.Note that this requires t a to have a 9 instance. If you are using a list, I recommend using  0https://hackage.haskell.org/package/vector-sized vector-sizedC instead: it's a fixed-length vector type with a very appropriate  instance!MUsing a , extract a single value inside a Ck, if it exists. If more than one traversal target exists, returns te first. Meant to evoke parallels to preview2 from lens. Really only intended to be used wth Prism'"s, or up-to-one target traversals.See documentation for ^^? for more information.NUsing a , extract all targeted values inside a C. Meant to evoke parallels to  from lens.See documentation for ^^.. for more information.Obackprop> generalized to multiple inputs of different types. See the Numeric.Backprop.Op#prod, for a mini-tutorial on heterogeneous lists.Not strictly necessary, because you can always uncurry a function by passing in all of the inputs in a data type containing all of the arguments or a tuple from Numeric.Backprop.Tuple0. You could also pass in a giant tuple with  0https://hackage.haskell.org/package/NumInstances NumInstances. However, this can be convenient if you don't want to make a custom larger tuple type or pull in orphan instances. This could potentially also be more performant.A  (C s) '[Double, Float, Double], for instance, is a tuple of C s , C s , and C s ', and can be pattern matched on using  (cons) and '' (nil).1Tuples can be built and pattern matched on using  (cons) and '' (nil), as well.The    asA in the constraint says that every value in the type-level list as must have a * instance. This means you can use, say, '[Double, Float, Int] , but not '[Double, Bool, String].If you stick to  concereteh, monomorphic usage of this (with specific types, typed into source code, known at compile-time), then    as# should be fulfilled automatically.PevalBPL generalized to multiple inputs of different types. See documentation for O for more details.*This will force the value inside, as well.BCDEFGHIJKLMNOPBC(c) Justin Le 2018BSD3 justin@jle.im experimental non-portableNone&'<QVdA QTurn a function C s a -> C s b into the function a -> b2 that it represents, also computing its gradient a as well.The Rank-N type  forall s.  s B => ... is used to ensure that C@s do not leak out of the context (similar to how it is used in Control.Monad.ST_), and also as a reference to an ephemeral Wengert tape used to track the graph of references.7Note that every type involved has to be an instance of U. This is because gradients all need to be "summable" (which is implemented using  and M), and we also need to able to generate gradients of 1 and 0. Really, only  and  methods are used from the  typeclass.iThis might change in the future, to allow easier integration with tuples (which typically do not have a I instance), and potentially make types easier to use (by only requiring %, 0, and 1, and not the rest of the  class).See the  "https://github.com/mstksg/backpropREADME/ for a more detailed discussion on this issue.If you need a T instance for tuples, you can use the canonical 2- and 3-tuples for the library in Numeric.Backprop.Tuple. If you need one for larger tuples, consider making a custom product type instead (making Num instances with something like  7https://hackage.haskell.org/package/one-liner-instancesone-liner-instances2). You can also use the orphan instances in the  0https://hackage.haskell.org/package/NumInstances NumInstances package (in particular, Data.NumInstances.TupleU) if you are writing an application and do not have to worry about orphan instances.RTurn a function C s a -> C s b into the function a -> b that it represents.UBenchmarks show that this should have virtually no overhead over directly writing a a -> b. CC is, in this situation, a zero-cost abstraction, performance-wise. Has a nice advantage over using Q in that it doesn't require % constraints on the input and output.See documentation of Q for more information.STake a function C s a -> C s b, interpreted as a function a -> b5, and compute its gradient with respect to its input.The resulting a -> a tells how the input (and its components) affects the output. Positive numbers indicate that the result will vary in the same direction as any adjustment in the input. Negative numbers indicate that the result will vary in the opposite direction as any adjustment in the input. Larger numbers indicate a greater sensitivity of change, and small numbers indicate lower sensitivity.See documentation of Q for more information.TSL generalized to multiple inputs of different types. See documentation for O for more details.UQ for a two-argument function.Not strictly necessary, because you can always uncurry a function by passing in all of the argument inside a data type, or use T26. However, this could potentially be more performant.)For 3 and more arguments, consider using O.VR# for a two-argument function. See U for notes.WS# for a two-argument function. See U for notes.XAn infix version of I, meant to evoke parallels to  from lens.KWith normal values, you can extract something from that value with a lens: x  myLens would extract a piece of x :: b, specified by  myLens ::  b a. The result has type a. xVar X myLens would extract a piece out of xVar :: C s b (a C holding a b), specified by myLens :: Lens' b a. The result has type C s a (a C holding a a)-This is the main way to pull out values from C of container types.YAn infix version of J, meant to evoke parallels to  from lens.JWith normal values, you can set something in a value with a lens: a lens: x  myLens  y would "set" a part of x :: b, specified by  myLens ::  a b, to a new value y :: a. xVar  myLens Y yVar would "set" a part of xVar :: C s b (a C holding a b), specified by  myLens ::  a b, to a new value given by  yVar :: C s a/. The result is a new (updated) value of type C s b.*This is the main way to set values inside Cs of container types.ZAn infix version of M, meant to evoke parallels to  from lens.YWith normal values, you can (potentially) extract something from that value with a lens: x  myPrism 'would (potentially) extract a piece of x :: b, specified by  myPrism ::  b a. The result has type  a. xVar Z myPrism +would (potentially) extract a piece out of xVar :: C s b (a C holding a b), specified by myPrism :: Prism' b a. The result has type  (C s a) ( a C holding a a).!This is intended to be used with Prism'@s (which hits at most one target), but will actually work with any W. If the traversal hits more than one target, the first one found will be extracted.'This can be used to "pattern match" on C$s, by using prisms on constructors.5Note that many automatically-generated prisms by the lens[ package use tuples, which cannot normally be backpropagated (because they do not have a  instance).}If you are writing an application or don't have to worry about orphan instances, you can pull in the orphan instances from  0https://hackage.haskell.org/package/NumInstances NumInstancesp. Alternatively, you can chain those prisms with conversions to the anonymous canonical strict tuple types in Numeric.Backprop.Tuple, which do have  instances. #myPrism :: Prism' c (a, b) myPrism . iso tupT2 t2Tup :: Prism' c (T2 a b) [An infix version of N, meant to evoke parallels to  from lens.5With normal values, you can extract all targets of a  from that value with a: x  myTraversal $would extract all targets inside of x :: b, specified by myTraversal ::  b a. The result has type [a]. xVar [ myTraversal $would extract all targets inside of xVar :: C s b (a C holding a b), specified by myTraversal :: Traversal' b a. The result has type [C s a] (A list of C s holding as).6  !"#$%BCDEFGHIJKLMNOPQRSTUVWXYZ[6CBQRSUVWOPT DXYZ[IJKLMNEFGH"! #$%  X8Y8(c) Justin Le 2018BSD3 justin@jle.im experimental non-portableNone136;=>?S\Strict 3-tuple with a  instance.^Strict 2-tuple with a  instance.`Convert to a Haskell tuple.Forms an isomorphism with a.aConvert from Haskell tuple.Forms an isomorphism with `.bConvert to a Haskell tuple.Forms an isomorphism with c.cConvert from Haskell tuple.Forms an isomorphism with b.d Uncurry a function to take in a ^ of its argumentseCurry a function taking a ^ of its argumentsf Uncurry a function to take in a \ of its argumentsgCurry a function taking a \ of its argumentshLens into the first field of a ^. Also exported as  from  Lens.Micro.i Lens into the second field of a ^. Also exported as  from  Lens.Micro.jLens into the first field of a \. Also exported as  from  Lens.Micro.k Lens into the second field of a \. Also exported as  from  Lens.Micro.lLens into the third field of a \. Also exported as  from  Lens.Micro.\]^_`abcdefghijkl^_`adehi\]bcjklfg\]^_              !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghhiijklmnopqrstuvwxyz{|}~ N'backprop-0.1.2.0-1gwnppePm3yIM657ybcUdENumeric.BackpropNumeric.Backprop.OpNumeric.Backprop.TupleData.Type.Util Numeric.LensexponentiatingNumeric.Backprop.Internal'reflection-2.1.3-E897NHjR81y5st3IxFkEloData.ReflectionReifies/type-combinators-0.2.4.3-3R2Zc1zHPEHI2ms5lHwgX2Data.Type.Productonly_head'only:>::<:<ØProdTupleData.Type.CombinatorgetIIData.Type.IndexEveryOp runOpWith composeOp' composeOp composeOp1' composeOp1~.evalOprunOp gradOpWithgradOpopCoerceidOpopTupopIsoopIsoNopLensopConst'opConstop0op1op2op3+.-.*./.**.negateOpsignumOpabsOprecipOpexpOplogOpsqrtOp logBaseOpsinOpcosOptanOpasinOpacosOpatanOpsinhOpcoshOptanhOpasinhOpacoshOpatanhOp $fFloatingOp$fFractionalOp$fNumOpWBVarconstVarliftOpliftOp1liftOp2liftOp3viewVarsetVar sequenceVar collectVar previewVar toListOfVar backpropNevalBPNbackpropevalBPgradBPgradBPN backprop2evalBP2gradBP2^^..~~^^?^^..T3T2t2TuptupT2t3TuptupT3 uncurryT2curryT2 uncurryT3curryT3t2_1t2_2t3_1t3_2t3_3 $fMonoidT2 $fSemigroupT2 $fFloatingT2$fFractionalT2$fNumT2$fField2T2T2bb'$fField1T2T2aa' $fBifunctorT2 $fNFDataT2 $fMonoidT3 $fSemigroupT3 $fFloatingT3$fFractionalT3$fNumT3$fField3T3T3cc'$fField2T3T3bb'$fField1T3T3aa' $fBifunctorT3 $fNFDataT3$fShowT2$fReadT2$fEqT2$fOrdT2 $fGenericT2 $fFunctorT2$fDataT2$fShowT3$fReadT3$fEqT3$fOrdT3 $fGenericT3 $fFunctorT3$fDataT3 Replicate vecToProdvecLen prodToVec' zipWithPM_zipPunzipP lengthProd listToVecDeffillProdOpContghc-prim GHC.TypesIntBoolDoubleData.Type.LengthLengthbaseData.TraversablesequenceGHC.Base.GHC.Primcoerceid(microlens-0.4.8.3-Hhd1pyfmRF65YkTRK2YoteLens.Micro.TypeLens'GHC.Numsignum GHC.FloatexplogBaseOC runOpContNumGHC.Real FractionalFloating Lens.Micro^. Traversal'bvConstdebugIRInpRefdebugSTN TraversabletoListOfFloat $fNFDataBVarRunnerR_rDelta_rInputswRef SomeTapeNodeSTNTapeNodeTN _tnInputs_tnGradIR_irIx_irUpd_irAddBRefBRInpBRIxBRCBV_bvRef_bvVal Data.Foldablesum+ fromInteger.~ Data.Function&^?Maybe^.. TraversalLens.Micro.Internal_1_2_3