J      !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ GHC only experimentalekmett@gmail.com^ is used by  deriveMode but is not exposed  via  # to prevent its abuse by end users  via the AD data type.  is used by  deriveMode but is not exposed  via the  ) class to prevent its abuse by end users  via the AD data type. QIt provides direct access to the result, stripped of its derivative information, K but this is unsafe in general as (lift . primal) would discard derivative N information. The end user is protected from accidentally using this function G by the universal quantification on the various combinators we expose. Embed a constant  Vector sum Scalar-vector multiplication Vector-scalar multiplication Scalar division   'zero' = 'lift' 0 6Higher-order versions of the stock numerical methods.  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNO  'one' = 'lift' 1 PP t provides   instance Lifted $t given supplied instances for  + instance Lifted $t => Primal $t where ... - instance Lifted $t => Jacobian $t where ... The seemingly redundant  $tB constraints are caused by Template Haskell staging restrictions. $Find all the members defined in the  data type QQ f g# provides the following instances: < instance ('Lifted' $f, 'Num' a, 'Enum' a) => 'Enum' ($g a) 8 instance ('Lifted' $f, 'Num' a, 'Eq' a) => 'Eq' ($g a) : instance ('Lifted' $f, 'Num' a, 'Ord' a) => 'Ord' ($g a) A instance ('Lifted' $f, 'Num a, 'Bounded' a) => 'Bounded' ($g a) 3 instance ('Lifted' $f, 'Show' a) => 'Show' ($g a) 1 instance ('Lifted' $f, 'Num' a) => 'Num' ($g a) ? instance ('Lifted' $f, 'Fractional' a) => 'Fractional' ($g a) ; instance ('Lifted' $f, 'Floating' a) => 'Floating' ($g a) = instance ('Lifted' $f, 'RealFloat' a) => 'RealFloat' ($g a) ; instance ('Lifted' $f, 'RealFrac' a) => 'RealFrac' ($g a) 3 instance ('Lifted' $f, 'Real' a) => 'Real' ($g a) R  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQR O PQ !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNR   = !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMN !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQGHC only experimentalekmett@gmail.comRSR* serves as a common wrapper for different  # instances, exposing a traditional X numerical tower. Universal quantification is used to limit the actions in user code to Z machinery that will return the same answers under all AD modes, allowing us to use modes ( interchangeably as both the type level "brand") and dictionary, providing a common API. STUVWXYZ[\ RSTUVWXYZ[\ UVRSTWXYZ[\ RSTSTUVWXXYZ[\GHC only experimentalekmett@gmail.com ]^_`abcdefg ]^_`abcdefg ]^_a`bcdefg ]^^_`abcdefgGHC only experimentalekmett@gmail.comhijThe kJ function calculates the first derivative of scalar-to-scalar function by ^] SR kThe kU function calculates the result and first derivative of scalar-to-scalar function by ^] SR lThe lL function calculates the first derivative of a scalar-to-scalar function by ^] SR mThe mU function calculates the result and first derivative of scalar-to-scalar function by ^] SR nThe nM function calculates the first derivative of scalar-to-nonscalar function by ^] SR oThe o[ function calculates the result and first derivative of a scalar-to-non-scalar function by ^] SR pBA fast, simple transposed Jacobian computed with forward-mode AD. qBA fast, simple transposed Jacobian computed with forward-mode AD. rstuvwxy RSThijklmnopqrstuvwxyvwxyrtpsuqlmnohijkRST hijklmnopqrstuvwxyGHC only experimentalekmett@gmail.comzReverse is a  A using reverse-mode automatic differentiation that provides fast diffFU, diff2FU, grad, grad2 and a fast jacobianF when you have a significantly smaller number of outputs than inputs. {|A TapeT records the information needed back propagate from the output to each input during {z   AD. }~>Used to mark variables for inspection during the reverse pass +back propagate sensitivities along a tape. 6This returns a list of contributions to the partials. 2 The variable ids returned in the list are likely not unique!  Return an  of $ given bounds for the variable IDs.  Return an  of sparse partials z{|}~z{|~}z{{|~}}~GHC only experimentalekmett@gmail.comThe J function calculates the gradient of a non-scalar-to-scalar function with {z AD in a single pass. The U function calculates the result and gradient of a non-scalar-to-scalar function with {z AD in a single pass.  g fE function calculates the gradient of a non-scalar-to-scalar function f( with reverse-mode AD in a single pass. L The gradient is combined element-wise with the argument using the function g. ! grad == gradWith (\_ dx -> dx)  id == gradWith const  g fG calculates the result and gradient of a non-scalar-to-scalar function f with {z AD in a single pass L the gradient is combined element-wise with the argument using the function g. " grad2 == gradWith2 (\_ dx -> dx) The c function calculates the jacobian of a non-scalar-to-non-scalar function with reverse AD lazily in m passes for m outputs. The b function calculates both the result and the Jacobian of a nonscalar-to-nonscalar function, using m invocations of reverse AD,  where m( is the output dimensionality. Applying fmap snd* to the result will recover the result of  'jacobianWith g f'@ calculates the jacobian of a non-scalar-to-non-scalar function f with reverse AD lazily in m passes for m outputs. kInstead of returning the Jacobian matrix, the elements of the matrix are combined with the input using the g. ( jacobian == jacobianWith (\_ dx -> dx) 1 jacobianWith const == (\f x -> const x <$> f x) 'jacobianWith2 g f'R calculates both the result and the Jacobian of a nonscalar-to-nonscalar function f, using m invocations of reverse AD,  where m( is the output dimensionality. Applying fmap snd* to the result will recover the result of  kInstead of returning the Jacobian matrix, the elements of the matrix are combined with the input using the g. * jacobian2 == jacobianWith2 (\_ dx -> dx) The 4 function calculates the value and derivative, as a ' pair, of a scalar-to-scalar function. The  function is a synonym for . The  function is a synonym for .  RSTRST GHC only experimentalekmett@gmail.com Tower is an AD  B that calculates a tangent tower by forward AD, and provides fast diffsUU, diffsUF GHC only experimentalekmett@gmail.com RST RST GHC only experimentalekmett@gmail.comThe 2 function finds a zero of a scalar function using  Newton':s method; its output is a stream of increasingly accurate ' results. (Modulo the usual caveats.)  Examples:  7 take 10 $ findZero (\\x->x^2-4) 1 -- converge to 2.0  module Data.Complex C take 10 $ findZero ((+1).(^2)) (1 :+ 1) -- converge to (0 :+ 1)@ The  inverseNewton* function inverts a scalar function using  Newton':s method; its output is a stream of increasingly accurate ' results. (Modulo the usual caveats.)  Example: = take 10 $ inverseNewton sqrt 1 (sqrt 10) -- converge to 10 The ( function find a fixedpoint of a scalar  function using Newton'$s method; its output is a stream of = increasingly accurate results. (Modulo the usual caveats.) The ( function finds an extremum of a scalar  function using Newton',s method; produces a stream of increasingly 0 accurate results. (Modulo the usual caveats.) The " function performs a multivariate ? optimization, based on the naive-gradient-descent in the file   stalingrad/examples/ flow-tests/pre-saddle-1a.vlad from the > VLAD compiler Stalingrad sources. Its output is a stream of = increasingly accurate results. (Modulo the usual caveats.) HIt uses reverse mode automatic differentiation to compute the gradient.  RSTRST  GHC only experimentalekmett@gmail.comCalculate the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs Calculate the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward- and reverse- mode AD based on the relative, number of inputs and outputs. If you need to support functions where the output is only a , consider using  jacobianT from Numeric.AD.Forward or  from Numeric.AD.Reverse directly.  g f calculates the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs. SThe resulting Jacobian matrix is then recombined element-wise with the input using g.  g f calculates the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs. SThe resulting Jacobian matrix is then recombined element-wise with the input using g. " RSTjklmno"jklnmoRST  GHC only experimentalekmett@gmail.com  RST RST    !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^^_`abbcdefgghijklmnopqrstuvwxyz{|}~l{}|~uwvxstjkuvst { } | ~     g u v s t { }  ad-0.15Numeric.AD.ClassesNumeric.AD.InternalNumeric.AD.Internal.ForwardNumeric.AD.ForwardNumeric.AD.Internal.ReverseNumeric.AD.ReverseNumeric.AD.Internal.TowerNumeric.AD.TowerNumeric.AD.Newton Numeric.ADNumeric.AD.DirectedJacobianDunarylift1lift1_binarylift2lift2_PrimalprimalModelift<+>*^^*^/zeroLifted showsPrec1==!compare1 fromInteger1+!*!-!negate1signum1abs1/!recip1 fromRational1 toRational1pi1exp1sqrt1log1**!logBase1sin1atan1acos1asin1tan1cos1sinh1atanh1acosh1asinh1tanh1cosh1properFraction1 truncate1floor1ceiling1round1 floatRadix1 floatDigits1 floatRange1 decodeFloat1 encodeFloat1 exponent1 significand1 scaleFloat1isNaN1isIEEE1isNegativeZero1isDenormalized1 isInfinite1atan21succ1pred1toEnum1 fromEnum1 enumFrom1 enumFromThen1 enumFromTo1enumFromThenTo1 minBound1 maxBound1one deriveLifted deriveNumericADrunADzipWithTzipWithDefaultTIdprobeunprobeprobedunprobedForwardtangentunbundlebundleapplybindbind2bindWith bindWith2 transposeWithdiffMUdiff2MUdiffdiff2diffUUdiff2UUdiffUFdiff2UF jacobianT jacobianWithTjacobian jacobianWith jacobian2 jacobianWith2gradgrad2gradWith gradWith2ReverseTapeUnaryBinaryVarLiftvarvarId derivative derivative2partials partialArray partialMapunbind unbindWith unbindMapunbindMapWithDefaultdiffFUdiff2FUTowergetTowerzeroPaddd2tangents getADTowerdiffsUUdiffs0UUdiffs0UFdiffsUFdiffsdiffs0taylortaylor0 maclaurin maclaurin0findZeroinverse fixedPointextremumgradientDescent DirectionMixednegOne withPrimalfromBy fromIntegral1square1on discrete1 discrete2 discrete3varA liftedMembers lowerInstanceIsoisoosipidunpidSrunS backPropagatebaseGHC.ArrArraycontainers-0.3.0.0 Data.IntMapIntMapGHC.BaseFunctor