!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ GHC only experimentalekmett@gmail.com^ is used by  deriveMode but is not exposed  via  # to prevent its abuse by end users  via the AD data type.  is used by  deriveMode but is not exposed  via the  ) class to prevent its abuse by end users  via the AD data type. QIt provides direct access to the result, stripped of its derivative information, K but this is unsafe in general as (lift . primal) would discard derivative N information. The end user is protected from accidentally using this function G by the universal quantification on the various combinators we expose. Embed a constant  Vector sum Scalar-vector multiplication Vector-scalar multiplication Scalar division   'zero' = 'lift' 0 6Higher-order versions of the stock numerical methods.  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNO  'one' = 'lift' 1 PP t provides   instance Lifted $t given supplied instances for  + instance Lifted $t => Primal $t where ... - instance Lifted $t => Jacobian $t where ... The seemingly redundant  $tB constraints are caused by Template Haskell staging restrictions. $Find all the members defined in the  data type QQ f g# provides the following instances: < instance ('Lifted' $f, 'Num' a, 'Enum' a) => 'Enum' ($g a) 8 instance ('Lifted' $f, 'Num' a, 'Eq' a) => 'Eq' ($g a) : instance ('Lifted' $f, 'Num' a, 'Ord' a) => 'Ord' ($g a) A instance ('Lifted' $f, 'Num a, 'Bounded' a) => 'Bounded' ($g a) 3 instance ('Lifted' $f, 'Show' a) => 'Show' ($g a) 1 instance ('Lifted' $f, 'Num' a) => 'Num' ($g a) ? instance ('Lifted' $f, 'Fractional' a) => 'Fractional' ($g a) ; instance ('Lifted' $f, 'Floating' a) => 'Floating' ($g a) = instance ('Lifted' $f, 'RealFloat' a) => 'RealFloat' ($g a) ; instance ('Lifted' $f, 'RealFrac' a) => 'RealFrac' ($g a) 3 instance ('Lifted' $f, 'Real' a) => 'Real' ($g a) R  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQR O PQ !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNR   = !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMN !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQGHC only experimentalekmett@gmail.comRSR* serves as a common wrapper for different  # instances, exposing a traditional X numerical tower. Universal quantification is used to limit the actions in user code to Z machinery that will return the same answers under all AD modes, allowing us to use modes ( interchangeably as both the type level "brand") and dictionary, providing a common API. STUVWXYZ[\ RSTUVWXYZ[\ UVRSTWXYZ[\ RSTSTUVWXXYZ[\GHC only experimentalekmett@gmail.com ]^_`abcdefg ]^_`abcdefg ]^_a`bcdefg ]^^_`abcdefgGHC only experimentalekmett@gmail.comhijklThe lY function calculates the first derivative of a scalar-to-scalar function by forward-mode SR  diff sin == cos mThe d'UUV function calculates the result and first derivative of scalar-to-scalar function by Forward SR  d' sin == sin &&& cos  d' f = f &&& d f nThe nN function calculates the first derivative of scalar-to-nonscalar function by Forward SR oThe o\ function calculates the result and first derivative of a scalar-to-non-scalar function by Forward SR pThe dUMS function calculates the first derivative of scalar-to-scalar monadic function by Forward SR qThe d'UM` function calculates the result and first derivative of a scalar-to-scalar monadic function by Forward SR rBA fast, simple transposed Jacobian computed with forward-mode AD. sBA fast, simple transposed Jacobian computed with forward-mode AD. tuvwxyz{ RSThijklmnopqrstuvwxyz{xyz{tvuwrslmnohijkpqRST hijklmnopqrstuvwxyz{GHC only experimentalekmett@gmail.com|Reverse is a  A using reverse-mode automatic differentiation that provides fast diffFU, diff2FU, grad, grad2 and a fast jacobianF when you have a significantly smaller number of outputs than inputs. }~A TapeT records the information needed back propagate from the output to each input during }|   AD. >Used to mark variables for inspection during the reverse pass +back propagate sensitivities along a tape. 6This returns a list of contributions to the partials. 2 The variable ids returned in the list are likely not unique!  Return an  of $ given bounds for the variable IDs.  Return an  of sparse partials |}~|}~|}}~GHC only experimentalekmett@gmail.comThe J function calculates the gradient of a non-scalar-to-scalar function with }| AD in a single pass. The U function calculates the result and gradient of a non-scalar-to-scalar function with }| AD in a single pass.  g fE function calculates the gradient of a non-scalar-to-scalar function f( with reverse-mode AD in a single pass. L The gradient is combined element-wise with the argument using the function g.  grad == gradWith (\_ dx -> dx)  id == gradWith const  g fG calculates the result and gradient of a non-scalar-to-scalar function f with }| AD in a single pass L the gradient is combined element-wise with the argument using the function g. " grad' == gradWith' (\_ dx -> dx) The c function calculates the jacobian of a non-scalar-to-non-scalar function with reverse AD lazily in m passes for m outputs.  An alias for  The b function calculates both the result and the Jacobian of a nonscalar-to-nonscalar function, using m invocations of reverse AD,  where m( is the output dimensionality. Applying fmap snd* to the result will recover the result of   An alias for  ' gradWithF g f'@ calculates the Jacobian of a non-scalar-to-non-scalar function f with reverse AD lazily in m passes for m outputs. kInstead of returning the Jacobian matrix, the elements of the matrix are combined with the input using the g. " gradF == gradWithF (\_ dx -> dx) . gradWithF const == (\f x -> const x <$> f x)  An alias for .  g f'R calculates both the result and the Jacobian of a nonscalar-to-nonscalar function f, using m invocations of reverse AD,  where m( is the output dimensionality. Applying fmap snd* to the result will recover the result of  kInstead of returning the Jacobian matrix, the elements of the matrix are combined with the input using the g. ' jacobian' == gradWithF' (\_ dx -> dx)  An alias for  The d'4 function calculates the value and derivative, as a ' pair, of a scalar-to-scalar function. RST RST GHC only experimentalekmett@gmail.com Tower is an AD  B that calculates a tangent tower by forward AD, and provides fast diffsUU, diffsUF GHC only experimentalekmett@gmail.com  RST RST GHC only experimentalekmett@gmail.com GHC only experimentalekmett@gmail.comThe 2 function finds a zero of a scalar function using  Newton':s method; its output is a stream of increasingly accurate ' results. (Modulo the usual caveats.)  Examples:  7 take 10 $ findZero (\\x->x^2-4) 1 -- converge to 2.0  module Data.Complex C take 10 $ findZero ((+1).(^2)) (1 :+ 1) -- converge to (0 :+ 1)@ The  inverseNewton* function inverts a scalar function using  Newton':s method; its output is a stream of increasingly accurate ' results. (Modulo the usual caveats.)  Example: = take 10 $ inverseNewton sqrt 1 (sqrt 10) -- converge to 10 The ( function find a fixedpoint of a scalar  function using Newton'$s method; its output is a stream of = increasingly accurate results. (Modulo the usual caveats.) The ( function finds an extremum of a scalar  function using Newton',s method; produces a stream of increasingly 0 accurate results. (Modulo the usual caveats.) The " function performs a multivariate ? optimization, based on the naive-gradient-descent in the file   stalingrad/examples/ flow-tests/pre-saddle-1a.vlad from the > VLAD compiler Stalingrad sources. Its output is a stream of = increasingly accurate results. (Modulo the usual caveats.) HIt uses reverse mode automatic differentiation to compute the gradient.  RSTRST  GHC only experimentalekmett@gmail.comCalculate the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs. <If you need to support functions where the output is only a  or  , consider Numeric.AD.Reverse.jacobian or  from Numeric.AD.Reverse. Calculate both the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward- and reverse- mode AD based on the relative, number of inputs and outputs. <If you need to support functions where the output is only a  or  , consider Numeric.AD.Reverse.jacobian' or  from Numeric.AD.Reverse.  g f calculates the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs. SThe resulting Jacobian matrix is then recombined element-wise with the input using g. <If you need to support functions where the output is only a  or  , consider Numeric.AD.Reverse.jacobianWith or  from Numeric.AD.Reverse.  g f calculates the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs. SThe resulting Jacobian matrix is then recombined element-wise with the input using g. <If you need to support functions where the output is only a  or  , consider  Numeric.AD.Reverse.jacobianWith' or  from Numeric.AD.Reverse. . RSThijklmnopqrs.rslnmohijkpqRST  GHC only experimentalekmett@gmail.com  RST RST   !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^__`abccdefghhijklmnopqrstuvwxyz{|}~m~vwxyz{klvw    ~      h v w ~ ad-0.19Numeric.AD.ClassesNumeric.AD.InternalNumeric.AD.Internal.ForwardNumeric.AD.ForwardNumeric.AD.Internal.ReverseNumeric.AD.ReverseNumeric.AD.Internal.TowerNumeric.AD.TowerNumeric.AD.Internal.CompositionNumeric.AD.Newton Numeric.ADNumeric.AD.DirectedJacobianDunarylift1lift1_binarylift2lift2_PrimalprimalModelift<+>*^^*^/zeroLifted showsPrec1==!compare1 fromInteger1+!*!-!negate1signum1abs1/!recip1 fromRational1 toRational1pi1exp1sqrt1log1**!logBase1sin1atan1acos1asin1tan1cos1sinh1atanh1acosh1asinh1tanh1cosh1properFraction1 truncate1floor1ceiling1round1 floatRadix1 floatDigits1 floatRange1 decodeFloat1 encodeFloat1 exponent1 significand1 scaleFloat1isNaN1isIEEE1isNegativeZero1isDenormalized1 isInfinite1atan21succ1pred1toEnum1 fromEnum1 enumFrom1 enumFromThen1 enumFromTo1enumFromThenTo1 minBound1 maxBound1one deriveLifted deriveNumericADrunADzipWithTzipWithDefaultTIdprobeunprobeprobedunprobedForwardtangentunbundlebundleapplybindbind'bindWith bindWith' transposeWithdudu'duFduF'diffdiff'diffFdiffF'diffMdiffM' jacobianT jacobianWithTjacobian jacobianWith jacobian' jacobianWith'gradgrad'gradWith gradWith'ReverseTapeUnaryBinaryVarLiftvarvarId derivative derivative'partials partialArray partialMapunbind unbindWith unbindMapunbindMapWithDefaultgradFgradF' gradWithF gradWithF'gradMgradM' gradWithM gradWithM'TowergetTowerzeroPaddd'tangents getADTowerdiffsdiffs0diffsFdiffs0FdiffsMdiffs0Mtaylortaylor0 maclaurin maclaurin0On:.OrunOcompose decomposefindZeroinverse fixedPointextremumgradientDescent DirectionMixednegOne withPrimalfromBy fromIntegral1square1on discrete1 discrete2 discrete3varA liftedMembers lowerInstanceIsoisoosipidunpidSrunS backPropagatebaseGHC.ArrArraycontainers-0.3.0.0 Data.IntMapIntMapGHC.BaseFunctorMonad