úΧO£k$      !"# Safe-Inferred$The $) function is like zipWith except that it B continues until both lists are exhausted, filling in any missing # elements with the given defaults. %The %# function indexes into a list like (!!), but E when it runs off the end either returns the given default or, if no 1 default is given, sticks with the last element. $%$%$% GHC only? experimentalbjorn.buckwalter@gmail.com Safe-Inferred;The 5 type is a concrete representation of a higher-order 9 Dual number, meaning a number augmented with a tower of E derivatives. These generalize the Dual numbers of Clifford (1873), ? which hold only a first derivative. They can be converted to A formal power series via division by the sequence of factorials. The 5 function injects a primal number into the domain of D derivative towers, with a zero tower. If generalized dual numbers  were a monad,  would be &. 'The '1 function takes a primal number and a derivative C tower and returns a derivative tower with the given tower shifted % up one and the new primal inserted.  Property: &x = bundle (primal x) (tangentTower x) (1The zero element of a Tower Number tower algebra )The )5 function applies a function to a number lifted from = the primal domain to the derivative tower domain, with unit C derivative, thus calculating the generalized push-forward, in the B differential geometric sense, of the given function at the given  point. *The *1 function finds the i-th element of a derivative ? | tower, where the 0-th element is the primal value, the 1-st ) | element is the first derivative, etc. +The +3 function converts a derivative tower to a list of A values with the i-th derivatives, i=0,1,..., possibly truncated 2 when all remaining values in the tower are zero. ,The ,, function converts a list of numbers into a  | derivative tower. The 3 function finds the primal value from a derivative  | tower. The inverse of . -The -2 function finds the tangent value of a derivative , | tower, i.e., the first-order derivative. .The ., function finds the entire tower of tangent E values of a derivative tower, starting at the 1st derivative. This = is equivalent, in an appropriate sense, to taking the first  derivative. /The /6 function lifts a scalar numeric function from a base ? numeric type to a function over derivative towers. Takes the @ primal function and the derivative function as arguments. The - derivative function should return a scalar.  EXAMPLE: liftA1 sin cos 0The 06 function lifts a binary numeric function from a base ? numeric type to a function over derivative towers. Takes the @ primal function and the derivative function as arguments. The 8 derivative function should return the Jacobian matrix,  representated as a pair.  EXAMPLE:  liftA2 (*) (\ x y -> (y,x)) 1The 10 function lifts a scalar numeric function, like  /<, except the the derivative function is given access to the 0 output value. This eases recursion and reuse.  EXAMPLE: liftA1_ exp const 2The 20 function lifts a binary numeric function, like  0<, except the the derivative function is given access to the 0 output value. This eases recursion and reuse.  EXAMPLE: liftA2_ (**) (z x y -> (y*z/ x, z*log x)) 3The 3/ function lifts a scalar function with numeric ; input and discrete output from the primal domain into the  derivative tower domain. 4The 4/ function lifts a binary function with numeric C inputs and discrete output from into the derivative tower domain. 5The 52 function lifts a scalar linear function from the ? primal domain into the derivative tower domain. WARNING: The E restriction to linear functions is not enforced by the type system. 6The 62 function lifts a binary linear function from the B primal domain into the derivative tower domain. WARNING 1: The E restriction to linear functions is not enforced by the type system. D WARNING 2: Binary linear means linear in both arguments together,  not bilinear. The / function calculates the first derivative of a  scalar-to-scalar function. The - function calculates the first derivative of  scalar-to-nonscalar function. The 5 function calculate the product of the Jacobian of a E nonscalar-to-scalar function with a given vector. Aka: directional  derivative. The 6 function calculates the product of the Jacobian of a < nonscalar-to-nonscalar function with a given vector. Aka:  directional derivative. The 4 function calculates the value and derivative, as a ' pair, of a scalar-to-scalar function. The diffUF24 function calculates the value and derivative, as a * pair, of a scalar-to-nonscalar function. The diffMU2/ function calculates the value and directional ; derivative, as a pair, of a nonscalar-to-scalar function. The diffMF2/ function calculates the value and directional > derivative, as a pair, of a nonscalar-to-nonscalar function. 7The 7- function is like Data.List.transpose except E that it fills in missing elements with 0 rather than skipping them. C It can give a ragged output to a ragged input, but the lengths in  the output are nonincreasing. 8The 8- function is like Data.List.transpose except E that it fills in missing elements with 0 rather than skipping them, C and is generalized to Foldable Functors. Unlike transposePad, it  gives a square output to a ragged input. The  0 function calculates a list of derivatives of a @ scalar-to-scalar function. The 0-th element of the list is the > primal value, the 1-st element is the first derivative, etc. The  5 function calculates an infinite list of derivatives E of a scalar-to-nonscalar function. The 0-th element of the list is B the primal value, the 1-st element is the first derivative, etc. The  5 function calculates an infinite list of derivatives E of a nonscalar-to-scalar function. The 0-th element of the list is B the primal value, the 1-st element is the first derivative, etc. ? The input is a (possibly truncated) list of the primal, first  derivative, etc, of the input. The 5 function calculates an infinite list of derivatives E of a nonscalar-to-nonscalar function. The 0-th element of the list E is the primal value, the 1-st element is the first derivative, etc. ? The input is a (possibly truncated) list of the primal, first  derivative, etc, of the input. The  function is like  # except the output is zero padded. The  function is like  # except the output is zero padded. The  function is like  # except the output is zero padded. The  function is like # except the output is zero padded. The  function is a synonym for . The  function is a synonym for . The  function is a synonym for  . The  function is a synonym for . The ' function calculates the gradient of a B nonscalar-to-scalar function, using n invocations of forward AD, ; where n is the input dimmensionality. NOTE: this is O(n) ( inefficient as compared to reverse AD. The ' function calcualtes the Jacobian of a E nonscalar-to-nonscalar function, using n invocations of forward AD, ' where n is the input dimmensionality. 9The 9/ function converts a tower of derivatives to a . pair of the primal and the first derivative. :The :+ function converts a functor of derivative C towers to a pair: one with the functor holding the primal values, ; the other with the functor holding the first derivatives. ;The ;/ function zip two lists of numbers into a list > of derivative towers with the given primal values andd first : derivatives. The two lists should have the same length. The 3 function lowers a function over dual numbers to a 6 function in the primal domain, where the function is  scalar-to-scalar. The 3 function lowers a function over dual numbers to a C function over primals, where the function is scalar-to-nonscalar. The 3 function lowers a function over dual numbers to a B function over primals where the function is nonscalar-to-scalar. The 3 function lowers a function over dual numbers to a E function over primals where the function is nonscalar-to-nonscalar. <The <3 function makes an identity matrix, represented as A list of lists of numbers. The dimensionality is taken from the  length of the argument list. =The =5 function formats a vector or matrix (represented as 4 list or list of lists) for convenient examination. The 0 function evaluate a Taylor series of the given E function around the given point with the given delta. It returns a 3 list of increasingly higher-order approximations.  EXAMPLE: taylor exp 0 1 The 4 function evaluates a two-dimensional Taylor series E of the given function. This is calculated by nested application of  the 5 function, and the exported signature reflects this. The 2 function finds a zero of a scalar function using  Newton':s method; its output is a stream of increasingly accurate ' results. (Modulo the usual caveats.)  TEST CASE:  take 10 $ zeroNewton (\x->x^2-4) 1 -- converge to 2.0  TEST CASE # :module Data.Complex Numeric.FAD  Btake 10 $ zeroNewton ((+1).(^2)) (1 :+ 1) -- converge to (0 :+ 1) The  * function inverts a scalar function using  Newton':s method; its output is a stream of increasingly accurate ' results. (Modulo the usual caveats.)  TEST CASE:  ;take 10 $ inverseNewton sqrt 1 (sqrt 10) -- converge to 10 !The !( function find a fixedpoint of a scalar  function using Newton'$s method; its output is a stream of = increasingly accurate results. (Modulo the usual caveats.) "The "( function finds an extremum of a scalar  function using Newton',s method; produces a stream of increasingly 0 accurate results. (Modulo the usual caveats.) #The #" function performs a multivariate ? optimization, based on the naive-gradient-descent in the file   stalingrad/examples/ flow-tests/pre-saddle-1a.vlad from the > VLAD compiler Stalingrad sources. Its output is a stream of B increasingly accurate results. (Modulo the usual caveats.) The D gradient is calculated using Forward AD, which is O(n) inefficient A as compared to Reverse AD, where n is the input dimensionality. I>'()*+,-./0123456? @A78 9:;<= !"#BCDEFGHIJK$  !"#$  !"#H>'()*+,-./0123456? @A78 9:;<= !"#BCDEFGHIJKL      !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOP fad-1.1.0.1 Numeric.FAD List.UtilTowerliftprimaldiffUUdiffUFdiffMUdiffMFdiff2UUdiff2UFdiff2MUdiff2MFdiffsUUdiffsUFdiffsMUdiffsMFdiffs0UUdiffs0UFdiffs0MUdiffs0MFdiffdiff2diffsdiffs0gradjacobianprimalUUprimalUFprimalFUprimalFFtaylortaylor2 zeroNewton inverseNewtonfixedPointNewtonextremumNewtonargminNaiveGradientzipWithDefaults indexDefaultbaseGHC.BasereturnbundlezeroapplytowerElt fromTowertoTowertangent tangentTowerliftA1liftA2liftA1_liftA2_ liftA1disc liftA2discliftLinliftLin2 transposePad transposePadF dualToPair fdualsToPair zipWithBundleidentityshow2d notComplexzeroPadzeroPadF$fIntegralTower $fEnumTower $fOrdTower $fEqTower $fRealTower$fRealFracTower$fRealFloatTower$fFloatingTower$fFractionalTower $fNumTower