úΘ”ø      GHC only? experimentalbjorn.buckwalter@gmail.com<The 5 type is a concrete representation of a higher-order 9 Dual number, meaning a number augmented with a tower of E derivatives. These generalize the Dual numbers of Clifford (1873), ? which hold only a first derivative. They can be converted to A formal power series via division by the sequence of factorials. The 5 function injects a primal number into the domain of A dual numbers, with a zero tower. If dual numbers were a monad,   would be . The  2 function takes a primal number and a dual number D tower and returns a dual number tower with the given tower shifted % up one and the new primal inserted.  Property: &x = bundle (primal x) (tangentTower x) !0The zero element of a Dual Number tower algebra "The "5 function applies a function to a number lifted from A the primal domain to the dual number domain, with derivative 1, D thus calculating the generalized push-forward, in the differential < geometric sense, of the given function at the given point. #The #2 function finds the i-th element of a dual number ? | tower, where the 0-th element is the primal value, the 1-st ) | element is the first derivative, etc. $The $1 function converts a dual number tower to a list D of values with the i-th derivatives, i=0,1,..., possibly truncated 2 when all remaining values in the tower are zero. %The %1 function converts a list of numbers into a dual  | number tower. &The &4 function finds the primal value from a dual number  | tower. The inverse of . 'The '3 function finds the tangent value of a dual number , | tower, i.e., the first-order derivative. (The (, function finds the entire tower of tangent @ values of a dual number tower, starting at the 1st derivative. B This is equivalent, in an appropriate sense, to taking the first  derivative. )The )6 function lifts a scalar numeric function from a base ? numeric type to a function over derivative towers. Takes the @ primal function and the derivative function as arguments. The - derivative function should return a scalar.  EXAMPLE: liftA1 sin cos *The *6 function lifts a binary numeric function from a base ? numeric type to a function over derivative towers. Takes the @ primal function and the derivative function as arguments. The 8 derivative function should return the Jacobian matrix,  representated as a pair.  EXAMPLE:  liftA2 (*) (\ x y -> (y,x)) +The +0 funciton lifts a scalar numeric function, like  )<, except the the derivative function is given access to the 0 output value. This eases recursion and reuse.  EXAMPLE: liftA1_ exp const ,The ,0 funciton lifts a binary numeric function, like  *<, except the the derivative function is given access to the 0 output value. This eases recursion and reuse.  EXAMPLE: liftA2_ (**) (z x y -> (y*z/ x, z*log x)) -The -/ function lifts a scalar function with numeric B input and discrete output from into the derivative tower domain. .The ./ function lifts a binary function with numeric C inputs and discrete output from into the derivative tower domain. /The /2 function lifts a linear scalar function from the ? primal domain into the derivative tower domain. WARNING: the E restriction to linear functions is not enforced by the type system. The / function calculates the first derivative of a  scalar-to-scalar function. The - function calculates the first derivative of  scalar-to-nonscalar function. The 5 function calculate the product of the Jacobian of a E nonscalar-to-scalar function with a given vector. Aka: directional  derivative. The 6 function calculates the product of the Jacobian of a < nonscalar-to-nonscalar function with a given vector. Aka:  directional derivative. The 4 function calculates the value and derivative, as a ' pair, of a scalar-to-scalar function. The diffUF24 function calculates the value and derivative, as a * pair, of a scalar-to-nonscalar function. The diffMU2/ function calculates the value and directional ; derivative, as a pair, of a nonscalar-to-scalar function. The diffMF2/ function calculates the value and directional > derivative, as a pair, of a nonscalar-to-nonscalar function. 012The 2- function is like Data.List.transpose except ? that it fills in missing elements with the given padding item E rather than skipping them. It can give a ragged output to a ragged 9 input, but the lengths in the output are nonincreasing. 3The 3- function is like Data.List.transpose except ? that it fills in missing elements with the given padding item E rather than skipping them, and is generalized to Foldable Functors. ! Unlike transposePad, it gives a square output to a ragged input. The  0 function calculates a list of derivatives of a @ scalar-to-scalar function. The 0-th element of the list is the > primal value, the 1-st element is the first derivative, etc. The  5 function calculates an infinite list of derivatives E of a scalar-to-nonscalar function. The 0-th element of the list is B the primal value, the 1-st element is the first derivative, etc. The  5 function calculates an infinite list of derivatives E of a nonscalar-to-scalar function. The 0-th element of the list is B the primal value, the 1-st element is the first derivative, etc. ? The input is a (possibly truncated) list of the primal, first  derivative, etc, of the input. The  5 function calculates an infinite list of derivatives E of a nonscalar-to-nonscalar function. The 0-th element of the list E is the primal value, the 1-st element is the first derivative, etc. ? The input is a (possibly truncated) list of the primal, first  derivative, etc, of the input. The  function is like  # except the output is zero padded. The  function is like  # except the output is zero padded. The  function is like  # except the output is zero padded. The  function is like  # except the output is zero padded. The  function is a synonym for . The  function is a synonym for . The  function is a synonym for  . The  function is a synonym for . The ' function calculates the gradient of a B nonscalar-to-scalar function, using n invocations of forward AD, ; where n is the input dimmensionality. NOTE: this is O(n) ( inefficient as compared to reverse AD. The ' function calcualtes the Jacobian of a E nonscalar-to-nonscalar function, using n invocations of forward AD, ' where n is the input dimmensionality. 4The 4/ function converts a tower of derivatives to a . pair of the primal and the first derivative. 5The 5+ function converts a functor of derivative C towers to a pair: one with the functor holding the primal values, ; the other with the functor holding the first derivatives. 6The 6/ function zip two lists of numbers into a list > of derivative towers with the given primal values andd first  derivatives. Like zipWith Bundle except that the two lists  should be the same length. 7The 73 function lowers a function over dual numbers to a 6 function in the primal domain, where the function is  scalar-to-scalar. 8The 83 function lowers a function over dual numbers to a C function over primals, where the function is scalar-to-nonscalar. 9The 93 function lowers a function over dual numbers to a B function over primals where the function is nonscalar-to-scalar. :The :3 function lowers a function over dual numbers to a E function over primals where the function is nonscalar-to-nonscalar. ;The ;3 function makes an identity matrix, represented as A list of lists of numbers. The dimensionality is taken from the  length of the argument list. <The <5 function formats a vector or matrix (represented as 4 list or list of lists) for convenient examination. The 0 function evaluate a Taylor series of the given E function around the given point with the given delta. It returns a 3 list of increasingly higher-order approximations.  EXAMPLE: taylor exp 0 1 The 2 function finds a zero of a scalar function using  Newton':s method; its output is a stream of increasingly accurate ' results. (Modulo the usual caveats.)  TEST CASE:  take 10 $ zeroNewton (\x->x^2-4) 1 -- converge to 2.0  TEST CASE # :module Data.Complex Numeric.FAD  Btake 10 $ zeroNewton ((+1).(^2)) (1 :+ 1) -- converge to (0 :+ 1) The * function inverts a scalar function using  Newton':s method; its output is a stream of increasingly accurate ' results. (Modulo the usual caveats.)  TEST CASE:  ;take 10 $ inverseNewton sqrt 1 (sqrt 10) -- converge to 10 The ( function find a fixedpoint of a scalar  function using Newton'$s method; its output is a stream of = increasingly accurate results. (Modulo the usual caveats.) The ( function finds an extremum of a scalar  function using Newton',s method; produces a stream of increasingly 0 accurate results. (Modulo the usual caveats.) The " function performs a multivariate ? optimization, based on the naive-gradient-descent in the file   stalingrad/examples/ flow-tests/pre-saddle-1a.vlad from the > VLAD compiler Stalingrad sources. Its output is a stream of B increasingly accurate results. (Modulo the usual caveats.) The D gradient is calculated using Forward AD, which is O(n) inefficient A as compared to Reverse AD, where n is the input dimensionality.    =      !"#$%&'()*+,-./0123456789:;<=>?@Afad-1.0 Numeric.FADDualliftdiffUUdiffUFdiffMUdiffMFdiff2UUdiff2UFdiff2MUdiff2MFdiffsUUdiffsUFdiffsMUdiffsMFdiffs0UUdiffs0UFdiffs0MUdiffs0MFdiffdiff2diffsdiffs0gradjacobiantaylor zeroNewton inverseNewtonfixedPointNewtonextremumNewtonargminNaiveGradientTowerbaseGHC.BasereturnbundlezeroapplytowerElt fromTowertoTowerprimaltangent tangentTowerliftA1liftA2liftA1_liftA2_ liftA1disc liftA2discliftLinzeroPadzeroPadF transposePad transposePadF dualToPair fdualsToPair zipWithBundlelowerUUlowerUFlowerFUlowerFFidentityshow2d