| Copyright | (c) Justin Le 2018 |
|---|---|
| License | BSD3 |
| Maintainer | justin@jle.im |
| Stability | experimental |
| Portability | non-portable |
| Safe Haskell | None |
| Language | Haskell2010 |
Numeric.LinearAlgebra.Static.Backprop
Contents
Description
A wrapper over Numeric.LinearAlgebra.Static (type-safe vector and
matrix operations based on blas/lapack) that allows its operations to
work with backprop. Also
provides orphan instances of Backprop for types in
Numeric.LinearAlgebra.Static.
In short, these functions are "lifted" to work with BVars.
Using evalBP will run the original operation:
evalBP:: (forall s.ReifiessW.BVars a ->BVars b) -> a -> b
But using gradBP or backprop will give you the gradient:
gradBP:: (forall s.ReifiessW.BVars a ->BVars b) -> a -> a
These can act as a drop-in replacement to the API of Numeric.LinearAlgebra.Static. Just change your imports, and your functions are automatically backpropagatable. Useful types are all re-exported.
Also contains sumElements BVar operation.
Formulas for gradients come from the following papers:
- https://people.maths.ox.ac.uk/gilesm/files/NA-08-01.pdf
- http://www.dtic.mil/dtic/tr/fulltext/u2/624426.pdf
- http://www.cs.cmu.edu/~zkolter/course/15-884/linalg-review.pdf
- https://arxiv.org/abs/1602.07527
Some functions are notably unlifted:
svd: I can't find any resources that allow you to backpropagate if the U and V matrices are used! If you find one, let me know, or feel free to submit a PR! Because of this, Currently only a version that exports only the singular values is exported.svdTall,svdFlat: Not sure where to start for theseqr: Same story. https://github.com/tensorflow/tensorflow/issues/6504 might yield a clue?her: NoNuminstance forHermakes this impossible at the moment with the current backprop APIexmp: Definitely possible, but I haven't dug deep enough to figure it out yet! There is a description here https://people.maths.ox.ac.uk/gilesm/files/NA-08-01.pdf but it requires some things I am not familiar with yet. Feel free to submit a PR!sqrtm: Also likely possible. Maybe try to translate http://people.cs.umass.edu/~smaji/projects/matrix-sqrt/ ? PRs welcomed!linSolve: Haven't figured out where to start!</>: Same story- Functions returning existential types, like
withNullSpace,withOrth,withRows, etc.; not quite sure what the best way to handle these are at the moment. withRowsandwithColumnsmade "type-safe", without existential types, withfromRowsandfromColumns.
- data R (n :: Nat) :: Nat -> *
- type ℝ = Double
- vec2 :: Reifies s W => BVar s ℝ -> BVar s ℝ -> BVar s (R 2)
- vec3 :: Reifies s W => BVar s ℝ -> BVar s ℝ -> BVar s ℝ -> BVar s (R 3)
- vec4 :: Reifies s W => BVar s ℝ -> BVar s ℝ -> BVar s ℝ -> BVar s ℝ -> BVar s (R 4)
- (&) :: (Reifies s W, KnownNat n, 1 <= n, KnownNat (n + 1)) => BVar s (R n) -> BVar s ℝ -> BVar s (R (n + 1))
- (#) :: (Reifies s W, KnownNat n, KnownNat m) => BVar s (R n) -> BVar s (R m) -> BVar s (R (n + m))
- split :: forall p n s. (Reifies s W, KnownNat p, KnownNat n, p <= n) => BVar s (R n) -> (BVar s (R p), BVar s (R (n - p)))
- headTail :: (Reifies s W, KnownNat n, 1 <= n) => BVar s (R n) -> (BVar s ℝ, BVar s (R (n - 1)))
- vector :: forall n s. (Reifies s W, KnownNat n) => Vector n (BVar s ℝ) -> BVar s (R n)
- linspace :: forall n s. (Reifies s W, KnownNat n) => BVar s ℝ -> BVar s ℝ -> BVar s (R n)
- range :: KnownNat n => R n
- dim :: KnownNat n => R n
- data L (m :: Nat) (n :: Nat) :: Nat -> Nat -> *
- type Sq (n :: Nat) = L n n
- row :: (Reifies s W, KnownNat n) => BVar s (R n) -> BVar s (L 1 n)
- col :: (Reifies s W, KnownNat n) => BVar s (R n) -> BVar s (L n 1)
- (|||) :: (Reifies s W, KnownNat c, KnownNat r1, KnownNat (r1 + r2)) => BVar s (L c r1) -> BVar s (L c r2) -> BVar s (L c (r1 + r2))
- (===) :: (Reifies s W, KnownNat c, KnownNat r1, KnownNat (r1 + r2)) => BVar s (L r1 c) -> BVar s (L r2 c) -> BVar s (L (r1 + r2) c)
- splitRows :: forall p m n s. (Reifies s W, KnownNat p, KnownNat m, KnownNat n, p <= m) => BVar s (L m n) -> (BVar s (L p n), BVar s (L (m - p) n))
- splitCols :: forall p m n s. (Reifies s W, KnownNat p, KnownNat m, KnownNat n, KnownNat (n - p), p <= n) => BVar s (L m n) -> (BVar s (L m p), BVar s (L m (n - p)))
- unrow :: (Reifies s W, KnownNat n) => BVar s (L 1 n) -> BVar s (R n)
- uncol :: (Reifies s W, KnownNat n) => BVar s (L n 1) -> BVar s (R n)
- tr :: (Reifies s W, Transposable m mt, Transposable mt m, Num m, Num mt) => BVar s m -> BVar s mt
- eye :: KnownNat n => Sq n
- diag :: (Reifies s W, KnownNat n) => BVar s (R n) -> BVar s (Sq n)
- matrix :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => [BVar s ℝ] -> BVar s (L m n)
- type ℂ = Complex Double
- data C (n :: Nat) :: Nat -> *
- data M (m :: Nat) (n :: Nat) :: Nat -> Nat -> *
- 𝑖 :: Sized ℂ s c => s
- (<>) :: (Reifies s W, KnownNat m, KnownNat k, KnownNat n) => BVar s (L m k) -> BVar s (L k n) -> BVar s (L m n)
- (#>) :: (Reifies s W, KnownNat m, KnownNat n) => BVar s (L m n) -> BVar s (R n) -> BVar s (R m)
- (<.>) :: (Reifies s W, KnownNat n) => BVar s (R n) -> BVar s (R n) -> BVar s ℝ
- svd :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => BVar s (L m n) -> BVar s (R n)
- svd_ :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => BVar s (L m n) -> (BVar s (L m m), BVar s (R n), BVar s (L n n))
- class Eigen m l v | m -> l, m -> v
- eigensystem :: forall n s. (Reifies s W, KnownNat n) => BVar s (Sym n) -> (BVar s (R n), BVar s (L n n))
- eigenvalues :: forall n s. (Reifies s W, KnownNat n) => BVar s (Sym n) -> BVar s (R n)
- chol :: forall n s. (Reifies s W, KnownNat n) => BVar s (Sym n) -> BVar s (Sq n)
- class Normed a
- norm_0 :: (Reifies s W, Normed a, Num a) => BVar s a -> BVar s ℝ
- norm_1V :: (Reifies s W, KnownNat n) => BVar s (R n) -> BVar s ℝ
- norm_1M :: (Reifies s W, KnownNat n, KnownNat m) => BVar s (L n m) -> BVar s ℝ
- norm_2V :: (Reifies s W, KnownNat n) => BVar s (R n) -> BVar s ℝ
- norm_2M :: (Reifies s W, KnownNat n, KnownNat m) => BVar s (L n m) -> BVar s ℝ
- norm_InfV :: (Reifies s W, KnownNat n) => BVar s (R n) -> BVar s ℝ
- norm_InfM :: (Reifies s W, KnownNat n, KnownNat m) => BVar s (L n m) -> BVar s ℝ
- mean :: (Reifies s W, KnownNat n, 1 <= n) => BVar s (R n) -> BVar s ℝ
- meanCov :: forall m n s. (Reifies s W, KnownNat n, KnownNat m, 1 <= m) => BVar s (L m n) -> (BVar s (R n), BVar s (Sym n))
- meanL :: forall m n s. (Reifies s W, KnownNat n, KnownNat m, 1 <= m) => BVar s (L m n) -> BVar s (R n)
- cov :: forall m n s. (Reifies s W, KnownNat n, KnownNat m, 1 <= m) => BVar s (L m n) -> BVar s (Sym n)
- class Disp t where
- class Domain field (vec :: Nat -> *) (mat :: Nat -> Nat -> *) | mat -> vec field, vec -> mat field, field -> mat vec
- mul :: (Reifies s W, KnownNat m, KnownNat k, KnownNat n, Domain field vec mat, Num (mat m k), Num (mat k n), Num (mat m n), Transposable (mat m k) (mat k m), Transposable (mat k n) (mat n k)) => BVar s (mat m k) -> BVar s (mat k n) -> BVar s (mat m n)
- app :: (Reifies s W, KnownNat m, KnownNat n, Domain field vec mat, Num (mat m n), Num (vec n), Num (vec m), Transposable (mat m n) (mat n m)) => BVar s (mat m n) -> BVar s (vec n) -> BVar s (vec m)
- dot :: (Reifies s W, KnownNat n, Domain field vec mat, Sized field (vec n) d, Num (vec n)) => BVar s (vec n) -> BVar s (vec n) -> BVar s field
- cross :: (Reifies s W, Domain field vec mat, Num (vec 3)) => BVar s (vec 3) -> BVar s (vec 3) -> BVar s (vec 3)
- diagR :: forall m n k field vec mat s. (Reifies s W, Domain field vec mat, Num (vec k), Num (mat m n), KnownNat m, KnownNat n, KnownNat k, Container Vector field, Sized field (mat m n) Matrix, Sized field (vec k) Vector) => BVar s field -> BVar s (vec k) -> BVar s (mat m n)
- vmap :: (Reifies s W, KnownNat n) => (BVar s ℝ -> BVar s ℝ) -> BVar s (R n) -> BVar s (R n)
- vmap' :: (Reifies s W, Num (vec n), Storable field, Sized field (vec n) Vector) => (forall s'. Reifies s' W => BVar s' field -> BVar s' field) -> BVar s (vec n) -> BVar s (vec n)
- dvmap :: (Reifies s W, KnownNat n, Domain field vec mat, Num (vec n), Num field) => (forall s'. Reifies s' W => BVar s' field -> BVar s' field) -> BVar s (vec n) -> BVar s (vec n)
- mmap :: (Reifies s W, KnownNat n, KnownNat m) => (BVar s ℝ -> BVar s ℝ) -> BVar s (L n m) -> BVar s (L n m)
- mmap' :: forall n m mat field s. (Reifies s W, KnownNat m, Num (mat n m), Sized field (mat n m) Matrix, Element field) => (forall s'. Reifies s' W => BVar s' field -> BVar s' field) -> BVar s (mat n m) -> BVar s (mat n m)
- dmmap :: (Reifies s W, KnownNat n, KnownNat m, Domain field vec mat, Num (mat n m), Num field) => (forall s'. Reifies s' W => BVar s' field -> BVar s' field) -> BVar s (mat n m) -> BVar s (mat n m)
- outer :: (Reifies s W, KnownNat m, KnownNat n, Domain field vec mat, Transposable (mat n m) (mat m n), Num (vec n), Num (vec m), Num (mat n m)) => BVar s (vec n) -> BVar s (vec m) -> BVar s (mat n m)
- zipWithVector :: (Reifies s W, KnownNat n) => (BVar s ℝ -> BVar s ℝ -> BVar s ℝ) -> BVar s (R n) -> BVar s (R n) -> BVar s (R n)
- zipWithVector' :: (Reifies s W, Num (vec n), Storable field, Sized field (vec n) Vector) => (forall s'. Reifies s' W => BVar s' field -> BVar s' field -> BVar s' field) -> BVar s (vec n) -> BVar s (vec n) -> BVar s (vec n)
- dzipWithVector :: (Reifies s W, KnownNat n, Domain field vec mat, Num (vec n), Num field) => (forall s'. Reifies s' W => BVar s' field -> BVar s' field -> BVar s' field) -> BVar s (vec n) -> BVar s (vec n) -> BVar s (vec n)
- det :: (Reifies s W, KnownNat n, Num (mat n n), Domain field vec mat, Sized field (mat n n) d, Transposable (mat n n) (mat n n)) => BVar s (mat n n) -> BVar s field
- invlndet :: forall n mat field vec d s. (Reifies s W, KnownNat n, Num (mat n n), Domain field vec mat, Sized field (mat n n) d, Transposable (mat n n) (mat n n), Backprop field, Backprop (mat n n)) => BVar s (mat n n) -> (BVar s (mat n n), (BVar s field, BVar s field))
- lndet :: forall n mat field vec d s. (Reifies s W, KnownNat n, Num (mat n n), Domain field vec mat, Sized field (mat n n) d, Transposable (mat n n) (mat n n)) => BVar s (mat n n) -> BVar s field
- inv :: (Reifies s W, KnownNat n, Num (mat n n), Domain field vec mat, Transposable (mat n n) (mat n n)) => BVar s (mat n n) -> BVar s (mat n n)
- toRows :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => BVar s (L m n) -> Vector m (BVar s (R n))
- toColumns :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => BVar s (L m n) -> Vector n (BVar s (R m))
- fromRows :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => Vector m (BVar s (R n)) -> BVar s (L m n)
- fromColumns :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => Vector n (BVar s (R m)) -> BVar s (L m n)
- konst :: forall t s d q. (Reifies q W, Sized t s d, Container d t, Num s) => BVar q t -> BVar q s
- sumElements :: forall t s d q. (Reifies q W, Sized t s d, Container d t, Num s) => BVar q s -> BVar q t
- extractV :: forall t s q. (Sized t s Vector, Konst t Int Vector, Container Vector t, Backprop t, Backprop s, Reifies q W) => BVar q s -> BVar q (Vector t)
- extractM :: forall t s q. (Sized t s Matrix, Backprop s, Konst t (Int, Int) Matrix, Container Matrix t, Num (Matrix t), Reifies q W) => BVar q s -> BVar q (Matrix t)
- create :: forall t s d q. (Reifies q W, Sized t s d, Backprop s, Num (d t), Backprop (d t)) => BVar q (d t) -> Maybe (BVar q s)
- class Diag m d | m -> d
- takeDiag :: (Reifies s W, KnownNat n, Diag (mat n n) (vec n), Domain field vec mat, Num (vec n), Num (mat n n), Num field) => BVar s (mat n n) -> BVar s (vec n)
- data Sym (n :: Nat) :: Nat -> *
- sym :: (Reifies s W, KnownNat n) => BVar s (Sq n) -> BVar s (Sym n)
- mTm :: (Reifies s W, KnownNat m, KnownNat n) => BVar s (L m n) -> BVar s (Sym n)
- unSym :: (Reifies s W, KnownNat n) => BVar s (Sym n) -> BVar s (Sq n)
- (<·>) :: (Reifies s W, KnownNat n) => BVar s (R n) -> BVar s (R n) -> BVar s ℝ
- data BVar s a :: Type -> * -> *
- class Reifies k (s :: k) a | s -> a
- data W :: *
Vector
(&) :: (Reifies s W, KnownNat n, 1 <= n, KnownNat (n + 1)) => BVar s (R n) -> BVar s ℝ -> BVar s (R (n + 1)) infixl 4 Source #
(#) :: (Reifies s W, KnownNat n, KnownNat m) => BVar s (R n) -> BVar s (R m) -> BVar s (R (n + m)) infixl 4 Source #
split :: forall p n s. (Reifies s W, KnownNat p, KnownNat n, p <= n) => BVar s (R n) -> (BVar s (R p), BVar s (R (n - p))) Source #
headTail :: (Reifies s W, KnownNat n, 1 <= n) => BVar s (R n) -> (BVar s ℝ, BVar s (R (n - 1))) Source #
vector :: forall n s. (Reifies s W, KnownNat n) => Vector n (BVar s ℝ) -> BVar s (R n) Source #
Potentially extremely bad for anything but short lists!!!
Matrix
data L (m :: Nat) (n :: Nat) :: Nat -> Nat -> * #
Instances
| Domain ℝ R L | |
| (KnownNat m, KnownNat n) => Sized ℝ (L m n) Matrix | |
| KnownNat n => Eigen (Sq n) (C n) (M n n) | |
| KnownNat n => Eigen (Sym n) (R n) (L n n) | |
| (KnownNat n, KnownNat m) => Floating (L n m) | |
| (KnownNat n, KnownNat m) => Fractional (L n m) | |
| (KnownNat n, KnownNat m) => Num (L n m) | |
| (KnownNat m, KnownNat n) => Show (L m n) | |
| Generic (L m n) | |
| (KnownNat n, KnownNat m) => Binary (L m n) | |
| NFData (L n m) | |
| (KnownNat m, KnownNat n) => Disp (L m n) | |
| (KnownNat m, KnownNat n) => Additive (L m n) | |
| KnownNat n => Diag (L n n) (R n) | |
| (KnownNat n, KnownNat m) => Transposable (L m n) (L n m) | |
| type Rep (L m n) | |
(|||) :: (Reifies s W, KnownNat c, KnownNat r1, KnownNat (r1 + r2)) => BVar s (L c r1) -> BVar s (L c r2) -> BVar s (L c (r1 + r2)) infixl 3 Source #
(===) :: (Reifies s W, KnownNat c, KnownNat r1, KnownNat (r1 + r2)) => BVar s (L r1 c) -> BVar s (L r2 c) -> BVar s (L (r1 + r2) c) infixl 2 Source #
splitRows :: forall p m n s. (Reifies s W, KnownNat p, KnownNat m, KnownNat n, p <= m) => BVar s (L m n) -> (BVar s (L p n), BVar s (L (m - p) n)) Source #
splitCols :: forall p m n s. (Reifies s W, KnownNat p, KnownNat m, KnownNat n, KnownNat (n - p), p <= n) => BVar s (L m n) -> (BVar s (L m p), BVar s (L m (n - p))) Source #
tr :: (Reifies s W, Transposable m mt, Transposable mt m, Num m, Num mt) => BVar s m -> BVar s mt Source #
matrix :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => [BVar s ℝ] -> BVar s (L m n) Source #
Potentially extremely bad for anything but short lists!!!
Complex
data M (m :: Nat) (n :: Nat) :: Nat -> Nat -> * #
Instances
| Domain ℂ C M | |
| (KnownNat m, KnownNat n) => Sized ℂ (M m n) Matrix | |
| KnownNat n => Eigen (Sq n) (C n) (M n n) | |
| (KnownNat n, KnownNat m) => Floating (M n m) | |
| (KnownNat n, KnownNat m) => Fractional (M n m) | |
| (KnownNat n, KnownNat m) => Num (M n m) | |
| (KnownNat m, KnownNat n) => Show (M m n) | |
| Generic (M m n) | |
| NFData (M n m) | |
| (KnownNat m, KnownNat n) => Disp (M m n) | |
| (KnownNat m, KnownNat n) => Additive (M m n) | |
| KnownNat n => Diag (M n n) (C n) | |
| (KnownNat n, KnownNat m) => Transposable (M m n) (M n m) | |
| type Rep (M m n) | |
Products
(<>) :: (Reifies s W, KnownNat m, KnownNat k, KnownNat n) => BVar s (L m k) -> BVar s (L k n) -> BVar s (L m n) infixr 8 Source #
Matrix product
(#>) :: (Reifies s W, KnownNat m, KnownNat n) => BVar s (L m n) -> BVar s (R n) -> BVar s (R m) infixr 8 Source #
Matrix-vector product
(<.>) :: (Reifies s W, KnownNat n) => BVar s (R n) -> BVar s (R n) -> BVar s ℝ infixr 8 Source #
Dot product
Factorizations
svd :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => BVar s (L m n) -> BVar s (R n) Source #
Can only get the singular values, for now. Let me know if you find an algorithm that can compute the gradients based on differentials for the other matricies!
svd_ :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => BVar s (L m n) -> (BVar s (L m m), BVar s (R n), BVar s (L n n)) Source #
Version of svd that returns the full SVD, but if you attempt to find
the gradient, it will fail at runtime if you ever use U or V.
eigensystem :: forall n s. (Reifies s W, KnownNat n) => BVar s (Sym n) -> (BVar s (R n), BVar s (L n n)) Source #
NOTE The gradient is not necessarily symmetric! The gradient is not
meant to be retireved directly; insteadl, eigenvalues is meant to be
used as a part of a larger computation, and the gradient as an
intermediate step.
eigenvalues :: forall n s. (Reifies s W, KnownNat n) => BVar s (Sym n) -> BVar s (R n) Source #
NOTE The gradient is not necessarily symmetric! The gradient is not
meant to be retireved directly; insteadl, eigenvalues is meant to be
used as a part of a larger computation, and the gradient as an
intermediate step.
chol :: forall n s. (Reifies s W, KnownNat n) => BVar s (Sym n) -> BVar s (Sq n) Source #
Algorithm from https://arxiv.org/abs/1602.07527
The paper also suggests a potential imperative algorithm that might help. Need to benchmark to see what is best.
NOTE The gradient is not necessarily symmetric! The gradient is not
meant to be retireved directly; insteadl, eigenvalues is meant to be
used as a part of a larger computation, and the gradient as an
intermediate step.
Norms
p-norm for vectors, operator norm for matrices
norm_1M :: (Reifies s W, KnownNat n, KnownNat m) => BVar s (L n m) -> BVar s ℝ Source #
Maximum norm_1 of columns
norm_2V :: (Reifies s W, KnownNat n) => BVar s (R n) -> BVar s ℝ Source #
Square root of sum of squares
Be aware that gradient diverges when the norm is zero
norm_2M :: (Reifies s W, KnownNat n, KnownNat m) => BVar s (L n m) -> BVar s ℝ Source #
Maximum singular value
norm_InfM :: (Reifies s W, KnownNat n, KnownNat m) => BVar s (L n m) -> BVar s ℝ Source #
Maximum norm_1 of rows
Misc
meanCov :: forall m n s. (Reifies s W, KnownNat n, KnownNat m, 1 <= m) => BVar s (L m n) -> (BVar s (R n), BVar s (Sym n)) Source #
meanL :: forall m n s. (Reifies s W, KnownNat n, KnownNat m, 1 <= m) => BVar s (L m n) -> BVar s (R n) Source #
meanCov, but if you know you won't use the covariance.
cov :: forall m n s. (Reifies s W, KnownNat n, KnownNat m, 1 <= m) => BVar s (L m n) -> BVar s (Sym n) Source #
cov, but if you know you won't use the covariance.
Minimal complete definition
Domain
class Domain field (vec :: Nat -> *) (mat :: Nat -> Nat -> *) | mat -> vec field, vec -> mat field, field -> mat vec #
mul :: (Reifies s W, KnownNat m, KnownNat k, KnownNat n, Domain field vec mat, Num (mat m k), Num (mat k n), Num (mat m n), Transposable (mat m k) (mat k m), Transposable (mat k n) (mat n k)) => BVar s (mat m k) -> BVar s (mat k n) -> BVar s (mat m n) Source #
app :: (Reifies s W, KnownNat m, KnownNat n, Domain field vec mat, Num (mat m n), Num (vec n), Num (vec m), Transposable (mat m n) (mat n m)) => BVar s (mat m n) -> BVar s (vec n) -> BVar s (vec m) Source #
dot :: (Reifies s W, KnownNat n, Domain field vec mat, Sized field (vec n) d, Num (vec n)) => BVar s (vec n) -> BVar s (vec n) -> BVar s field Source #
cross :: (Reifies s W, Domain field vec mat, Num (vec 3)) => BVar s (vec 3) -> BVar s (vec 3) -> BVar s (vec 3) Source #
Arguments
| :: (Reifies s W, Domain field vec mat, Num (vec k), Num (mat m n), KnownNat m, KnownNat n, KnownNat k, Container Vector field, Sized field (mat m n) Matrix, Sized field (vec k) Vector) | |
| => BVar s field | default value |
| -> BVar s (vec k) | diagonal |
| -> BVar s (mat m n) |
Create matrix with diagonal, and fill with default entries
vmap :: (Reifies s W, KnownNat n) => (BVar s ℝ -> BVar s ℝ) -> BVar s (R n) -> BVar s (R n) Source #
Note: if possible, use the potentially much more performant vmap'.
vmap' :: (Reifies s W, Num (vec n), Storable field, Sized field (vec n) Vector) => (forall s'. Reifies s' W => BVar s' field -> BVar s' field) -> BVar s (vec n) -> BVar s (vec n) Source #
dvmap :: (Reifies s W, KnownNat n, Domain field vec mat, Num (vec n), Num field) => (forall s'. Reifies s' W => BVar s' field -> BVar s' field) -> BVar s (vec n) -> BVar s (vec n) Source #
Note: Potentially less performant than vmap'.
mmap :: (Reifies s W, KnownNat n, KnownNat m) => (BVar s ℝ -> BVar s ℝ) -> BVar s (L n m) -> BVar s (L n m) Source #
Note: if possible, use the potentially much more performant mmap'.
mmap' :: forall n m mat field s. (Reifies s W, KnownNat m, Num (mat n m), Sized field (mat n m) Matrix, Element field) => (forall s'. Reifies s' W => BVar s' field -> BVar s' field) -> BVar s (mat n m) -> BVar s (mat n m) Source #
dmmap :: (Reifies s W, KnownNat n, KnownNat m, Domain field vec mat, Num (mat n m), Num field) => (forall s'. Reifies s' W => BVar s' field -> BVar s' field) -> BVar s (mat n m) -> BVar s (mat n m) Source #
Note: Potentially less performant than mmap'.
outer :: (Reifies s W, KnownNat m, KnownNat n, Domain field vec mat, Transposable (mat n m) (mat m n), Num (vec n), Num (vec m), Num (mat n m)) => BVar s (vec n) -> BVar s (vec m) -> BVar s (mat n m) Source #
zipWithVector :: (Reifies s W, KnownNat n) => (BVar s ℝ -> BVar s ℝ -> BVar s ℝ) -> BVar s (R n) -> BVar s (R n) -> BVar s (R n) Source #
Note: if possible, use the potentially much more performant
zipWithVector'.
zipWithVector' :: (Reifies s W, Num (vec n), Storable field, Sized field (vec n) Vector) => (forall s'. Reifies s' W => BVar s' field -> BVar s' field -> BVar s' field) -> BVar s (vec n) -> BVar s (vec n) -> BVar s (vec n) Source #
dzipWithVector :: (Reifies s W, KnownNat n, Domain field vec mat, Num (vec n), Num field) => (forall s'. Reifies s' W => BVar s' field -> BVar s' field -> BVar s' field) -> BVar s (vec n) -> BVar s (vec n) -> BVar s (vec n) Source #
A version of zipWithVector' that is potentially less performant but
is based on zipWithVector from Domain.
det :: (Reifies s W, KnownNat n, Num (mat n n), Domain field vec mat, Sized field (mat n n) d, Transposable (mat n n) (mat n n)) => BVar s (mat n n) -> BVar s field Source #
invlndet :: forall n mat field vec d s. (Reifies s W, KnownNat n, Num (mat n n), Domain field vec mat, Sized field (mat n n) d, Transposable (mat n n) (mat n n), Backprop field, Backprop (mat n n)) => BVar s (mat n n) -> (BVar s (mat n n), (BVar s field, BVar s field)) Source #
The inverse and the natural log of the determinant together. If you
know you don't need the inverse, it is best to use lndet.
lndet :: forall n mat field vec d s. (Reifies s W, KnownNat n, Num (mat n n), Domain field vec mat, Sized field (mat n n) d, Transposable (mat n n) (mat n n)) => BVar s (mat n n) -> BVar s field Source #
The natural log of the determinant.
inv :: (Reifies s W, KnownNat n, Num (mat n n), Domain field vec mat, Transposable (mat n n) (mat n n)) => BVar s (mat n n) -> BVar s (mat n n) Source #
Conversions
toRows :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => BVar s (L m n) -> Vector m (BVar s (R n)) Source #
toColumns :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => BVar s (L m n) -> Vector n (BVar s (R m)) Source #
fromRows :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => Vector m (BVar s (R n)) -> BVar s (L m n) Source #
fromColumns :: forall m n s. (Reifies s W, KnownNat m, KnownNat n) => Vector n (BVar s (R m)) -> BVar s (L m n) Source #
Misc Operations
konst :: forall t s d q. (Reifies q W, Sized t s d, Container d t, Num s) => BVar q t -> BVar q s Source #
sumElements :: forall t s d q. (Reifies q W, Sized t s d, Container d t, Num s) => BVar q s -> BVar q t Source #
extractV :: forall t s q. (Sized t s Vector, Konst t Int Vector, Container Vector t, Backprop t, Backprop s, Reifies q W) => BVar q s -> BVar q (Vector t) Source #
If there are extra items in the total derivative, they are dropped. If there are missing items, they are treated as zero.
extractM :: forall t s q. (Sized t s Matrix, Backprop s, Konst t (Int, Int) Matrix, Container Matrix t, Num (Matrix t), Reifies q W) => BVar q s -> BVar q (Matrix t) Source #
If there are extra items in the total derivative, they are dropped. If there are missing items, they are treated as zero.
create :: forall t s d q. (Reifies q W, Sized t s d, Backprop s, Num (d t), Backprop (d t)) => BVar q (d t) -> Maybe (BVar q s) Source #
Minimal complete definition
takeDiag :: (Reifies s W, KnownNat n, Diag (mat n n) (vec n), Domain field vec mat, Num (vec n), Num (mat n n), Num field) => BVar s (mat n n) -> BVar s (vec n) Source #
sym :: (Reifies s W, KnownNat n) => BVar s (Sq n) -> BVar s (Sym n) Source #
\[ \frac{1}{2} (M + M^T) \]
mTm :: (Reifies s W, KnownNat m, KnownNat n) => BVar s (L m n) -> BVar s (Sym n) Source #
\[ M^T M \]
unSym :: (Reifies s W, KnownNat n) => BVar s (Sym n) -> BVar s (Sq n) Source #
Warning: the gradient is going necessarily symmetric, and so is not meant to be used directly. Rather, it is meant to be used in the middle (or at the end) of a longer computation.
(<·>) :: (Reifies s W, KnownNat n) => BVar s (R n) -> BVar s (R n) -> BVar s ℝ infixr 8 Source #
Unicode synonym for <.>>
Backprop types re-exported
Re-exported for convenience.
Since: 0.1.1.0
data BVar s a :: Type -> * -> * #
A BVar s aa that can be "backpropagated".
Functions referring to BVars are tracked by the library and can be
automatically differentiated to get their gradients and results.
For simple numeric values, you can use its Num, Fractional, and
Floating instances to manipulate them as if they were the numbers they
represent.
If a contains items, the items can be accessed and extracted using
lenses. A Lens' b aa inside a b, using
^^. (viewVar):
(^.) :: a ->Lens'a b -> b (^^.) ::BVars a ->Lens'a b ->BVars b
There is also ^^? (previewVar), to use a Prism' or Traversal' to
extract a target that may or may not be present (which can implement
pattern matching), ^^.. (toListOfVar) to use a Traversal' to
extract all targets inside a BVar, and .~~ (setVar) to set and
update values inside a BVar.
For more complex operations, libraries can provide functions on BVars
using liftOp and related functions. This is how you can create
primitive functions that users can use to manipulate your library's
values.
For example, the hmatrix library has a matrix-vector multiplication
function, #> :: L m n -> R n -> L m.
A library could instead provide a function #> :: , which the user can then use to manipulate their
BVar (L m n) -> BVar
(R n) -> BVar (R m)BVars of L m ns and R ns, etc.
See Numeric.Backprop and documentation for liftOp for more
information.
Instances
| Eq a => Eq (BVar s a) | Compares the values inside the Since: 0.1.5.0 |
| (Floating a, Reifies Type s W) => Floating (BVar s a) | |
| (Fractional a, Reifies Type s W) => Fractional (BVar s a) | |
| (Num a, Reifies Type s W) => Num (BVar s a) | |
| Ord a => Ord (BVar s a) | Compares the values inside the Since: 0.1.5.0 |
| NFData a => NFData (BVar s a) | This will force the value inside, as well. |
class Reifies k (s :: k) a | s -> a #
Minimal complete definition
Instances
| KnownNat n => Reifies Nat n Integer | |
| KnownSymbol n => Reifies Symbol n String | |
| Reifies * Z Int | |
| Reifies * n Int => Reifies * (D n) Int | |
| Reifies * n Int => Reifies * (SD n) Int | |
| Reifies * n Int => Reifies * (PD n) Int | |
| (B * b0, B * b1, B * b2, B * b3, B * b4, B * b5, B * b6, B * b7, (~) * w0 (W b0 b1 b2 b3), (~) * w1 (W b4 b5 b6 b7)) => Reifies * (Stable w0 w1 a) a | |