goal-graphical-0.20: Optimization of latent variable and dynamical models with Goal

Goal.Graphical.Learning

Description

A collection of algorithms for optimizing harmoniums.

Synopsis

# Expectation Maximization

expectationMaximization :: (DuallyFlatExponentialFamily (AffineHarmonium f y x z w), ExponentialFamily z, Map Natural f x y, Bilinear f y x, Translation z y, Translation w x, LegendreExponentialFamily w) => Sample z -> (Natural # AffineHarmonium f y x z w) -> Natural # AffineHarmonium f y x z w Source #

A single iteration of EM for Harmonium based models.

expectationMaximizationAscent :: (LegendreExponentialFamily (AffineHarmonium f y x z w), ExponentialFamily z, Map Natural f x y, Bilinear f y x, Translation z y, Translation w x, LegendreExponentialFamily w) => Double -> GradientPursuit -> Sample z -> (Natural # AffineHarmonium f y x z w) -> [Natural # AffineHarmonium f y x z w] Source #

Ascent of the EM objective on harmoniums for when the expectation step can't be computed in closed-form. The convergent harmonium distribution of the output harmonium-list is the result of 1 iteration of the EM algorithm.

Arguments

 :: (ExponentialFamily z, Map Natural f x y, Manifold w, Map Natural f y x, Translation z y, Translation w x, SamplePoint y ~ SamplePoint z, SamplePoint w ~ SamplePoint x, ExponentialFamily y, Generative Natural w, ExponentialFamily x, Generative Natural z, Manifold (AffineHarmonium f y x z w), Bilinear f y x, LegendreExponentialFamily w) => Double -> Int -> Int -> GradientPursuit -> Sample z Observations -> (Natural # AffineHarmonium f y x z w) Current Harmonium -> Chain Random (Natural # AffineHarmonium f y x z w) Harmonium Chain

Ascent of the EM objective on harmoniums for when the expectation step can't be computed in closed-form. The convergent harmonium distribution of the output harmonium-list is the result of 1 iteration of the EM algorithm.

latentProcessExpectationMaximization :: (ConjugatedLikelihood g x x w w, ConjugatedLikelihood f y x z w, Transition Natural Mean w, Transition Natural Mean (AffineHarmonium g x x w w), Transition Mean Natural w, Transition Mean Natural (AffineHarmonium f y x z w), Transition Mean Natural (AffineHarmonium g x x w w), Manifold (AffineHarmonium g x x w w), Bilinear g x x, Map Natural f x y, Bilinear f y x, SamplePoint y ~ SamplePoint z) => Observations (LatentProcess f g y x z w) -> (Natural # LatentProcess f g y x z w) -> Natural # LatentProcess f g y x z w Source #

Direct expectation maximization for LatentProcesses.

latentProcessExpectationMaximizationAscent :: (ConjugatedLikelihood g x x w w, ConjugatedLikelihood f y x z w, DuallyFlatExponentialFamily w, LegendreExponentialFamily (AffineHarmonium f y x z w), LegendreExponentialFamily (AffineHarmonium g x x w w), Bilinear g x x, Map Natural f x y, Bilinear f y x, SamplePoint y ~ SamplePoint z) => Double -> Int -> GradientPursuit -> [Sample z] -> (Natural # LatentProcess f g y x z w) -> Natural # LatentProcess f g y x z w Source #

Expectation maximization for LatentProcesses approximated through gradient ascent.

# Differentials

Arguments

 :: (Map Natural f y x, LegendreExponentialFamily z, SamplePoint w ~ SamplePoint x, Translation z y, ExponentialFamily x, ExponentialFamily w, Generative Natural w) => Int -> (Natural # AffineHarmonium f y x z w) Harmonium -> (Natural # w) Model Distribution -> Random (Mean # w) Differential Estimate

The differential of the dual relative entropy. Minimizing this results in the information projection of the model against the marginal distribution of the given harmonium. This is more efficient than the generic version.

Arguments

 :: (Generative Natural z, ExponentialFamily z, Translation w x, Generative Natural w, ExponentialFamily y, Translation z y, LegendreExponentialFamily w, Bilinear f y x, Map Natural f x y, Map Natural f y x, SamplePoint y ~ SamplePoint z, SamplePoint x ~ SamplePoint w, ExponentialFamily x) => Int The number of contrastive divergence steps -> Sample z The initial states of the Gibbs chains -> (Natural # AffineHarmonium f y x z w) The harmonium -> Random (Mean # AffineHarmonium f y x z w) The gradient estimate

Contrastive divergence on harmoniums (Hinton, 2019).