MaxEnt

The maximum entropy method, or MAXENT, is variational approach for computing probability distributions given a list of moment, or expected value, constraints.

Here are some links for background info. A good overview of applications: http://cmm.cit.nih.gov/maxent/letsgo.html On the idea of maximum entropy in general: http://en.wikipedia.org/wiki/Principle_of_maximum_entropy

Use this package to compute discrete maximum entropy distributions over a list of values and list of constraints.

Here is a the example from Probability the Logic of Science

  maxent ([1,2,3], [average 1.5])

Right [0.61, 0.26, 0.11]

The classic dice example

  maxent ([1,2,3,4,5,6], [average 4.5])

Right [.05, .07, 0.11, 0.16, 0.23, 0.34]

One can use different constraints besides the average value there.

As for why you want to maximize the entropy to find the probability constraint, I will say this for now. In the case of the average constraint it is a kin to choosing a integer partition with the most interger compositions. I doubt that makes any sense, but I will try to explain more with a blog post soon.

Documentation

type Constraint a = (ExpectationFunction a, a)

Constraint type. A function and the constant it equals.

Think of it as the pair (f, c) in the constraint

     Σ pₐ f(xₐ) = c

such that we are summing over all values .

For example, for a variance constraint the f would be (\x -> x*x) and c would be the variance.

type ExpectationFunction a = a -> a

A function that takes an index and value and returns a value. See average and variance for examples.

constraint :: Floating a => ExpectationFunction a -> a -> Constraint a

average :: Floating a => a -> Constraint a

variance :: Floating a => a -> Constraint a

maxent

Arguments

:: (forall a. Floating a => ([a], [Constraint a]))

A pair of values that the distributions is over and the constraints

-> Either (Result, Statistics) [Double]

Either the a discription of what wrong or the probability distribution

The main entry point for computing discrete maximum entropy distributions. Where the constraints are all moment constraints.

maxentLinear

Arguments

:: (forall a. Floating a => ([a], ([[a]], [a])))

The values and a matrix A and column vector b

-> Either (Result, Statistics) [Double]

Either the a discription of what wrong or the probability distribution

This is for the linear case Ax = b x in this situation is the vector of probablities.

For example.

   maxentLinear ([1,1,1], ([[0.85, 0.1, 0.05], [0.25, 0.5, 0.25], [0.05, 0.1, 0.85]], [0.29, 0.42, 0.29]))

Right [0.1, 0.8, 0.1]

To be honest I am not sure why I can't use the maxent version to solve this type of problem, but it doesn't work. I'm still learning

generalMaxent

Arguments

:: (forall a. Floating a => ([a], [(GeneralConstraint a, a)]))

A pair of values that the distributions is over and the constraints

-> Either (Result, Statistics) [Double]

Either the a discription of what wrong or the probability distribution

Most general solver This will solve the langrangian by added the constraint that the probabilities must add up to zero. This is the slowest but most flexible method.