Safe Haskell  None 

Language  Haskell98 
A continuoustime Markov decision process (CTMDP) is an MDP where transitions between states take a random amount of time. Each transition time is assumed to be exponentially distributed with an action and statedependent transition rate.
The record accessors of the CTMDP
type conflict with those of the
MDP
type, so either import only the mkCTMDP
and uniformize
functions or import this module qualified.
 data CTMDP a b t = CTMDP {
 _states :: Vector a
 _actions :: Vector b
 _fixedCosts :: Vector (Vector t)
 _rateCosts :: Vector (Vector t)
 _rates :: Vector (Vector t)
 _trans :: Vector (Vector (Vector t))
 _discount :: t
 _actionSet :: Vector (Vector Int)
 mkCTMDP :: Eq b => [a] > [b] > Transitions a b t > Rates a b t > Costs a b t > Costs a b t > ActionSet a b > t > CTMDP a b t
 type Rates a b t = b > a > t
 uniformize :: (Ord t, Fractional t) => CTMDP a b t > MDP a b t
Documentation
A Continuoustime Markov decision process.
A CTMDP is a continuoustime analog of an MDP. In a CTMDP each stage takes a variable amount of time. Each stage lasts an expontially distributed amount of time characterized by a state and actiondependent rate parameter. Instead of simply having costs associated with a state and an action, the costs of a CTMDP are broken up into fixed and rate costs. Fixed costs are incured as an action are chosen, while rate costs are paid for the duration of the stage.
Here the type variable a
represents the type of the states, b
represents the type of the actions, and t
represents the numeric
type used in computations. Generally choosing t
to be a Double is
fine, although there is no reason a higherprecision type cannot be
used.
This type should not be constructed directly; use the mkCTMDP
constructor instead.
CTMDP  

:: Eq b  
=> [a]  The state space 
> [b]  The action space 
> Transitions a b t  The transition probabilities 
> Rates a b t  The transition rates 
> Costs a b t  The actiondependent fixed costs 
> Costs a b t  The actiondependent rate costs 
> ActionSet a b  The statedependent actions 
> t  The discount factor in (0, 1] 
> CTMDP a b t  The resulting CTMDP 
Create a CTMDP.
type Rates a b t = b > a > t Source #
A function mapping an action and a state to a transition rate.
uniformize :: (Ord t, Fractional t) => CTMDP a b t > MDP a b t Source #
Convert a CTMDP into an MDP.