Safe Haskell | None |
---|
The module provides first-order, linear-chain conditional random fields (CRFs).
Important feature of the implemented flavour of CRFs is that transition features which are not included in the CRF model are considered to have probability of 0. It is particularly useful when the training material determines the set of possible label transitions (e.g. when using the IOB encoding method). Furthermore, this design decision makes the implementation much faster for sparse datasets.
- type Word a = Set a
- type Sent a = [Word a]
- data Dist a
- mkDist :: Ord a => [(a, Double)] -> Dist a
- type WordL a b = (Word a, Dist b)
- annotate :: Word a -> b -> WordL a b
- type SentL a b = [WordL a b]
- data CRF a b = CRF {}
- train :: (Ord a, Ord b) => SgdArgs -> IO [SentL a b] -> Maybe (b, IO [SentL a b]) -> ([(Xs, Ys)] -> [Feature]) -> IO (CRF a b)
- tag :: (Ord a, Ord b) => CRF a b -> Sent a -> [b]
- hiddenFeats :: [(Xs, Ys)] -> [Feature]
- presentFeats :: [(Xs, Ys)] -> [Feature]
Data types
A probability distribution defined over elements of type a. All elements not included in the map have probability equal to 0.
type WordL a b = (Word a, Dist b)Source
A WordL is a labeled word, i.e. a word with probability distribution defined over labels.
CRF
A conditional random field model with additional codec used for data encoding.
Training
:: (Ord a, Ord b) | |
=> SgdArgs | Args for SGD |
-> IO [SentL a b] | Training data |
-> Maybe (b, IO [SentL a b]) | Default label and evalation data |
-> ([(Xs, Ys)] -> [Feature]) | Feature selection |
-> IO (CRF a b) | Resulting model |
Train the CRF using the stochastic gradient descent method.
The resulting model will contain features extracted with
the user supplied extraction function.
You can use the functions provided by the Data.CRF.Chain1.Feature.Present
and Data.CRF.Chain1.Feature.Hidden modules for this purpose.
When the evaluation data IO
action is Just
, the iterative
training process will notify the user about the current accuracy
on the evaluation part every full iteration over the training part.
Tagging
tag :: (Ord a, Ord b) => CRF a b -> Sent a -> [b]Source
Determine the most probable label sequence within the context of the
given sentence using the model provided by the CRF
.