nerf-0.5.2: Nerf, the named entity recognition tool based on linear-chain CRFs
NLP.Nerf.Tokenize
Contents
Description
The module implements the tokenization used within Nerf and some other tokenization-related stuff.
Synopsis
tokenize :: String -> [String]Source
Tokenize sentence using the default tokenizer.
class Word a whereSource
A class of objects which can be converted to String.
String
Methods
word :: a -> StringSource
Instances
syncSource
Arguments
NE forest
New tokenization
Resulting NE forest
Synchronize the list of NE trees with the new tokenization.