Safe Haskell | None |
---|
- tag :: Tag t => Map Text t -> CaseSensitive -> [Sentence] -> [TaggedSentence t]
- tagSentence :: Tag t => Map Text t -> CaseSensitive -> Sentence -> TaggedSentence t
- mkTagger :: Tag t => Map Text t -> CaseSensitive -> Maybe (POSTagger t) -> POSTagger t
- taggerID :: ByteString
- readTagger :: Tag t => ByteString -> Maybe (POSTagger t) -> Either String (POSTagger t)
- data CaseSensitive
- = Sensitive
- | Insensitive
- protectTerms :: [Text] -> CaseSensitive -> Tokenizer
Documentation
tag :: Tag t => Map Text t -> CaseSensitive -> [Sentence] -> [TaggedSentence t]Source
tagSentence :: Tag t => Map Text t -> CaseSensitive -> Sentence -> TaggedSentence tSource
mkTagger :: Tag t => Map Text t -> CaseSensitive -> Maybe (POSTagger t) -> POSTagger tSource
Create a Literal Tagger using the specified back-off tagger as a fall-back, if one is specified.
This uses a tokenizer adapted from the tokenize
package for a
tokenizer, and Erik Kow's fullstop sentence segmenter as a sentence
splitter.
readTagger :: Tag t => ByteString -> Maybe (POSTagger t) -> Either String (POSTagger t)Source
deserialization for Literal Taggers. The serialization logic is in the posSerialize record of the POSTagger created in mkTagger.
data CaseSensitive Source
Boolean type to indicate case sensitivity for textual comparisons.
Read CaseSensitive | |
Show CaseSensitive | |
Generic CaseSensitive | |
Arbitrary CaseSensitive | |
Serialize CaseSensitive |
protectTerms :: [Text] -> CaseSensitive -> TokenizerSource
Create a tokenizer that protects the provided terms (to tokenize multi-word terms)