Skylighting.Tokenizer

Documentation

tokenize :: TokenizerConfig -> Syntax -> Text -> Either String [SourceLine]

Tokenize some text using Syntax.

data TokenizerConfig

Configuration options for tokenize.

Constructors

TokenizerConfig

Fields

• syntaxMap :: SyntaxMap

  Syntax map to use

• traceOutput :: Bool

  Generate trace output for debugging

Instances

Show TokenizerConfig

Methods

showsPrec :: Int -> TokenizerConfig -> ShowS

show :: TokenizerConfig -> String

showList :: [TokenizerConfig] -> ShowS