Language.GroteTrap.Lexer

Deriving a lexer from a Language.

Types

data Token

The tokenizer produces a list of tokens.

Constructors

TId String

TInt Int

TOperator String

TFunction String

TOpen

TClose

TComma

TWhite Int

Instances

Eq Token

Show Token

type TokenPos = (Pos, Token)

Tokenizing

run :: Monad m => String -> GenParser tok () a -> [tok] -> m a

run sourceName p input runs the specified parser on the input, returning the result in a monad.

tokenize :: Monad m => Language a -> String -> m [TokenPos]

When given a language, transforms a list of characters into a list of tokens.

isWhite :: Token -> Bool

Whether the token is whitespace.