Data.YAML.Token

SPDX-License-Identifier: GPL-3.0

Tokenizer for the YAML 1.2 syntax as defined in http://yaml.org/spec/1.2/spec.html.

Documentation

tokenize :: ByteString -> Bool -> [Token]

tokenize input emit_unparsed converts the Unicode input (using the UTF-8, UTF-16 (LE or BE), or UTF-32 (LE or BE) encoding) to a list of Token according to the YAML 1.2 specification.

Errors are reported as tokens with Error :: Code, and the unparsed text following an error may be attached as a final Unparsed token (if the emit_unparsed argument is True). Note that tokens are available "immediately", allowing for streaming of large YAML files with memory requirements depending only on the YAML nesting level.

data Token

Parsed token.

Constructors

Token

Fields

• tByteOffset :: !Int

  0-base byte offset in stream.

• tCharOffset :: !Int

  0-base character offset in stream.

• tLine :: !Int

  1-based line number.

• tLineChar :: !Int

  0-based character in line.

• tCode :: !Code

  Specific token Code.

• tText :: !String

  Contained input chars, if any.

Instances

Show Token

Methods

showsPrec :: Int -> Token -> ShowS

show :: Token -> String

showList :: [Token] -> ShowS

data Code

Token codes.

Constructors

Bom

BOM, contains "TF8", "TF16LE", "TF32BE", etc.

Text

Content text characters.

Meta

Non-content (meta) text characters.

Break

Separation line break.

LineFeed

Line break normalized to content line feed.

LineFold

Line break folded to content space.

Indicator

Character indicating structure.

White

Separation white space.

Indent

Indentation spaces.

DirectivesEnd

Document start marker.

DocumentEnd

Document end marker.

BeginEscape

Begins escape sequence.

EndEscape

Ends escape sequence.

BeginComment

Begins comment.

EndComment

Ends comment.

BeginDirective

Begins directive.

EndDirective

Ends directive.

BeginTag

Begins tag.

EndTag

Ends tag.

BeginHandle

Begins tag handle.

EndHandle

Ends tag handle.

BeginAnchor

Begins anchor.

EndAnchor

Ends anchor.

BeginProperties

Begins node properties.

EndProperties

Ends node properties.

BeginAlias

Begins alias.

EndAlias

Ends alias.

BeginScalar

Begins scalar content.

EndScalar

Ends scalar content.

BeginSequence

Begins sequence content.

EndSequence

Ends sequence content.

BeginMapping

Begins mapping content.

EndMapping

Ends mapping content.

BeginPair

Begins mapping key:value pair.

EndPair

Ends mapping key:value pair.

BeginNode

Begins complete node.

EndNode

Ends complete node.

BeginDocument

Begins document.

EndDocument

Ends document.

BeginStream

Begins YAML stream.

EndStream

Ends YAML stream.

Error

Parsing error at this point.

Unparsed

Unparsed due to errors (or at end of test).

Detected

Detected parameter (for testing).

Instances

Eq Code

Methods

(==) :: Code -> Code -> Bool

(/=) :: Code -> Code -> Bool

Show Code

Methods

showsPrec :: Int -> Code -> ShowS

show :: Code -> String

showList :: [Code] -> ShowS