Monadic and General Iteratees: incremental input parsers, processors and transformers

- data StreamChunk c el => StreamG c el
- data IterateeG s el m a
- = Done a (StreamG s el)
- | Cont (StreamG s el -> IterateeGM s el m a)
- | Seek FileOffset (StreamG s el -> IterateeGM s el m a)

- newtype IterateeGM s el m a = IM {}
- type EnumeratorN s_outer el_outer s_inner el_inner m a = IterateeG s_inner el_inner m a -> IterateeGM s_outer el_outer m (IterateeG s_inner el_inner m a)
- type EnumeratorGM s el m a = IterateeG s el m a -> IterateeGM s el m a
- type EnumeratorGMM sfrom elfrom sto elto m a = IterateeG sto elto m a -> IterateeGM sfrom elfrom m a
- liftI :: Monad m => IterateeG s el m a -> IterateeGM s el m a
- (>>==) :: Monad m => IterateeGM s el m a -> (IterateeG s el m a -> IterateeGM s' el' m b) -> IterateeGM s' el' m b
- (==<<) :: Monad m => (IterateeG s el m a -> IterateeGM s' el' m b) -> IterateeGM s el m a -> IterateeGM s' el' m b
- joinI :: (StreamChunk s el, StreamChunk s' el', Monad m) => IterateeGM s el m (IterateeG s' el' m a) -> IterateeGM s el m a
- stream2list :: (StreamChunk s el, Monad m) => IterateeGM s el m [el]
- iterErr :: (StreamChunk s el, Monad m) => String -> IterateeGM s el m ()
- iterReportError :: (StreamChunk s el, Monad m) => IterateeGM s el m (Maybe String)
- break :: (StreamChunk s el, Monad m) => (el -> Bool) -> IterateeGM s el m (s el, Maybe el)
- dropWhile :: (StreamChunk s el, Monad m) => (el -> Bool) -> IterateeGM s el m ()
- drop :: (StreamChunk s el, Monad m) => Int -> IterateeGM s el m ()
- head :: (StreamChunk s el, Monad m) => IterateeGM s el m (Maybe el)
- peek :: (StreamChunk s el, Monad m) => IterateeGM s el m (Maybe el)
- skipToEof :: (StreamChunk s el, Monad m) => IterateeGM s el m ()
- seek :: (StreamChunk s el, Monad m) => FileOffset -> IterateeGM s el m ()
- take :: (StreamChunk s el, Monad m) => Int -> EnumeratorN s el s el m a
- takeR :: (StreamChunk s el, Monad m) => Int -> EnumeratorN s el s el m a
- mapStream :: (StreamChunk s el, StreamChunk s el', Monad m) => (el -> el') -> EnumeratorN s el s el' m a
- convStream :: (StreamChunk s el, StreamChunk s' el', Monad m) => IterateeGM s el m (Maybe (s' el')) -> EnumeratorN s el s' el' m a
- enumEof :: (StreamChunk s el, Monad m) => EnumeratorGM s el m a
- enumErr :: (StreamChunk s el, Monad m) => String -> EnumeratorGM s el m a
- (>.) :: (StreamChunk s el, Monad m) => EnumeratorGM s el m a -> EnumeratorGM s el m a -> EnumeratorGM s el m a
- enumPure1Chunk :: (StreamChunk s el, Monad m) => s el -> EnumeratorGM s el m a
- enumPureNChunk :: (StreamChunk s el, Monad m) => s el -> Int -> EnumeratorGM s el m a
- type FileOffset = COff
- bindm :: Monad m => m (Maybe a) -> (a -> m (Maybe b)) -> m (Maybe b)

# Types

data StreamChunk c el => StreamG c el Source

A stream is a (continuing) sequence of elements bundled in Chunks. The first two variants indicate termination of the stream. Chunk a gives the currently available part of the stream. The stream is not terminated yet. The case (null Chunk) signifies a stream with no currently available data but which is still continuing. A stream processor should, informally speaking, ``suspend itself'' and wait for more data to arrive.

data IterateeG s el m a Source

Iteratee -- a generic stream processor, what is being folded over
a stream
When Iteratee is in the `done`

state, it contains the computed
result and the remaining part of the stream.
In the `cont`

state, the iteratee has not finished the computation
and needs more input.
We assume that all iteratees are `good`

-- given bounded input,
they do the bounded amount of computation and take the bounded amount
of resources. The monad m describes the sort of computations done
by the iteratee as it processes the stream. The monad m could be
the identity monad (for pure computations) or the IO monad
(to let the iteratee store the stream processing results as they
are computed).
We also assume that given a terminated stream, an iteratee
moves to the done state, so the results computed so far could be returned.

Done a (StreamG s el) | |

Cont (StreamG s el -> IterateeGM s el m a) | |

Seek FileOffset (StreamG s el -> IterateeGM s el m a) |

newtype IterateeGM s el m a Source

StreamChunk s el => MonadTrans (IterateeGM s el) | |

(StreamChunk s el, Monad m) => Monad (IterateeGM s el m) | |

(Monad m, Functor m) => Functor (IterateeGM s el m) | |

(StreamChunk s el, MonadIO m) => MonadIO (IterateeGM s el m) |

type EnumeratorN s_outer el_outer s_inner el_inner m a = IterateeG s_inner el_inner m a -> IterateeGM s_outer el_outer m (IterateeG s_inner el_inner m a)Source

The type of the converter from the stream with elements el_outer to the stream with element el_inner. The result is the iteratee for the outer stream that uses an `IterateeG el_inner m a' to process the embedded, inner stream as it reads the outer stream.

type EnumeratorGM s el m a = IterateeG s el m a -> IterateeGM s el m aSource

Each enumerator takes an iteratee and returns an iteratee an Enumerator is an iteratee transformer. The enumerator normally stops when the stream is terminated or when the iteratee moves to the done state, whichever comes first. When to stop is of course up to the enumerator...

type EnumeratorGMM sfrom elfrom sto elto m a = IterateeG sto elto m a -> IterateeGM sfrom elfrom m aSource

More general enumerator type: enumerator that maps streams (not necessarily in lock-step). This is a flattened (`joinI-ed') EnumeratorN sfrom elfrom sto elto m a

# Iteratees

## Iteratee Combinators

liftI :: Monad m => IterateeG s el m a -> IterateeGM s el m aSource

Lift an `IterateeG`

into an `IterateeGM`

.

(>>==) :: Monad m => IterateeGM s el m a -> (IterateeG s el m a -> IterateeGM s' el' m b) -> IterateeGM s' el' m bSource

Just like bind (at run-time, this is indeed exactly bind)

(==<<) :: Monad m => (IterateeG s el m a -> IterateeGM s' el' m b) -> IterateeGM s el m a -> IterateeGM s' el' m bSource

Just like an application -- a call-by-value-like application

joinI :: (StreamChunk s el, StreamChunk s' el', Monad m) => IterateeGM s el m (IterateeG s' el' m a) -> IterateeGM s el m aSource

The following is a `variant`

of join in the IterateeGM s el m monad
When el' is the same as el, the type of joinI is indeed that of
true monadic join. However, joinI is subtly different: since
generally el' is different from el, it makes no sense to
continue using the internal, IterateeG el' m a: we no longer
have elements of the type el' to feed to that iteratee.
We thus send EOF to the internal Iteratee and propagate its result.
This join function is useful when dealing with `derived iteratees'
for embedded/nested streams. In particular, joinI is useful to
process the result of take, mapStream, or convStream below.

stream2list :: (StreamChunk s el, Monad m) => IterateeGM s el m [el]Source

Read a stream to the end and return all of its elements as a list

## Error handling

iterErr :: (StreamChunk s el, Monad m) => String -> IterateeGM s el m ()Source

Report and propagate an error. Disregard the input first and then propagate the error.

iterReportError :: (StreamChunk s el, Monad m) => IterateeGM s el m (Maybe String)Source

Check to see if the stream is in error

## Basic Iteratees

break :: (StreamChunk s el, Monad m) => (el -> Bool) -> IterateeGM s el m (s el, Maybe el)Source

The analogue of List.break
It takes an element predicate and returns a pair:
(str, Just c) -- the element `c`

is the first element of the stream
satisfying the break predicate;
The chunk str is the prefix of the stream up
to but including `c`

(str,Nothing) -- The stream is terminated with EOF or error before
any element satisfying the break predicate was found.
str is the scanned part of the stream.
None of the element in str satisfy the break predicate.

dropWhile :: (StreamChunk s el, Monad m) => (el -> Bool) -> IterateeGM s el m ()Source

A particular optimized case of `drop`

: skip all elements of the stream
satisfying the given predicate - until the first element
that does not satisfy the predicate, or the end of the stream.
This is the analogue of List.dropWhile

drop :: (StreamChunk s el, Monad m) => Int -> IterateeGM s el m ()Source

Skip n elements of the stream, if there are that many This is the analogue of List.drop

head :: (StreamChunk s el, Monad m) => IterateeGM s el m (Maybe el)Source

Attempt to read the next element of the stream Return (Just c) if successful, return Nothing if the stream is terminated (by EOF or an error)

peek :: (StreamChunk s el, Monad m) => IterateeGM s el m (Maybe el)Source

Look ahead at the next element of the stream, without removing it from the stream. Return (Just c) if successful, return Nothing if the stream is terminated (by EOF or an error)

skipToEof :: (StreamChunk s el, Monad m) => IterateeGM s el m ()Source

Skip the rest of the stream

seek :: (StreamChunk s el, Monad m) => FileOffset -> IterateeGM s el m ()Source

Create a request to seek within an input stream. This will result in an error if the enumerator is not capable of responding to a seek request.

## Advanced iteratee combinators

take :: (StreamChunk s el, Monad m) => Int -> EnumeratorN s el s el m aSource

Read n elements from a stream and apply the given iteratee to the stream of the read elements. Unless the stream is terminated early, we read exactly n elements (even if the iteratee has accepted fewer).

takeR :: (StreamChunk s el, Monad m) => Int -> EnumeratorN s el s el m aSource

Read n elements from a stream and apply the given iteratee to the
stream of the read elements. If the given iteratee accepted fewer
elements, we stop.
This is the variation of `take`

with the early termination
of processing of the outer stream once the processing of the inner stream
finished early. This variation is particularly useful for randomIO,
where we do not have to care to `drain the input stream'.

mapStream :: (StreamChunk s el, StreamChunk s el', Monad m) => (el -> el') -> EnumeratorN s el s el' m aSource

Map the stream: yet another iteratee transformer Given the stream of elements of the type el and the function el->el', build a nested stream of elements of the type el' and apply the given iteratee to it. Note the contravariance

convStream :: (StreamChunk s el, StreamChunk s' el', Monad m) => IterateeGM s el m (Maybe (s' el')) -> EnumeratorN s el s' el' m aSource

Convert one stream into another, not necessarily in `lockstep`

The transformer mapStream maps one element of the outer stream
to one element of the nested stream. The transformer below is more
general: it may take several elements of the outer stream to produce
one element of the inner stream, or the other way around.
The transformation from one stream to the other is specified as
IterateeGM s el m (Maybe [el']). The `Maybe`

type reflects the
possibility of the conversion error.

# Enumerators

enumEof :: (StreamChunk s el, Monad m) => EnumeratorGM s el m aSource

The most primitive enumerator: applies the iteratee to the terminated stream. The result is the iteratee usually in the done state.

enumErr :: (StreamChunk s el, Monad m) => String -> EnumeratorGM s el m aSource

Another primitive enumerator: report an error

(>.) :: (StreamChunk s el, Monad m) => EnumeratorGM s el m a -> EnumeratorGM s el m a -> EnumeratorGM s el m aSource

The composition of two enumerators: essentially the functional composition It is convenient to flip the order of the arguments of the composition though: in e1 >. e2, e1 is executed first

enumPure1Chunk :: (StreamChunk s el, Monad m) => s el -> EnumeratorGM s el m aSource

The pure 1-chunk enumerator It passes a given list of elements to the iteratee in one chunk This enumerator does no IO and is useful for testing of base parsing

enumPureNChunk :: (StreamChunk s el, Monad m) => s el -> Int -> EnumeratorGM s el m aSource

The pure n-chunk enumerator It passes a given lift of elements to the iteratee in n chunks This enumerator does no IO and is useful for testing of base parsing and handling of chunk boundaries

# Misc.

type FileOffset = COff