enumerator: Reliable, high-performance processing with left-fold enumerators

[ data, deprecated, enumerator, library, mit ] [ Propose Tags ]
Deprecated

Typical buffer–based incremental I/O is based around a single loop, which reads data from some source (such as a socket or file), transforms it, and generates one or more outputs (such as a line count, HTTP responses, or modified file). Although efficient and safe, these loops are all single–purpose; it is difficult or impossible to compose buffer–based processing loops.

Haskell’s concept of “lazy I/O” allows pure code to operate on data from an external source. However, lazy I/O has several shortcomings. Most notably, resources such as memory and file handles can be retained for arbitrarily long periods of time, causing unpredictable performance and error conditions.

Enumerators are an efficient, predictable, and safe alternative to lazy I/O. Discovered by Oleg Kiselyov, they allow large datasets to be processed in near–constant space by pure code. Although somewhat more complex to write, using enumerators instead of lazy I/O produces more correct programs.

This library contains an enumerator implementation for Haskell, designed to be both simple and efficient. Three core types are defined, along with numerous helper functions:

  • Iteratee: Data sinks, analogous to left folds. Iteratees consume a sequence of input values, and generate a single output value. Many iteratees are designed to perform side effects (such as printing to stdout), so they can also be used as monad transformers.

  • Enumerator: Data sources, which generate input sequences. Typical enumerators read from a file handle, socket, random number generator, or other external stream. To operate, enumerators are passed an iteratee, and provide that iteratee with input until either the iteratee has completed its computation, or EOF.

  • Enumeratee: Data transformers, which operate as both enumerators and iteratees. Enumeratees read from an outer enumerator, and provide the transformed data to an inner iteratee.


[Skip to Readme]

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

  • No Candidates
Versions [RSS] 0.1, 0.1.1, 0.2, 0.2.1, 0.3, 0.3.0.1, 0.4, 0.4.0.1, 0.4.0.2, 0.4.1, 0.4.2, 0.4.3, 0.4.3.1, 0.4.4, 0.4.5, 0.4.6, 0.4.7, 0.4.8, 0.4.9, 0.4.9.1, 0.4.10, 0.4.11, 0.4.12, 0.4.13, 0.4.13.1, 0.4.14, 0.4.15, 0.4.16, 0.4.17, 0.4.18, 0.4.19, 0.4.20
Dependencies base (>=3 && <5), bytestring (>=0.9 && <0.10), extensible-exceptions (>=0.1 && <0.2), text (>=0.7 && <0.12), transformers (>=0.2 && <0.3) [details]
License MIT
Copyright Copyright (c) John Millikin 2010
Author John Millikin <jmillikin@gmail.com>
Maintainer jmillikin@gmail.com
Category Data, Enumerator
Home page http://john-millikin.com/software/enumerator/
Bug tracker mailto:jmillikin@gmail.com
Source repo head: bzr branch http://john-millikin.com/software/enumerator/
Uploaded by JohnMillikin at 2011-03-29T18:15:29Z
Distributions FreeBSD:0.4.20
Reverse Dependencies 98 direct, 3918 indirect [details]
Downloads 59582 total (43 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs uploaded by user
Build status unknown [no reports yet]

Readme for enumerator-0.4.9.1

[back to package description]
The source code for "enumerator" is literate. To build the library from scratch,
install the "anansi" application and then run:

    anansi -o hs/ src/enumerator.anansi

To generate the woven PDF, install NoWeb and then run:

    anansi -w -l latex-noweb -o enumerator.tex src/enumerator.anansi
    xelatex enumerator.tex
    xelatex enumerator.tex