conduit: Streaming data processing library.

[ conduit, data, library, mit ] [ Propose Tags ]

conduit is a solution to the streaming data problem, allowing for production, transformation, and consumption of streams of data in constant memory. It is an alternative to lazy I/O which guarantees deterministic resource handling, and fits in the same general solution space as enumerator/iteratee and pipes. For a brief tutorial, please see the Data.Conduit module.

Release history:

The internals of the package are now separated to the .Internal module, leaving only the higher-level interface in the advertised API. Internally, switched to a Leftover constructor and slightly tweaked the finalization semantics.
Inspired by the design of the pipes package: we now have a single unified type underlying Source, Sink, and Conduit. This type is named Pipe. There are type synonyms provided for the other three types. Additionally, BufferedSource is no longer provided. Instead, the connect-and-resume operator, $$+, can be used for the same purpose.
ResourceT has been greatly simplified, specialized for IO, and moved into a separate package. Instead of hard-coding ResourceT into the conduit datatypes, they can now live around any monad. The Conduit datatype has been enhanced to better allow generation of streaming output. The SourceResult, SinkResult, and ConduitResult datatypes have been removed entirely.
Instead of storing state in mutable variables, we now use CPS. A Source returns the next Source, and likewise for Sinks and Conduits. Not only does this take better advantage of GHC's optimizations (about a 20% speedup), but it allows some operations to have a reduction in algorithmic complexity from exponential to linear. This also allowed us to remove the Prepared set of types. Also, the State functions (e.g., sinkState) use better constructors for return types, avoiding the need for a dummy state on completion.
BufferedSource is now an abstract type, and has a much more efficient internal representation. The result was a 41% speedup on microbenchmarks (note: do not expect speedups anywhere near that in real usage). In general, we are moving towards BufferedSource being a specific tool used internally as needed, but using Source for all external APIs.
Initial release.


Automatic Flags

Turn on some runtime check to ensure invariants are respected.


Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info


Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees


Versions [RSS] 0.0.0,,, 0.0.1,, 0.0.2, 0.0.3, 0.0.4, 0.1.0, 0.1.1,, 0.2.0, 0.2.1, 0.2.2, 0.3.0, 0.4.0,, 0.4.1,, 0.4.2, 0.5.0, 0.5.1, 0.5.2,,,,,,,, 0.5.3, 0.5.4,,, 0.5.5, 0.5.6, 1.0.0,,, 1.0.1, 1.0.2, 1.0.3, 1.0.4,,, 1.0.5,, 1.0.6, 1.0.7,,,,, 1.0.8, 1.0.9,,,, 1.0.10,, 1.0.11,, 1.0.12, 1.0.13,, 1.0.14, 1.0.15,, 1.0.16, 1.0.17,, 1.1.0,,, 1.1.1,, 1.1.2,, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.1.7, 1.2.0,,, 1.2.1, 1.2.2,,, 1.2.3,, 1.2.4,,,, 1.2.5,, 1.2.6,,,,,,, 1.2.7, 1.2.8, 1.2.9,, 1.2.10, 1.2.11, 1.2.12,, 1.2.13,, 1.3.0,,,, 1.3.1,,, 1.3.2,, 1.3.3, 1.3.4,,,, 1.3.5
Dependencies base (>=4.3 && <5), bytestring (>=0.9), containers, lifted-base (>=0.1), monad-control (>=0.3.1 && <0.4), resourcet (>=0.4.3 && <0.5), text (>=0.11), transformers (>=0.2.2 && <0.4), transformers-base (>=0.4.1 && <0.5), void (>=0.5.5 && <0.6) [details]
License BSD-3-Clause
Author Michael Snoyman
Category Data, Conduit
Home page
Source repo head: git clone git://
Uploaded by MichaelSnoyman at 2012-11-27T08:24:34Z
Distributions Arch:1.3.5, Debian:1.3.2, Fedora:1.3.5, FreeBSD:1.2.5, LTSHaskell:1.3.5, NixOS:1.3.5, Stackage:1.3.5, openSUSE:1.3.5
Reverse Dependencies 589 direct, 4079 indirect [details]
Downloads 436634 total (683 in the last 30 days)
Rating 2.25 (votes: 9) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs uploaded by user
Build status unknown [no reports yet]