streaming: an elementary streaming prelude and general stream type.
Streaming.Prelude exports an elementary streaming prelude focussed on
a simple "source" or "producer" type, namely
Stream (Of a) m r.
This is a sort of effectful version of
([a],r) in which monadic action
is interleaved between successive elements.
The main module,
Streaming, exports a much more general type,
Stream f m r, which can be used to stream successive distinct
steps characterized by any
f, though we are here interested only in a limited range of
The streaming-IO libraries have various devices for dealing
with effectful variants of
([a],r). But it is only with
the general type
Stream f m r, or some equivalent,
that one can envisage (for example) the connected streaming of their
sorts of stream -- as one makes lists of lists in the Haskell
Data.List. One needs some such type if we are
to express properly streaming equivalents of e.g.
group :: Ord a => [a] -> [[a]] chunksOf :: Int -> [a] -> [[a]] lines :: [Char] -> [[Char]] -- but similarly with bytestring, etc.
to mention a few obviously desirable operations. But once one grasps
the iterable stream concept needed to express those functions - to wit,
Stream f m r or some equivalent - then one will also see that,
with it, one is already in possession of a complete
elementary streaming library - since one possesses
Stream ((,) a) m r
Stream (Of a) m r. This
is the type of a 'generator' or 'producer' or whatever
you call an effectful stream of items.
Streaming.Prelude is thus the simplest streaming
library that can replicate anything like the API of the
The emphasis of the library is on interoperation; for
the rest its advantages are: extreme simplicity and re-use of
intuitions the user has gathered from mastery of
Data.List. The two conceptual pre-requisites are some
comprehension of monad transformers and some familiarity
with 'rank 2 types'.
for an explanation, including the examples linked there. Elementary usage can be divined from the ghci examples in
Streaming.Prelude and perhaps from this rough beginning of a
Note also the
The simplest form of interoperation with pipes is accomplished with this isomorphism:
Pipes.unfoldr Streaming.next :: Stream (Of a) m r -> Producer a m r Streaming.unfoldr Pipes.next :: Producer a m r -> Stream (Of a) m r
Interoperation with io-streams is thus:
Streaming.reread IOStreams.read :: InputStream a -> Stream (Of a) IO () IOStreams.unfoldM Streaming.uncons :: Stream (Of a) IO () -> IO (InputStream a)
A simple exit to conduit would be, e.g.:
Conduit.unfoldM Streaming.uncons :: Stream (Of a) m () -> Source m a
These conversions should never be more expensive than a single
Here is a simple example that runs a single underlying stream with several
streaming-io libraries at once, superimposing their effects
without any accumulation:
module Main (main) where import Streaming import Pipes import Data.Conduit import qualified Streaming.Prelude as S import qualified Data.Conduit.List as CL import qualified Pipes.Prelude as P import qualified System.IO.Streams as IOS import Data.ByteString.Char8 (pack) import Data.Function ((&)) mkConduit = CL.unfoldM S.uncons mkPipe = P.unfoldr S.next mkIOStream = IOS.unfoldM S.uncons main = iostreamed where urstream = S.take 4 S.readLn :: Stream (Of Int) IO () streamed = S.copy urstream & S.map (\n -> "streaming says: " ++ show n) & S.stdoutLn piped = runEffect $ mkPipe (S.copy streamed) >-> P.map (\n -> "pipes says: " ++ show n) >-> P.stdoutLn conduited = mkConduit (S.copy piped) $$ CL.map (\n -> "conduit says: " ++ show n) =$ CL.mapM_ (liftIO . putStrLn) iostreamed = do str0 <- mkIOStream conduited str1 <- IOS.map (\n -> pack $ "io-streams says: " ++ show n ++ "\n") str0 IOS.supply str1 IOS.stdout
This program successively parses four
Ints from standard input,
and simulaneously passes them to (here trivial) stream-consuming
processes from four different libraries, using the
copy function from
Streaming.Prelude. I mark my own input with
>>> main 1 <Enter> streaming says: 1 pipes says: 1 conduit says: 1 io-streams says: 1 2 <Enter> streaming says: 2 pipes says: 2 conduit says: 2 io-streams says: 2 3 <Enter> streaming says: 3 pipes says: 3 conduit says: 3 io-streams says: 3 4 <Enter> streaming says: 4 pipes says: 4 conduit says: 4 io-streams says: 4
Of course, I could as well have passed the stream to several independent conduits, for example. Further points of comparison with the going streaming-IO libraries are discussed in the readme below.
Because these are microbenchmarks for individual functions, they represent a sort of "worst case"; many other factors can influence the speed of a complex program.
[Skip to Readme]
For package maintainers and hackage trustees