-- Hoogle documentation, generated by Haddock -- See Hoogle, http://www.haskell.org/hoogle/ -- | Streaming data processing library. -- -- Conduits are an approach to the streaming data problem. It is meant as -- an alternative to enumerators/iterators, hoping to address the same -- issues with different trade-offs based on real-world experience with -- enumerators. For more information, see -- http://www.yesodweb.com/book/conduit. -- -- Release history: -- -- @package conduit @version 0.3.0 -- | The main module, exporting types, utility functions, and fuse and -- connect operators. module Data.Conduit -- | A Source has two operations on it: pull some data, and close -- the Source. A Source should free any resources it -- allocated when either it returns Closed or when it is -- explicitly closed (the second record on either the Open or -- SourceM constructors). -- -- Since 0.3.0 data Source m a -- | A Source providing more data. Provides records for the next -- Source in the stream, a close action, and the data provided. Open :: (Source m a) -> (m ()) -> a -> Source m a -- | A Source which has no more data available. Closed :: Source m a -- | Requires a monadic action to retrieve the next Source in the -- stream. Second record allows you to close the Source. SourceM :: (m (Source m a)) -> (m ()) -> Source m a -- | When actually interacting with Sources, we sometimes want to -- be able to buffer the output, in case any intermediate steps return -- leftover data. A BufferedSource allows for such buffering. -- -- A BufferedSource, unlike a Source, is resumable, -- meaning it can be passed to multiple Sinks without -- restarting. Therefore, a BufferedSource relaxes the main -- invariant of this package: the same value may be used multiple times. -- -- The intention of a BufferedSource is to be used internally by -- an application or library, not to be part of its user-facing API. For -- example, the Warp webserver uses a BufferedSource internally -- for parsing the request headers, but then passes a normal -- Source to the web application for reading the request body. -- -- One caveat: while the types will allow you to use the buffered source -- in multiple threads, there is no guarantee that all -- BufferedSources will handle this correctly. -- -- Since 0.3.0 data BufferedSource m a -- | Places the given Source and a buffer into a mutable variable. -- Note that you should manually call bsourceClose when the -- BufferedSource is no longer in use. -- -- Since 0.3.0 bufferSource :: MonadIO m => Source m a -> m (BufferedSource m a) -- | Turn a BufferedSource into a Source. Note that in -- general this will mean your original BufferedSource will be -- closed. Additionally, all leftover data from usage of the returned -- Source will be discarded. In other words: this is a -- no-going-back move. -- -- Note: bufferSource . unbufferSource is not -- the identity function. -- -- Since 0.3.0 unbufferSource :: MonadIO m => BufferedSource m a -> Source m a -- | Close the underlying Source for the given -- BufferedSource. Note that this function can safely be called -- multiple times, as it will first check if the Source was -- previously closed. -- -- Since 0.3.0 bsourceClose :: MonadIO m => BufferedSource m a -> m () -- | A typeclass allowing us to unify operators for Source and -- BufferedSource. -- -- Since 0.3.0 class IsSource src m -- | In general, a sink will consume data and eventually produce an output -- when it has consumed "enough" data. There are two caveats to that -- statement: -- -- -- -- Note that you can indicate any leftover data from processing via the -- Maybe input field of the Done constructor. However, -- it is a violation of the Sink invariants to return leftover -- data when no input has been consumed. Concrete, that means that a -- function like yield is invalid: -- -- 
--   yield :: input -> Sink input m ()
--   yield input = Done (Just input) ()
--
-- -- A Sink should clean up any resources it has allocated when it -- returns a value. -- -- Since 0.3.0 data Sink input m output -- | Awaiting more input. Processing :: (SinkPush input m output) -> (SinkClose m output) -> Sink input m output -- | Processing complete. Done :: (Maybe input) -> output -> Sink input m output -- | Perform some monadic action to continue. SinkM :: (m (Sink input m output)) -> Sink input m output -- | Push a value into a Sink and get a new Sink as a -- result. -- -- Since 0.3.0 type SinkPush input m output = input -> Sink input m output -- | Closing a Sink returns the final output. -- -- Since 0.3.0 type SinkClose m output = m output -- | A Conduit allows data to be pushed to it, and for each new -- input, can produce a stream of output values (possibly an empty -- stream). It can be considered a hybrid of a Sink and a -- Source. -- -- A Conduit has four constructors, corresponding to four -- distinct states of operation. -- -- Since 0.3.0 data Conduit input m output -- | Indicates that the Conduit needs more input in order to -- produce output. It also provides an action to close the -- Conduit early, for cases when there is no more input -- available, or when no more output is requested. Closing at this point -- returns a Source to allow for either consuming or ignoring -- the new stream. NeedInput :: (ConduitPush input m output) -> (ConduitClose m output) -> Conduit input m output -- | Indicates that the Conduit has more output available. It has -- three records: the next Conduit to continue the stream, a -- close action for early termination, and the output currently -- available. Note that, unlike NeedInput, the close action here -- returns () instead of Source. The reasoning is that -- HaveOutput will only be closed early if no more output is -- requested, since no input is required. HaveOutput :: (Conduit input m output) -> (m ()) -> output -> Conduit input m output -- | Indicates that no more output is available, and no more input may be -- sent. It provides an optional leftover input record. Note: It is a -- violation of Conduit's invariants to return leftover output -- that was never consumed, similar to the invariants of a Sink. Finished :: (Maybe input) -> Conduit input m output -- | Indicates that a monadic action must be taken to determine the next -- Conduit. It also provides an early close action. Like -- HaveOutput, this action returns (), since it should -- only be used when no more output is requested. ConduitM :: (m (Conduit input m output)) -> (m ()) -> Conduit input m output -- | Pushing new data to a Conduit produces a new -- Conduit. -- -- Since 0.3.0 type ConduitPush input m output = input -> Conduit input m output -- | When closing a Conduit, it can produce a final stream of -- values. -- -- Since 0.3.0 type ConduitClose m output = Source m output -- | The connect operator, which pulls data from a source and pushes to a -- sink. There are two ways this process can terminate: -- --
    --
  1. If the Sink is a Done constructor, the -- Source is closed.
    2. --
  2. If the Source is a Closed constructor, the -- Sink is closed.
    3. --
-- -- This function will automatically close any Sources, but will -- not close any BufferedSources, allowing them to be reused. -- Also, leftover data will be discarded when connecting a -- Source, but will be buffered when using a -- BufferedSource. -- -- Since 0.3.0 ($$) :: IsSource src m => src m a -> Sink a m b -> m b -- | Left fuse, combining a source and a conduit together into a new -- source. -- -- Any Source passed in will be automatically closed, while a -- BufferedSource will be left open. Leftover input will be -- discarded for a Source, and buffered for a -- BufferedSource. -- -- Since 0.3.0 ($=) :: IsSource src m => src m a -> Conduit a m b -> Source m b -- | Right fuse, combining a conduit and a sink together into a new sink. -- -- Any leftover data returns from the Sink will be discarded. -- -- Since 0.3.0 (=$) :: Monad m => Conduit a m b -> Sink b m c -> Sink a m c -- | Middle fuse, combining two conduits together into a new conduit. -- -- Any leftovers provided by the inner Conduit will be -- discarded. -- -- Since 0.3.0 (=$=) :: Monad m => Conduit a m b -> Conduit b m c -> Conduit a m c -- | Construct a Source with some stateful functions. This function -- addresses threading the state value for you. -- -- Since 0.3.0 sourceState :: Monad m => state -> (state -> m (SourceStateResult state output)) -> Source m output -- | A combination of sourceIO and sourceState. -- -- Since 0.3.0 sourceStateIO :: MonadResource m => IO state -> (state -> IO ()) -> (state -> m (SourceStateResult state output)) -> Source m output -- | The return value when pulling in the sourceState function. -- Either indicates no more data, or the next value and an updated state. -- -- Since 0.3.0 data SourceStateResult state output StateOpen :: state -> output -> SourceStateResult state output StateClosed :: SourceStateResult state output -- | Construct a Source based on some IO actions for alloc/release. -- -- Since 0.3.0 sourceIO :: MonadResource m => IO state -> (state -> IO ()) -> (state -> m (SourceIOResult output)) -> Source m output -- | The return value when pulling in the sourceIO function. -- Either indicates no more data, or the next value. -- -- Since 0.3.0 data SourceIOResult output IOOpen :: output -> SourceIOResult output IOClosed :: SourceIOResult output -- | Transform the monad a Source lives in. -- -- Note that this will not thread the individual monads together, -- meaning side effects will be lost. This function is most useful for -- transformers only providing context and not producing side-effects, -- such as ReaderT. -- -- Since 0.3.0 transSource :: Monad m => (forall a. m a -> n a) -> Source m output -> Source n output -- | Close a Source, regardless of its current state. -- -- Since 0.3.0 sourceClose :: Monad m => Source m a -> m () -- | Construct a Sink with some stateful functions. This function -- addresses threading the state value for you. -- -- Since 0.3.0 sinkState :: Monad m => state -> (state -> input -> m (SinkStateResult state input output)) -> (state -> m output) -> Sink input m output -- | A helper type for sinkState, indicating the result of being -- pushed to. It can either indicate that processing is done, or to -- continue with the updated state. -- -- Since 0.3.0 data SinkStateResult state input output StateDone :: (Maybe input) -> output -> SinkStateResult state input output StateProcessing :: state -> SinkStateResult state input output -- | Construct a Sink. Note that your push and close functions need -- not explicitly perform any cleanup. -- -- Since 0.3.0 sinkIO :: MonadResource m => IO state -> (state -> IO ()) -> (state -> input -> m (SinkIOResult input output)) -> (state -> m output) -> Sink input m output -- | A helper type for sinkIO, indicating the result of being -- pushed to. It can either indicate that processing is done, or to -- continue. -- -- Since 0.3.0 data SinkIOResult input output IODone :: (Maybe input) -> output -> SinkIOResult input output IOProcessing :: SinkIOResult input output -- | Transform the monad a Sink lives in. -- -- See transSource for more information. -- -- Since 0.3.0 transSink :: Monad m => (forall a. m a -> n a) -> Sink input m output -> Sink input n output -- | Close a Sink if it is still open, discarding any output it -- produces. -- -- Since 0.3.0 sinkClose :: Monad m => Sink input m output -> m () -- | A helper function for returning a list of values from a -- Conduit. -- -- Since 0.3.0 haveMore :: Conduit a m b -> m () -> [b] -> Conduit a m b -- | Construct a Conduit with some stateful functions. This function -- addresses threading the state value for you. -- -- Since 0.3.0 conduitState :: Monad m => state -> (state -> input -> m (ConduitStateResult state input output)) -> (state -> m [output]) -> Conduit input m output -- | A helper type for conduitState, indicating the result of -- being pushed to. It can either indicate that processing is done, or to -- continue with the updated state. -- -- Since 0.3.0 data ConduitStateResult state input output StateFinished :: (Maybe input) -> [output] -> ConduitStateResult state input output StateProducing :: state -> [output] -> ConduitStateResult state input output -- | Construct a Conduit. -- -- Since 0.3.0 conduitIO :: MonadResource m => IO state -> (state -> IO ()) -> (state -> input -> m (ConduitIOResult input output)) -> (state -> m [output]) -> Conduit input m output -- | A helper type for conduitIO, indicating the result of being -- pushed to. It can either indicate that processing is done, or to -- continue. -- -- Since 0.3.0 data ConduitIOResult input output IOFinished :: (Maybe input) -> [output] -> ConduitIOResult input output IOProducing :: [output] -> ConduitIOResult input output -- | Transform the monad a Conduit lives in. -- -- See transSource for more information. -- -- Since 0.3.0 transConduit :: Monad m => (forall a. m a -> n a) -> Conduit input m output -> Conduit input n output -- | Close a Conduit early, discarding any output. -- -- Since 0.3.0 conduitClose :: Monad m => Conduit input m output -> m () -- | Helper type for constructing a Conduit based on -- Sinks. This allows you to write higher-level code that takes -- advantage of existing conduits and sinks, and leverages a sink's -- monadic interface. -- -- Since 0.3.0 type SequencedSink state input m output = state -> Sink input m (SequencedSinkResponse state input m output) -- | Convert a SequencedSink into a Conduit. -- -- Since 0.3.0 sequenceSink :: Monad m => state -> SequencedSink state input m output -> Conduit input m output -- | Specialised version of sequenceSink -- -- Note that this function will return an infinite stream if provided a -- SinkNoData constructor. In other words, you probably don't -- want to do sequence . return. -- -- Since 0.3.0 sequence :: Monad m => Sink input m output -> Conduit input m output -- | Return value from a SequencedSink. -- -- Since 0.3.0 data SequencedSinkResponse state input m output -- | Set a new state, and emit some new output. Emit :: state -> [output] -> SequencedSinkResponse state input m output -- | End the conduit. Stop :: SequencedSinkResponse state input m output -- | Pass control to a new conduit. StartConduit :: (Conduit input m output) -> SequencedSinkResponse state input m output -- | Provide for a stream of data that can be flushed. -- -- A number of Conduits (e.g., zlib compression) need the -- ability to flush the stream at some point. This provides a single -- wrapper datatype to be used in all such circumstances. -- -- Since 0.3.0 data Flush a Chunk :: a -> Flush a Flush :: Flush a -- | The Resource transformer. This transformer keeps track of all -- registered actions, and calls them upon exit (via -- runResourceT). Actions may be registered via register, -- or resources may be allocated atomically via allocate. -- allocate corresponds closely to bracket. -- -- Releasing may be performed before exit via the release -- function. This is a highly recommended optimization, as it will ensure -- that scarce resources are freed early. Note that calling -- release will deregister the action, so that a release action -- will only ever be called once. -- -- Since 0.3.0 data ResourceT (m :: * -> *) a :: (* -> *) -> * -> * -- | A Monad which allows for safe resource allocation. In theory, -- any monad transformer stack included a ResourceT can be an -- instance of MonadResource. -- -- Note: runResourceT has a requirement for a -- MonadBaseControl IO m monad, which allows control operations -- to be lifted. A MonadResource does not have this requirement. -- This means that transformers such as ContT can be an instance -- of MonadResource. However, the ContT wrapper will -- need to be unwrapped before calling runResourceT. -- -- Since 0.3.0 class (MonadThrow m, MonadUnsafeIO m, MonadIO m) => MonadResource (m :: * -> *) -- | A Monad which can throw exceptions. Note that this does not -- work in a vanilla ST or Identity monad. Instead, you -- should use the ExceptionT transformer in your stack if you are -- dealing with a non-IO base monad. -- -- Since 0.3.0 class Monad m => MonadThrow (m :: * -> *) monadThrow :: (MonadThrow m, Exception e) => e -> m a -- | A Monad based on some monad which allows running of some -- IO actions, via unsafe calls. This applies to IO and -- ST, for instance. -- -- Since 0.3.0 class Monad m => MonadUnsafeIO (m :: * -> *) unsafeLiftIO :: MonadUnsafeIO m => IO a -> m a -- | Unwrap a ResourceT transformer, and call all registered release -- actions. -- -- Note that there is some reference counting involved due to -- resourceForkIO. If multiple threads are sharing the same -- collection of resources, only the last call to runResourceT -- will deallocate the resources. -- -- Since 0.3.0 runResourceT :: MonadBaseControl IO m => ResourceT m a -> m a instance Show a => Show (Flush a) instance Eq a => Eq (Flush a) instance Ord a => Ord (Flush a) instance Functor Flush instance MonadIO m => IsSource BufferedSource m instance Monad m => IsSource Source m -- | Higher-level functions to interact with the elements of a stream. Most -- of these are based on list functions. -- -- Note that these functions all deal with individual elements of a -- stream as a sort of "black box", where there is no introspection of -- the contained elements. Values such as ByteString and -- Text will likely need to be treated specially to deal with -- their contents properly (Word8 and Char, -- respectively). See the Data.Conduit.Binary and -- Data.Conduit.Text modules. module Data.Conduit.List -- | Convert a list into a source. -- -- Since 0.3.0 sourceList :: Monad m => [a] -> Source m a -- | A source that returns nothing. Note that this is just a -- type-restricted synonym for mempty. -- -- Since 0.3.0 sourceNull :: Monad m => Source m a -- | A strict left fold. -- -- Since 0.3.0 fold :: Monad m => (b -> a -> b) -> b -> Sink a m b -- | Take some values from the stream and return as a list. If you want to -- instead create a conduit that pipes data to another sink, see -- isolate. This function is semantically equivalent to: -- -- 
--   take i = isolate i =$ consume
--
-- -- Since 0.3.0 take :: Monad m => Int -> Sink a m [a] -- | Ignore a certain number of values in the stream. This function is -- semantically equivalent to: -- -- 
--   drop i = take i >> return ()
--
-- -- However, drop is more efficient as it does not need to hold -- values in memory. -- -- Since 0.3.0 drop :: Monad m => Int -> Sink a m () -- | Take a single value from the stream, if available. -- -- Since 0.3.0 head :: Monad m => Sink a m (Maybe a) -- | Combines two sources. The new source will stop producing once either -- source has been exhausted. -- -- Since 0.3.0 zip :: Monad m => Source m a -> Source m b -> Source m (a, b) -- | Look at the next value in the stream, if available. This function will -- not change the state of the stream. -- -- Since 0.3.0 peek :: Monad m => Sink a m (Maybe a) -- | Consume all values from the stream and return as a list. Note that -- this will pull all values into memory. For a lazy variant, see -- Data.Conduit.Lazy. -- -- Since 0.3.0 consume :: Monad m => Sink a m [a] -- | Ignore the remainder of values in the source. Particularly useful when -- combined with isolate. -- -- Since 0.3.0 sinkNull :: Monad m => Sink a m () -- | A monadic strict left fold. -- -- Since 0.3.0 foldM :: Monad m => (b -> a -> m b) -> b -> Sink a m b -- | Apply the action to all values in the stream. -- -- Since 0.3.0 mapM_ :: Monad m => (a -> m ()) -> Sink a m () -- | Apply a transformation to all values in a stream. -- -- Since 0.3.0 map :: Monad m => (a -> b) -> Conduit a m b -- | Apply a transformation to all values in a stream, concatenating the -- output values. -- -- Since 0.3.0 concatMap :: Monad m => (a -> [b]) -> Conduit a m b -- | concatMap with an accumulator. -- -- Since 0.3.0 concatMapAccum :: Monad m => (a -> accum -> (accum, [b])) -> accum -> Conduit a m b -- | Grouping input according to an equality function. -- -- Since 0.3.0 groupBy :: Monad m => (a -> a -> Bool) -> Conduit a m [a] -- | Ensure that the inner sink consumes no more than the given number of -- values. Note this this does not ensure that the sink consumes -- all of those values. To get the latter behavior, combine with -- sinkNull, e.g.: -- -- 
--   src $$ do
--       x <- isolate count =$ do
--           x <- someSink
--           sinkNull
--           return x
--       someOtherSink
--       ...
--
-- -- Since 0.3.0 isolate :: Monad m => Int -> Conduit a m a -- | Keep only values in the stream passing a given predicate. -- -- Since 0.3.0 filter :: Monad m => (a -> Bool) -> Conduit a m a -- | Apply a monadic transformation to all values in a stream. -- -- If you do not need the transformed values, and instead just want the -- monadic side-effects of running the action, see mapM_. -- -- Since 0.3.0 mapM :: Monad m => (a -> m b) -> Conduit a m b -- | Apply a monadic transformation to all values in a stream, -- concatenating the output values. -- -- Since 0.3.0 concatMapM :: Monad m => (a -> m [b]) -> Conduit a m b -- | concatMapM with an accumulator. -- -- Since 0.3.0 concatMapAccumM :: Monad m => (a -> accum -> m (accum, [b])) -> accum -> Conduit a m b -- | Functions for interacting with bytes. module Data.Conduit.Binary -- | Stream the contents of a file as binary data. -- -- Since 0.3.0 sourceFile :: MonadResource m => FilePath -> Source m ByteString -- | Stream the contents of a Handle as binary data. Note that this -- function will not automatically close the Handle when -- processing completes, since it did not acquire the Handle in -- the first place. -- -- Since 0.3.0 sourceHandle :: MonadResource m => Handle -> Source m ByteString -- | An alternative to sourceHandle. Instead of taking a pre-opened -- Handle, it takes an action that opens a Handle (in read -- mode), so that it can open it only when needed and close it as soon as -- possible. -- -- Since 0.3.0 sourceIOHandle :: MonadResource m => IO Handle -> Source m ByteString -- | Stream the contents of a file as binary data, starting from a certain -- offset and only consuming up to a certain number of bytes. -- -- Since 0.3.0 sourceFileRange :: MonadResource m => FilePath -> Maybe Integer -> Maybe Integer -> Source m ByteString -- | Stream all incoming data to the given file. -- -- Since 0.3.0 sinkFile :: MonadResource m => FilePath -> Sink ByteString m () -- | Stream all incoming data to the given Handle. Note that this -- function will not automatically close the Handle when -- processing completes. -- -- Since 0.3.0 sinkHandle :: MonadResource m => Handle -> Sink ByteString m () -- | An alternative to sinkHandle. Instead of taking a pre-opened -- Handle, it takes an action that opens a Handle (in write -- mode), so that it can open it only when needed and close it as soon as -- possible. -- -- Since 0.3.0 sinkIOHandle :: MonadResource m => IO Handle -> Sink ByteString m () -- | Stream the contents of the input to a file, and also send it along the -- pipeline. Similar in concept to the Unix command tee. -- -- Since 0.3.0 conduitFile :: MonadResource m => FilePath -> Conduit ByteString m ByteString -- | Ensure that only up to the given number of bytes are consume by the -- inner sink. Note that this does not ensure that all of those -- bytes are in fact consumed. -- -- Since 0.3.0 isolate :: Monad m => Int -> Conduit ByteString m ByteString -- | Open a file Handle safely by automatically registering a -- release action. -- -- While you are not required to call hClose on the resulting -- handle, you should do so as early as possible to free scarce -- resources. -- -- Since 0.3.0 openFile :: MonadResource m => FilePath -> IOMode -> m Handle -- | Return the next byte from the stream, if available. -- -- Since 0.3.0 head :: Monad m => Sink ByteString m (Maybe Word8) -- | Return all bytes while the predicate returns True. -- -- Since 0.3.0 takeWhile :: Monad m => (Word8 -> Bool) -> Conduit ByteString m ByteString -- | Ignore all bytes while the predicate returns True. -- -- Since 0.3.0 dropWhile :: Monad m => (Word8 -> Bool) -> Sink ByteString m () -- | Take the given number of bytes, if available. -- -- Since 0.3.0 take :: Monad m => Int -> Sink ByteString m ByteString -- | Split the input bytes into lines. In other words, split on the LF byte -- (10), and strip it from the output. -- -- Since 0.3.0 lines :: Monad m => Conduit ByteString m ByteString -- | Handle streams of text. -- -- Parts of this code were taken from enumerator and adapted for -- conduits. module Data.Conduit.Text -- | A specific character encoding. -- -- Since 0.3.0 data Codec -- | Convert text into bytes, using the provided codec. If the codec is not -- capable of representing an input character, an exception will be -- thrown. -- -- Since 0.3.0 encode :: MonadThrow m => Codec -> Conduit Text m ByteString -- | Convert bytes into text, using the provided codec. If the codec is not -- capable of decoding an input byte sequence, an exception will be -- thrown. -- -- Since 0.3.0 decode :: MonadThrow m => Codec -> Conduit ByteString m Text -- | Since 0.3.0 utf8 :: Codec -- | Since 0.3.0 utf16_le :: Codec -- | Since 0.3.0 utf16_be :: Codec -- | Since 0.3.0 utf32_le :: Codec -- | Since 0.3.0 utf32_be :: Codec -- | Since 0.3.0 ascii :: Codec -- | Since 0.3.0 iso8859_1 :: Codec instance Typeable TextException instance Show TextException instance Exception TextException instance Show Codec -- | Use lazy I/O for consuming the contents of a source. Warning: All -- normal warnings of lazy I/O apply. However, if you consume the content -- within the ResourceT, you should be safe. module Data.Conduit.Lazy -- | Use lazy I/O to consume all elements from a Source. -- -- This function relies on monadActive to determine if the -- underlying monadic state has been closed. -- -- Since 0.3.0 lazyConsume :: (MonadBaseControl IO m, MonadActive m) => Source m a -> m [a]