pipes-break-0.2.0.6: Pipes to group by any delimiter (such as lines with carriage returns)

Pipes.Break.Text

Description

Synopsis

# Group Producers By A Delimiter

Break any producer up into groups with the delimiter stripped out with the presumption that two Producers are "equivalent" if they produce the same string when drained.

  unBreaksBy delim (breaksBy delim foo) ≡ foo
unBreakBy delim (breakBy delim foo) ≡ foo


breakBy :: Monad m => Text -> Producer Text m r -> Producer Text m (Producer Text m r) Source #

Yield argument until it reaches delimiter or end of stream, then returns the remainder (minus the delimiter).

>>> rest <- runEffect $for (breakBy "\r\n" (yield "A\r\nB\r\nC\r\n")) (lift . Prelude.print) "A"  >>> runEffect$ for rest (lift . print)
"B\r\nC\r\n"


This is almost equivalent to Pipes.Text.line except that it works for any delimiter, not just '\n', and it also consumes the delimiter.

unBreakBy :: Monad m => Text -> Producer Text m (Producer Text m r) -> Producer Text m r Source #

Recombine a producer that had been previously broken recombining it with the provided delimiter.

>>> mconcat $Pipes.Prelude.toList (unBreakBy "\n" (yield "abc" >> return (yield "def"))) "abc\ndef"  >>> mconcat$ Pipes.Prelude.toList (unBreakBy "\n" (yield "abc" >> return (return ())))
"abc"


The inverse of breakBy

breaksBy :: Monad m => Text -> Producer Text m r -> FreeT (Producer Text m) m r Source #

Group a producer on any delimiter.

unBreaksBy :: Monad m => Text -> FreeT (Producer Text m) m r -> Producer Text m r Source #

The inverse of breaksBy.

# Group Producers Ending By A Delimiter

These are almost equivalent to the breakBy functions, however they imply that every chunk "ends" with a delimiter, rather than just being separated by them.

Unfortunately it is impossible to know in a streaming fashion for sure that your next group will ever end with a delimiter (or end at all for that matter). The only way to find out is to store every line you receive until you find it. Therefore, the endsBy family of functions are not invertible.

>>> mconcat $Pipes.Prelude.toList (unEndsBy "\r\n" (endsBy "\r\n" (yield "A\r\nB\r\nC"))) "A\r\nB\r\nC\r\n"  In other words:  unEndsBy delim (endsBy delim foo) ≠ foo unEndBy delim (endBy delim foo) ≠ foo  endBy :: Monad m => Text -> Producer Text m r -> Producer Text m (Producer Text m r) Source # Yield argument until it reaches delimiter or end of stream, then return the remainder (minus the delimiter). This is equivalent to breakBy. unEndBy :: Monad m => Text -> Producer Text m (Producer Text m r) -> Producer Text m r Source # Recombine a producer that had been previously broken by using a separator. If the second stream is empty the delimiter will be added on at the end of the first producer anyways. >>> mconcat$ Pipes.Prelude.toList (unEndBy "\n" (yield "abc" >> return (yield "def")))
"abc\ndef"

>>> mconcat $Pipes.Prelude.toList (unEndBy "\n" (yield "abc" >> return (return ()))) "abc\n"  This is not equivalent to unBreakBy and is not quite an inverse of endBy endsBy :: Monad m => Text -> Producer Text m r -> FreeT (Producer Text m) m r Source # unEndsBy :: Monad m => Text -> FreeT (Producer Text m) m r -> Producer Text m r Source # Not quite the inverse of endsBy. # Utilities replace :: Monad m => Text -> Text -> Producer Text m r -> Producer Text m r Source # Replace one delimiter with another in a stream. >>> fmap mconcat <$> toListM \$ replace "\r\n" "\n" (yield "abc\ndef\r\nfoo\n\r\nbar")
"abc\ndef\nfoo\n\nbar"