Data.Conduit.Text

Handle streams of text.

Parts of this code were taken from enumerator and adapted for conduits.

For many purposes, it's recommended to use the conduit-combinators library, which provides a more complete set of functions.

Text codecs

data Codec

A specific character encoding.

Since 0.3.0

Instances

Show Codec

Methods

showsPrec :: Int -> Codec -> ShowS

show :: Codec -> String

showList :: [Codec] -> ShowS

encode :: MonadThrow m => Codec -> Conduit Text m ByteString

Convert text into bytes, using the provided codec. If the codec is not capable of representing an input character, an exception will be thrown.

Since 0.3.0

decode :: MonadThrow m => Codec -> Conduit ByteString m Text

Convert bytes into text, using the provided codec. If the codec is not capable of decoding an input byte sequence, an exception will be thrown.

Since 0.3.0

utf8 :: Codec

Since 0.3.0

utf16_le :: Codec

Since 0.3.0

utf16_be :: Codec

Since 0.3.0

utf32_le :: Codec

Since 0.3.0

utf32_be :: Codec

Since 0.3.0

ascii :: Codec

Since 0.3.0

iso8859_1 :: Codec

Since 0.3.0

lines :: Monad m => Conduit Text m Text

Emit each line separately

Since 0.4.1

linesBounded :: MonadThrow m => Int -> Conduit Text m Text

Variant of the lines function with an integer parameter. The text length of any emitted line never exceeds the value of the parameter. Whenever this is about to happen a LengthExceeded exception is thrown. This function should be used instead of the lines function whenever we are dealing with user input (e.g. a file upload) because we can't be sure that user input won't have extraordinarily large lines which would require large amounts of memory if consumed.

data TextException

Since 0.3.0

Constructors

DecodeException Codec Word8

EncodeException Codec Char

LengthExceeded Int

TextException SomeException

NewDecodeException !Text !Int !ByteString

Instances

Show TextException

Methods

showsPrec :: Int -> TextException -> ShowS

show :: TextException -> String

showList :: [TextException] -> ShowS

Exception TextException

Methods

toException :: TextException -> SomeException

fromException :: SomeException -> Maybe TextException

displayException :: TextException -> String

takeWhile :: Monad m => (Char -> Bool) -> Conduit Text m Text

Since 1.0.8

dropWhile :: Monad m => (Char -> Bool) -> Consumer Text m ()

Since 1.0.8

take :: Monad m => Int -> Conduit Text m Text

Since 1.0.8

drop :: Monad m => Int -> Consumer Text m ()

Since 1.0.8

foldLines :: Monad m => (a -> ConduitM Text o m a) -> a -> ConduitM Text o m a

Since 1.0.8

withLine :: Monad m => Sink Text m a -> Consumer Text m (Maybe a)

Since 1.0.8

decodeUtf8 :: MonadThrow m => Conduit ByteString m Text

Decode a stream of UTF8-encoded bytes into a stream of text, throwing an exception on invalid input.

Since 1.0.15

decodeUtf8Lenient :: Monad m => Conduit ByteString m Text

Decode a stream of UTF8 data, and replace invalid bytes with the Unicode replacement character.

Since 1.1.1

encodeUtf8 :: Monad m => Conduit Text m ByteString

Encode a stream of text into a stream of bytes.

Since 1.0.15

detectUtf :: MonadThrow m => Conduit ByteString m Text

Automatically determine which UTF variant is being used. This function checks for BOMs, removing them as necessary. It defaults to assuming UTF-8.

Since 1.1.9