Pipes.Csv

This module allows constant-space CSV parsing.

It feeds ByteStrings into cassavas incremental CSV parser to attain true constant-space record streaming.

Example

Heres a simple example that reads from stdin and writes to a file

    import Pipes.Safe (runSafeT)
    import qualified Pipes.Safe.Prelude  as PS
    import qualified Pipes.ByteString    as PB
    import Data.Vector (fromList)
    import System.IO (IOMode(WriteMode))

    data Person = Person String Int
                deriving (Show)

    instance FromNamedRecord Person where
      parseNamedRecord p =
        Person <$> p .: "name"
               <*> p .: "age"

    personRec ~(Person name age) = ["name" .= name, "age" .= age]

    instance ToNamedRecord Person where
      toNamedRecord = namedRecord . personRec

    persons :: Monad m => Producer ByteString m () -> Producer Person m ()
    persons p = decodeByName p >-> right

    -- note: right can be replaced with Pipes.Prelude.concat in ghc-7.8,
    --       thanks to a Foldable instance for Either
    right :: (Monad m) => Pipe (Either a b) b m r
    right = loop
      where
        loop = await >>= s -> case s of
          Left _  -> loop
          Right v -> yield v >> loop

    write f = PS.withFile f WriteMode PB.toHandle

    main = runSafeT $ runEffect $ pipeline
      where
        header = fromList $ map fst $ personRec undefined
        pipeline = persons stdin
               >-> right
               >-> encodeByName header
               >-> write "persons_out.csv"

Decode records

decode :: (Monad m, FromRecord a) => HasHeader -> Producer ByteString m () -> Producer (Either String a) m ()

Equivalent to decodeWith defaultDecodeOptions.

decodeWith :: (Monad m, FromRecord a) => DecodeOptions -> HasHeader -> Producer ByteString m () -> Producer (Either String a) m ()

Create a Producer that takes a ByteString Producer as input, producing either errors or FromRecords.

Decode named records

decodeByName :: (Monad m, FromNamedRecord a) => Producer ByteString m () -> Producer (Either String a) m ()

Equivalent to decodeByNameWith defaultDecodeOptions.

decodeByNameWith :: (Monad m, FromNamedRecord a) => DecodeOptions -> Producer ByteString m () -> Producer (Either String a) m ()

Create a Producer that takes a ByteString Producer as input, producing either errors or FromNamedRecords.

Decode parsed records

feedParser :: Monad m => Parser a -> Producer ByteString m () -> Producer (Either String a) m ()

Create a Record Producer by feeding ByteStrings into a Parser

feedHeaderParser :: Monad m => HeaderParser (Parser a) -> Producer ByteString m () -> Producer (Either String a) m ()

Create a NamedRecord Producer by feeding ByteStrings into a Parser

Encode records

encode :: (Monad m, ToRecord a) => Pipe a ByteString m r

Encode records as strict ByteStrings

encodeWith :: (Monad m, ToRecord a) => EncodeOptions -> Pipe a ByteString m r

Encode records as strict ByteStrings

Encode named records

encodeByName :: (Monad m, ToNamedRecord a) => Header -> Pipe a ByteString m r

Encode named records as strict ByteStrings

encodeByNameWith :: (Monad m, ToNamedRecord a) => EncodeOptions -> Header -> Pipe a ByteString m r

Encode named records as strict ByteStrings

Re-exports

Data.Csv re-exports common types and operators:

• FromRecord, FromNamedRecord, ToRecord, ToNamedRecord
• ToField, FromField
• Record, Field, NamedRecord
• '(.!)', '(.:)', '(.=)'
• DecodeOptions, defaultDecodeOptions

module Data.Csv

data HasHeader :: *

Is the CSV data preceded by a header?

Constructors

HasHeader

The CSV data is preceded by a header

NoHeader

The CSV data is not preceded by a header