System.Process.Streaming

This module contains helper functions and types built on top of System.Process and Pipes.

They provide concurrent, streaming access to the inputs and outputs of system processes.

Error conditions that are not directly related to IO are made explicit in the types.

Regular Consumers, Parsers from pipes-parse and folds from Pipes.Prelude (also folds from pipes-bytestring and pipes-text) can be used to consume the output streams of the external processes.

Execution

execute :: PipingPolicy Void a -> CreateProcess -> IO (ExitCode, a)

executeFallibly :: PipingPolicy e a -> CreateProcess -> IO (Either e (ExitCode, a))

Executes an external process. The standard streams are piped and consumed in a way defined by the PipingPolicy argument.

This fuction re-throws any IOExceptions it encounters.

If the consumption of the standard streams fails with e, the whole computation is immediately aborted and e is returned. (An exception is not thrown in this case.).

If an error e or an exception happens, the external process is terminated.

Piping Policies

data PipingPolicy e a

A PipingPolicy determines what standard streams will be piped and what to do with them.

The user doesn't need to manually set the std_in, std_out and std_err fields of the CreateProcess record to CreatePipe, this is done automatically.

A PipingPolicy is parametrized by the type e of errors that can abort the processing of the streams.

Instances

Bifunctor PipingPolicy

Functor (PipingPolicy e)

nopiping :: PipingPolicy e ()

Do not pipe any standard stream.

pipeo :: (Show e, Typeable e) => Siphon ByteString e a -> PipingPolicy e a

Pipe stdout.

pipee :: (Show e, Typeable e) => Siphon ByteString e a -> PipingPolicy e a

Pipe stderr.

pipeoe :: (Show e, Typeable e) => Siphon ByteString e a -> Siphon ByteString e b -> PipingPolicy e (a, b)

Pipe stdout and stderr.

pipeoec :: (Show e, Typeable e) => LinePolicy e -> LinePolicy e -> Siphon Text e a -> PipingPolicy e a

Pipe stdout and stderr and consume them combined as Text.

pipei :: (Show e, Typeable e) => Pump ByteString e i -> PipingPolicy e i

Pipe stdin.

pipeio :: (Show e, Typeable e) => Pump ByteString e i -> Siphon ByteString e a -> PipingPolicy e (i, a)

Pipe stdin and stdout.

pipeie :: (Show e, Typeable e) => Pump ByteString e i -> Siphon ByteString e a -> PipingPolicy e (i, a)

Pipe stdin and stderr.

pipeioe :: (Show e, Typeable e) => Pump ByteString e i -> Siphon ByteString e a -> Siphon ByteString e b -> PipingPolicy e (i, a, b)

Pipe stdin, stdout and stderr.

pipeioec :: (Show e, Typeable e) => Pump ByteString e i -> LinePolicy e -> LinePolicy e -> Siphon Text e a -> PipingPolicy e (i, a)

Pipe stdin, stdout and stderr, consuming the last two combined as Text.

Pumping bytes into stdin

newtype Pump b e a

Constructors

Pump

Fields

runPump :: Consumer b IO () -> IO (Either e a)

Instances

Bifunctor (Pump b)

Functor (Pump b e)

(Show e, Typeable * e) => Applicative (Pump b e)

(Show e, Typeable * e, Monoid a) => Monoid (Pump b e a)

fromProducer :: Producer b IO r -> Pump b e ()

fromSafeProducer :: Producer b (SafeT IO) r -> Pump b e ()

fromFallibleProducer :: Producer b (ExceptT e IO) r -> Pump b e ()

Siphoning bytes out of stdout/stderr

data Siphon b e a

A Siphon represents a computation that completely drains a producer, but may fail early with an error of type e.

pure creates a Siphon that does nothing besides draining the Producer.

<*> executes its arguments concurrently. The Producer is forked so that each argument receives its own copy of the data.

Instances

Bifunctor (Siphon b)

Functor (Siphon b e)

(Show e, Typeable * e) => Applicative (Siphon b e)

(Show e, Typeable * e, Monoid a) => Monoid (Siphon b e a)

siphon :: (Producer b IO () -> IO (Either e a)) -> Siphon b e a

Builds a Siphon out of a computation that does something with a Producer, but may fail with an error of type e.

Even if the original computation doesn't completely drain the Producer, the constructed Siphon will.

siphon' :: (forall r. Producer b IO r -> IO (Either e (a, r))) -> Siphon b e a

Builds a Siphon out of a computation that drains a Producer completely, but may fail with an error of type e.

fromFold :: (Producer b IO () -> IO a) -> Siphon b e a

fromFold' :: (forall r. Producer b IO r -> IO (a, r)) -> Siphon b e a

Builds a Siphon out of a computation that folds a Producer and drains it completely.

fromFold'_ :: (forall r. Producer b IO r -> IO r) -> Siphon b e ()

fromConsumer :: Consumer b IO r -> Siphon b e ()

fromSafeConsumer :: Consumer b (SafeT IO) r -> Siphon b e ()

fromFallibleConsumer :: Consumer b (ExceptT e IO) r -> Siphon b e ()

fromParser :: Parser b IO (Either e a) -> Siphon b e a

Turn a Parser from pipes-parse into a Sihpon.

unwanted :: a -> Siphon b b a

Constructs a Siphon that aborts the computation if the underlying Producer produces anything.

type DecodingFunction bytes text = forall r. Producer bytes IO r -> Producer text IO (Producer bytes IO r)

See the section Non-lens decoding functions in the documentation for the pipes-text package.

encoded :: (Show e, Typeable e) => DecodingFunction bytes text -> Siphon bytes e (a -> b) -> Siphon text e a -> Siphon bytes e b

Constructs a Siphon that works on encoded values out of a Siphon that works on decoded values.

The two first arguments are a decoding function and a Siphon that determines how to handle leftovers. Pass pure id to ignore leftovers. Pass unwanted id to abort the computation if leftovers remain.

Line handling

data LinePolicy e

Defines how to decode a stream of bytes into text, including what to do in presence of leftovers. Also defines how to manipulate each individual line of text.

Instances

Functor LinePolicy

linePolicy :: (Show e, Typeable e) => DecodingFunction ByteString Text -> Siphon ByteString e () -> (forall r. Producer Text IO r -> Producer Text IO r) -> LinePolicy e

Constructs a LinePolicy.

The second argument is a Siphon value that specifies how to handle decoding failures. Passing pure () will ignore any leftovers. Passing unwanted () will abort the computation if leftovers remain.

The third argument is a function that modifies each individual line. The line is represented as a Producer to avoid having to keep it wholly in memory. If you want the lines unmodified, just pass id. Line prefixes are easy to add using applicative notation:

(\x -> yield "prefix: " *> x)

Pipelines

executePipeline :: PipingPolicy Void a -> CreatePipeline Void -> IO a

executePipelineFallibly :: (Show e, Typeable e) => PipingPolicy e a -> CreatePipeline e -> IO (Either e a)

Similar to executeFallibly, but instead of a single process it executes a (possibly branching) pipeline of external processes.

The PipingPolicy argument views the pipeline as a synthetic process for which stdin is the stdin of the first stage, stdout is the stdout of the leftmost terminal stage among those closer to the root, and stderr is a combination of the stderr streams of all the stages.

The combined stderr stream always has UTF-8 encoding.

This function has a limitation compared to the standard UNIX pipelines. If a downstream process terminates early without error, the upstream processes are not notified and keep going. There is no SIGPIPE-like functionality, in other words.

data CreatePipeline e

Constructors

CreatePipeline (Stage e) (NonEmpty (Tree (SubsequentStage e)))

Instances

Functor CreatePipeline

simplePipeline :: DecodingFunction ByteString Text -> CreateProcess -> NonEmpty (Tree CreateProcess) -> CreatePipeline String

Builds a (possibly branching) pipeline assuming that stderr has the same encoding in all the stages, that no computation is perfored between the stages, and that any exit code besides ExitSuccess in a stage actually represents an error.

data Stage e

An individual stage in a process pipeline.

The LinePolicy field defines how to handle stderr when stderr is piped.

Also required is a function that determines if the returned exit code represents an error or not. This is necessary because some programs use non-standard exit codes.

Constructors

Stage

Fields

processDefinition :: CreateProcess

stderrLinePolicy :: LinePolicy e

exitCodePolicy :: Int -> Maybe e

Instances

Functor Stage

data SubsequentStage e

Any stage beyond the first in a process pipeline.

Incoming data is passed through the Pipe before being fed to the process.

Use cat (the identity Pipe from Pipes) if no pre-processing is required.

Constructors

SubsequentStage (forall a. Pipe ByteString ByteString (ExceptT e IO) a) (Stage e)

Instances

Functor SubsequentStage

Re-exports

System.Process is re-exported for convenience.

module System.Process