Safe Haskell | Safe-Inferred |
---|---|
Language | Haskell2010 |
Spawn subprocesses and interact with them using Pipes
The interface in this module deliberately resembles the interface in System.Process. However, one consequence of this is that you will not want to have unqualified names from this module and from System.Process in scope at the same time.
As in System.Process, you create a subprocess by creating a
CreateProcess
record and then applying a function to that
record. Unlike System.Process, you use functions such as
pipeInput
or pipeInputOutput
to specify what streams you want
to use a Proxy
for and what streams you wish to be Inherit
ed
or if you want to UseHandle
. You then send or receive
information using one or more Proxy
.
Use the -threaded
GHC option when compiling your programs or
when using GHCi. Internally, this module uses
waitForProcess
from the System.Process module.
As the documentation for waitForProcess
states, you must use the
-threaded
option to prevent every thread in the system from
suspending when waitForProcess
is used. So, if your program
experiences deadlocks, be sure you used the -threaded
option.
This module relies on the Pipes, Pipes.Safe, Control.Concurrent.Async, and System.Process modules. You will want to have basic familiarity with what all of those modules do before using this module.
All communcation with subprocesses is done with strict
ByteString
s. If you are dealing with textual data, the text
library has functions to convert a ByteString
to a Text
; you
will want to look at Data.Text.Encoding
.
Nobody would mistake this module for a shell; nothing beats the shell as a language for starting other programs, as the shell is designed for that. This module allows you to perform simple streaming with subprocesses without leaving the comfort of Haskell. Take a look at the README.md file, which is distributed with the tarball or is available at Github at
https://github.com/massysett/pipes-cliff
There you will find references to other libraries that you might find more useful than this one.
You will want to consult Pipes.Cliff.Examples for some examples before getting started. There are some important notes in there about how to run pipelines.
- data CmdSpec
- data NonPipe
- data CreateProcess = CreateProcess {}
- procSpec :: String -> [String] -> CreateProcess
- squelch :: Monad m => a -> m ()
- type Stdin m a = Consumer (Either a ByteString) m (Maybe a, ExitCode)
- type Outstream r m a = Producer (Either a ByteString) m r
- type Stdout r m a = Outstream r m a
- type Stderr r m a = Outstream r m a
- pipeInput :: (MonadSafe m, MonadCatch (Base m)) => NonPipe -> NonPipe -> CreateProcess -> IO (Stdin m a, ProcessHandle)
- pipeOutput :: (MonadSafe mo, MonadCatch (Base mo)) => NonPipe -> NonPipe -> CreateProcess -> IO (Stdout r mo ExitCode, ProcessHandle)
- pipeError :: (MonadSafe me, MonadCatch (Base me)) => NonPipe -> NonPipe -> CreateProcess -> IO (Stderr r me ExitCode, ProcessHandle)
- pipeInputOutput :: (MonadSafe mi, MonadCatch (Base mi), MonadSafe mo, MonadCatch (Base mo)) => NonPipe -> CreateProcess -> IO ((Stdin mi a, Stdout r mo ExitCode), ProcessHandle)
- pipeInputError :: (MonadSafe mi, MonadCatch (Base mi), MonadSafe me, MonadCatch (Base me)) => NonPipe -> CreateProcess -> IO ((Stdin mi a, Stderr r me ExitCode), ProcessHandle)
- pipeOutputError :: (MonadSafe mo, MonadCatch (Base mo), MonadSafe me, MonadCatch (Base me)) => NonPipe -> CreateProcess -> IO ((Stdout ro mo ExitCode, Stderr re me ExitCode), ProcessHandle)
- pipeInputOutputError :: (MonadSafe mi, MonadCatch (Base mi), MonadSafe mo, MonadCatch (Base mo), MonadSafe me, MonadCatch (Base me)) => CreateProcess -> IO ((Stdin mi a, Stdout ro mo ExitCode, Stderr re me ExitCode), ProcessHandle)
- forwardRight :: Monad m => Pipe (Either a b) b m a
- wrapRight :: Monad m => Pipe a (Either l a) m r
- conveyor :: Effect (SafeT IO) a -> IO (Async a)
- safeEffect :: Effect (SafeT IO) a -> IO a
- immortal :: Monad m => r -> Producer' (Either r a) m r'
- data ProcessHandle
- originalCreateProcess :: ProcessHandle -> CreateProcess
- isStillRunning :: ProcessHandle -> IO Bool
- waitForProcess :: ProcessHandle -> IO ExitCode
- terminateProcess :: ProcessHandle -> IO ()
- withProcess :: IO (a, ProcessHandle) -> (a -> IO b) -> IO b
- withConveyor :: Effect (SafeT IO) a -> IO b -> IO b
- data Activity
- data Outbound
- data HandleDesc
- data Oopsie = Oopsie Activity HandleDesc CmdSpec IOException
- module Control.Concurrent.Async
- module Pipes
- module Pipes.Safe
- module System.Exit
Specifying a subprocess's properties
Like CmdSpec
in System.Process, but also has an
instance for Show
.
data CreateProcess Source
Like CreateProcess
in System.Process,
this gives the necessary information to create a subprocess. All
but one of these fields is also present in
CreateProcess
, and they all have the same meaning;
the only field that is different is the handler
field.
CreateProcess | |
|
:: String | The name of the program to run, such as |
-> [String] | Command-line arguments |
-> CreateProcess |
Create a CreateProcess
record with default settings. The
default settings are:
- a raw command (as opposed to a shell command) is created
- the current working directory is not changed from the parent process
- the environment is not changed from the parent process
- the parent's file descriptors (other than standard input, standard output, and standard error) are inherited
- no new process group is created
delegate_ctlc
isFalse
handler
isdefaultHandler
squelch :: Monad m => a -> m () Source
Do not show or do anything with exceptions; useful to use as a
handler
.
Side effects: None.
Type synonyms
type Stdin m a = Consumer (Either a ByteString) m (Maybe a, ExitCode) Source
Consumer that reads values for a process standard input. Its
input value is described in Outstream
. The result type is a
tuple (a, b)
, where a
is the return code from the upstream
process, and b
is the return code from this process. a
will be
Nothing if the downstream process terminated before the upstream
one, or Just
if the upstream process terminated first. The
Consumer
process's process exit code is always available and is
returned in b
.
type Outstream r m a = Producer (Either a ByteString) m r Source
Producer of values from a process standard output or error. yield
a
if the stream is done producing values, or a
Left
if the stream is still producing values.
Right
ByteString
Outstream
is polymorphic in its return type, r
, becasuse the
Outstream
never stops yielding values; instead, it just yield
s
its exit code over and over again after the process terminates.
Creating processes
Each of these functions creates a process. The process begins
running immediately in a separate process while your Haskell program
continues concurrently. A function is provided for each possible
combination of standard input, standard output, and standard error.
Use the NonPipe
type to describe what you want to do with streams
you do NOT want to create a stream for. For example, to create a
subprocess that does not create a Pipe for any of the standard
streams, use pipeNone
. You must describe what you want done with
standard input, standard output, and standard error. To create a
subprocess that creates a Proxy
for standard input and standard
output, use pipeInputOutput
. You must describe what you want done
with standard error. A Producer
is returned for standard output
and a Consumer
for standard input.
Each function also returns a ProcessHandle
; this is not the same
ProcessHandle
that you will find in System.Process. You can use
this ProcessHandle
to obtain some information about the process that
is created and to get the eventual ExitCode
.
Every time you create a process with one of these functions, some
additional behind-the-scenes resources are created, such as some
threads to move data to and from the process. In normal usage, these
threads will be cleaned up after the process exits. However, if
exceptions are thrown, there could be resource leaks. Applying
terminateProcess
to a ProcessHandle
makes a best effort to clean
up all the resources that Cliff may create, including the process
itself and any additional threads. To guard against resource leaks,
use the functions found in Control.Exception or in
Control.Monad.Catch. Control.Monad.Catch provides operations that
are the same as those in Control.Exception, but they are not limited
to IO
.
I say that terminateProcess
"makes a best effort" to release
resources because in UNIX it merely sends a SIGTERM
to the process.
That should kill well-behaved processes, but terminateProcess
does
not send a SIGKILL
. terminateProcess
always closes all handles
associated with the process and it kills all Haskell threads that were
moving data to and from the process. (terminateProcess
does not
kill threads it does not know about, such as threads you created with
conveyor
.)
There is no function that will create a process that has no Proxy
at all. For that, just use createProcess
in
System.Process.
:: (MonadSafe m, MonadCatch (Base m)) | |
=> NonPipe | Standard output |
-> NonPipe | Standard error |
-> CreateProcess | |
-> IO (Stdin m a, ProcessHandle) | A |
Create a Consumer
for standard input.
:: (MonadSafe mo, MonadCatch (Base mo)) | |
=> NonPipe | Standard input |
-> NonPipe | Standard error |
-> CreateProcess | |
-> IO (Stdout r mo ExitCode, ProcessHandle) | A |
Create a Producer
for standard output.
:: (MonadSafe me, MonadCatch (Base me)) | |
=> NonPipe | Standard input |
-> NonPipe | Standard output |
-> CreateProcess | |
-> IO (Stderr r me ExitCode, ProcessHandle) | A |
Create a Producer
for standard error.
:: (MonadSafe mi, MonadCatch (Base mi), MonadSafe mo, MonadCatch (Base mo)) | |
=> NonPipe | Standard error |
-> CreateProcess | |
-> IO ((Stdin mi a, Stdout r mo ExitCode), ProcessHandle) | A |
:: (MonadSafe mi, MonadCatch (Base mi), MonadSafe me, MonadCatch (Base me)) | |
=> NonPipe | Standard output |
-> CreateProcess | |
-> IO ((Stdin mi a, Stderr r me ExitCode), ProcessHandle) | A |
:: (MonadSafe mo, MonadCatch (Base mo), MonadSafe me, MonadCatch (Base me)) | |
=> NonPipe | Standard input |
-> CreateProcess | |
-> IO ((Stdout ro mo ExitCode, Stderr re me ExitCode), ProcessHandle) | A |
:: (MonadSafe mi, MonadCatch (Base mi), MonadSafe mo, MonadCatch (Base mo), MonadSafe me, MonadCatch (Base me)) | |
=> CreateProcess | |
-> IO ((Stdin mi a, Stdout ro mo ExitCode, Stderr re me ExitCode), ProcessHandle) | A |
Proxy
combinators
forwardRight :: Monad m => Pipe (Either a b) b m a Source
Forwards only Right values; terminates on the first Left value
and returns its value. Useful to forward the output of an
Outstream
to a pipeline that expects only ByteString
s.
wrapRight :: Monad m => Pipe a (Either l a) m r Source
Forwards all values, after rewrapping them in a Right. Useful to
convert a producer of ByteString
into a Producer
of Either
which can be fed to a Stdin
.
conveyor :: Effect (SafeT IO) a -> IO (Async a) Source
Runs in the background an effect, typically one that is moving data from one process to another. For examples of its usage, see Pipes.Cliff.Examples.
safeEffect :: Effect (SafeT IO) a -> IO a Source
Runs in the foreground an effect in the SafeT
monad.
immortal :: Monad m => r -> Producer' (Either r a) m r' Source
Converts a Producer
that returns a particular type
to one that never returns a value at all but that, instead, takes
that return type and yield
s it forever as a Left
. Use it with
>>=
, like so:
alwaysUnit :: Monad m => Producer (Either () a) m r alwaysUnit = return () >>= immortal
This is useful to convert a producer of values that might terminate
into one that does not terminate, so that it can be fed into a
Stdin
. For an example of its use, see
limitedAlphaNumbers
.
Querying and terminating the process
data ProcessHandle Source
Allows you to terminate the process, as well as to obtain some information about the process.
originalCreateProcess :: ProcessHandle -> CreateProcess Source
Tells you the CreateProcess
that was originally used to create
the process associated with this ProcessHandle
.
isStillRunning :: ProcessHandle -> IO Bool Source
Is this process still running?
Side effects: examines the process handle to see if it has yet returned a value. Does not block; should return immediately.
waitForProcess :: ProcessHandle -> IO ExitCode Source
Gets the exit code of the process that belongs to the ProcessHandle
.
Side effects: may block if process has not yet exited. Usually you
can get the exit code through more idiomatic pipes
functions, as
the various Proxy
return the ExitCode
.
terminateProcess :: ProcessHandle -> IO () Source
Terminates a process. Cleans up all associated resources. Use
this with bracket
to ensure proper cleanup of
resources.
Exception safety
These are some simple combinators built with
bracket
; feel free to use your own favorite
idioms for exception safety.
:: IO (a, ProcessHandle) | Creates the process |
-> (a -> IO b) | Uses the process |
-> IO b |
Creates a process, uses it, and terminates it when the last
computation ends. Don't try to use any of the process resources
after the last computation ends, because the process will already
have been terminated. For an example of its use, see
standardOutputAndErrorBracketed
.
:: Effect (SafeT IO) a | The |
-> IO b | The rest of the computation to run |
-> IO b |
Runs an Effect
in the backgroud (typically one that is moving
data from one process to another). If the background thread is
still running when the second computation ends, the background
thread is terminated. For an example of its use, see
standardOutputAndErrorBracketed
.
Errors and warnings
You will only need what's in this section if you want to examine errors more closely.
When dealing with a Handle
, errors can occur when reading from,
writing to, or closing the handle.
The two kinds of outbound handles.
data HandleDesc Source
Describes a handle. From the perspective of the subprocess; for
example, Input
means that this handle is connected to the
process's standard input.
Describes all IO exceptions. The Oopsie
contains the
IOException
itself, along with the CmdSpec
that was running
when the exception occurred.
The exceptions that are caught and placed into an Oopsie
may
arise from reading data from or writing data to a Handle
. In
these errors, the associated Producer
or Consumer
will
terminate (which may trigger various cleanup actions in the
MonadSafe
computation) but the exception itself is not
re-thrown; rather, it is passed to the handler
. Similarly, an
exception may occur while closing a handle; these exceptions are
caught, not rethrown, and are passed to the handler
. If an
exception arises when terminating a process (I'm not sure this is
possible) then it is also caught, not rethrown, and passed to the
handler
.
If an exception arises when creating a process--such as a command
not being found--the exception is not caught, handled, or
passed to the handler
. In addition, no exceptions are caught
if they originated during a waitForProcess
. (I can't
conceive of how any synchronous exceptions could arise from
waitForProcess
, but if they do, Cliff does not handle
them.) Also, an Oopsie
is created only for an IOException
;
no other exceptions of any kind are caught or handled. However,
exceptions of any kind will still trigger appropriate cleanup
actions in the MonadSafe
computation.
Re-exports
- Control.Concurrent.Async reexports all bindings
- Pipes reexports all bindings
- Pipes.Safe reexports
runSafeT
- System.Exit reexports all bindings
module Control.Concurrent.Async
module Pipes
module Pipes.Safe
module System.Exit
Some design notes
Two overarching principles guided the design of this library. First, I wanted the interface to use simple ByteStrings. That most closely represents what a UNIX process sees. If the user wants to use Text or String, it's easy enough to convert between those types and a ByteString. Then the user has to pay explicit attention to encoding issues--as she should, because not all UNIX processes deal with encoded textual data.
Second, I paid meticulous attention to resource management. Resources
are deterministically destroyed immediately after use. This
eliminates many bugs. Even so, I decided to leave it up to the user
to use something like bracket
to ensure that all
resources are cleaned up if there is an exception. Originally I tried
to have the library do this, but that turned out not to be very
composable. There are already many exception-handling mechanisms
available in Control.Exception, Pipes.Safe, and
Control.Monad.Catch, and it seems best to let the user choose how to
handle this issue; she can just perform a bracket
and may combine this with the ContT
monad in transformers
or mtl
if she wishes, or perhaps with the managed
library.
You might wonder why, if you are using an external process as
a pipeline, why can't you create, well, a Pipe
? Wouldn't
that be an obvious choice? Well, if such an interface is
possible using Pipes in its current incarnation, nobody has
figured it out yet. I don't think it's possible. See also
https://groups.google.com/d/msg/haskell-pipes/JFfyquj5HAg/Lxz7p50JOh4J
for a discussion about this.