Streamly.System.Command

Use command strings to execute OS processes. These processes can be used just like native Haskell functions - to generate, transform or consume streams. It provides a powerful way to write high-level Haskell scripts to perform tasks similar to shell scripts without requiring the shell. Moreover, the Haskell scripts provide C-like performance.

This module is a wrapper over the Streamly.System.Process module.

See also: Streamly.Internal.System.Command.

Setup

To execute the code examples provided in this module in ghci, please run the following commands first.

>>> :set -XFlexibleContexts
>>> :set -XQuasiQuotes
>>> import Data.Char (toUpper)
>>> import Data.Function ((&))
>>> import Streamly.Unicode.String (str)
>>> import qualified Streamly.Data.Array as Array
>>> import qualified Streamly.Console.Stdio as Stdio
>>> import qualified Streamly.Data.Fold as Fold
>>> import qualified Streamly.Data.Stream.Prelude as Stream
>>> import qualified Streamly.System.Command as Command
>>> import qualified Streamly.Unicode.Stream as Unicode

>>> import qualified Streamly.Internal.System.Process as Process
>>> import qualified Streamly.Internal.Console.Stdio as Stdio
>>> import qualified Streamly.Internal.FileSystem.Dir as Dir

Overview

Please see the Streamly.System.Process for basics.

Streamly.System.Process module requires specifying the command executable name and its arguments separately (e.g. "ls" "-al") whereas using this module we can specify the executable and its arguments more conveniently as a single command string e.g. we can execute "ls -al".

A command string is parsed in the same way as a posix shell would parse it. A command string consists of whitespace separated tokens with the first token treated as the executable name and the rest as arguments. Whitespace can be escaped using \. Alternatively, double quotes or single quotes can be used to enclose tokens with whitespaces. Quotes can be escaped using \. Single quotes inside double quotes or vice-versa are treated as normal characters.

You can use the string quasiquoter str to write commands conveniently, it allows Haskell variable expansion as well e.g.:

>>> f = "file name"
>>> [str|ls -al "#{f} with spaces"|]
"ls -al \"file name with spaces\""

With the Streamly.System.Command module you can write the examples in the Streamly.System.Process module more conveniently.

Executables as functions

The shell command echo "hello world" | tr [a-z] [A-Z] can be written as follows using this module:

>>> :{
   Command.toBytes [str|echo "hello world"|]
 & Command.pipeBytes [str|tr [a-z] [A-Z]|]
 & Stream.fold Stdio.write
 :}
 HELLO WORLD

Shell commands as functions

If you want to execute the same command using the shell:

>>> :{
   Command.toBytes [str|sh "-c" "echo 'hello world' | tr [a-z] [A-Z]"|]
 & Stream.fold Stdio.write
 :}
 HELLO WORLD

Running Commands Concurrently

Running grep concurrently on many files:

>>> :{
grep file =
   Command.toBytes [str|grep -H "pattern" #{file}|]
 & Stream.handle (\(_ :: Command.ProcessFailure) -> Stream.nil)
 & Stream.foldMany (Fold.takeEndBy (== 10) Array.write)
 :}

>>> :{
pgrep =
   Dir.readFiles "."
 & Stream.parConcatMap id grep
 & Stream.fold Stdio.writeChunks
:}

Types

newtype ProcessFailure

An exception that is raised when a process fails.

Since: 0.1.0

Constructors

ProcessFailure Int

The exit code of the process.

Instances

Exception ProcessFailure

Defined in Streamly.Internal.System.Process

Methods

toException :: ProcessFailure -> SomeException

fromException :: SomeException -> Maybe ProcessFailure

displayException :: ProcessFailure -> String

Show ProcessFailure

Defined in Streamly.Internal.System.Process

Methods

showsPrec :: Int -> ProcessFailure -> ShowS

show :: ProcessFailure -> String

showList :: [ProcessFailure] -> ShowS

Generation

toBytes :: (MonadAsync m, MonadCatch m) => String -> Stream m Word8

>>> toBytes = streamWith Process.toBytes

>>> toBytes "echo hello world" & Stdio.putBytes
hello world
>>> toBytes "echo hello\\ world" & Stdio.putBytes
hello world
>>> toBytes "echo 'hello world'" & Stdio.putBytes
hello world
>>> toBytes "echo \"hello world\"" & Stdio.putBytes
hello world

Pre-release

toChunks :: (MonadAsync m, MonadCatch m) => String -> Stream m (Array Word8)

>>> toChunks = streamWith Process.toChunks

>>> toChunks "echo hello world" & Stdio.putChunks
hello world

Pre-release

toChars :: (MonadAsync m, MonadCatch m) => String -> Stream m Char

>>> toChars = streamWith Process.toChars

>>> toChars "echo hello world" & Stdio.putChars
hello world

Pre-release

toLines

Arguments

:: (MonadAsync m, MonadCatch m)

=> Fold m Char a

-> String

Command

-> Stream m a

Output Stream

>>> toLines f = streamWith (Process.toLines f)

>>> toLines Fold.toList "echo -e hello\\\\nworld" & Stream.fold Fold.toList
["hello","world"]

Pre-release

Effects

toString

Arguments

:: (MonadAsync m, MonadCatch m)

=> String

Command

-> m String

>>> toString = runWith Process.toString

>>> toString "echo hello world"
"hello world\n"

Pre-release

toStdout

Arguments

:: (MonadAsync m, MonadCatch m)

=> String

Command

-> m ()

>>> toStdout = runWith Process.toStdout

>>> toStdout "echo hello world"
hello world

Pre-release

toNull

Arguments

:: (MonadAsync m, MonadCatch m)

=> String

Command

-> m ()

>>> toNull = runWith Process.toNull

>>> toNull "echo hello world"

Pre-release

Transformation

pipeBytes :: (MonadAsync m, MonadCatch m) => String -> Stream m Word8 -> Stream m Word8

Like pipeChunks except that it works on a stream of bytes instead of a stream of chunks.

>>> :{
   toBytes "echo hello world"
 & pipeBytes "tr [a-z] [A-Z]"
 & Stdio.putBytes
 :}
HELLO WORLD

Pre-release

pipeChars :: (MonadAsync m, MonadCatch m) => String -> Stream m Char -> Stream m Char

Like pipeChunks except that it works on a stream of chars instead of a stream of chunks.

>>> :{
   toChars "echo hello world"
 & pipeChars "tr [a-z] [A-Z]"
 & Stdio.putChars
 :}
HELLO WORLD

Pre-release

pipeChunks :: (MonadAsync m, MonadCatch m) => String -> Stream m (Array Word8) -> Stream m (Array Word8)

pipeChunks command input runs the executable with arguments specified by command and supplying input stream as its standard input. Returns the standard output of the executable as a stream of byte arrays.

If only the name of an executable file is specified instead of its path then the file name is searched in the directories specified by the PATH environment variable.

If the input stream throws an exception or if the output stream is garbage collected before it could finish then the process is terminated with SIGTERM.

If the process terminates with a non-zero exit code then a ProcessFailure exception is raised.

The following code is equivalent to the shell command echo "hello world" | tr [a-z] [A-Z]:

>>> :{
   toChunks "echo hello world"
 & pipeChunks "tr [a-z] [A-Z]"
 & Stdio.putChunks
 :}
HELLO WORLD

Pre-release