arx: Archive execution tool.

[ bsd3, library, text ] [ Propose Tags ]

The ARX system provides services for packaging, deploying and running source code. No particular format or framework is needed -- a directory of code and a command to run are enough. The system has no in-built notion of remote connections, job servers or clusters; all automation is captured as Bourne compatible scripts.

An archive of the source code, a command and optionally an environment are encoded together in a Bourne shell script that uses a small number of UNIX utilities in a broadly portable way. The generated scripts can be run directly or fed to sh on STDIN. This latter feature is useful when one would like to use ssh and sudo to set an appropriate executation context, for example running: ssh user@example.com sudo sh.

The shell tools used are head, sed, date, tr and tar. The calls to tar sometimes use -j and -z; these calls to tar may result in calls to bzip2 and gzip. Scripts have been tested with dash and the GNU tools as well as the sh and tools that are part of busybox.

The arx command line tool provides the tmpx subcommand for preparing jobs to run and the shdat subcommand for access to the low-level shell encoder. The System.Posix.ARX module provides access to the routines used for constructing commands and environments, describing archives and building Bourne shell scripts.

One way I have used arx is to test the Cabal source archive for this package:

arx tmpx ./dist/arx-* // 'cd arx-* && cabal configure && cabal build' | sh

There are binary arx command line tool releases available from:

https://github.com/solidsnack/arx/downloads

For each supported platform, there is an archive containing arx and signature files (SHA 512 and GPG).


[Skip to Readme]
Versions [faq] 0.0.0, 0.0.1, 0.0.2, 0.0.3, 0.0.4, 0.1.0, 0.1.1, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.3.0, 0.3.1, 0.3.2
Dependencies attoparsec (>=0.9.1.2), base (>=2 && <=5), blaze-builder (>=0.3), bytestring (>=0.9), bytestring-nums (>=0.3.3), containers, file-embed (>=0.0.4.1), parsec (>=3.1.2), process (>=1.0), shell-escape (>=0.1.1), template-haskell, vector (>=0.9), vector-algorithms (>=0.5.3) [details]
License BSD-3-Clause
Author Jason Dusek
Maintainer oss@solidsnack.be
Category Text
Home page http://github.com/solidsnack/arx/
Source repo head: git clone http://github.com/solidsnack/arx.git
Uploaded by JasonDusek at Thu Mar 1 00:57:01 UTC 2012
Distributions NixOS:0.3.2
Executables arx
Downloads 5273 total (261 in the last 30 days)
Rating (no votes yet) [estimated by rule of succession]
Your Rating
  • λ
  • λ
  • λ
Status Hackage Matrix CI
Docs uploaded by user
Build status unknown [no reports yet]

Modules

[Index]

Flags

NameDescriptionDefaultType
no-cli

Disable command line tool.

DisabledAutomatic

Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info

Downloads

Maintainer's Corner

For package maintainers and hackage trustees


Readme for arx-0.1.1

[back to package description]
SYNOPSIS
       arx ... (-h|-[?]|--help)? ...
       arx shdat (-b <size>)? (-o <output file>)? < input
       arx shdat (-b <size>)? (-o <output file>)? <input file>+
       arx tmpx <option,archive>* (//+ <command> (//+ <option,archive>*)?)?

DESCRIPTION
       A  UNIX  executable is a simple thing -- a file the kernel can execute,
       one way or another, via an interpreter  or  directly  as  object  code.
       Every  executable induces a family of executions -- instances of execu-
       tion with different command line arguments, with different files in the
       working directory and with different environment variables present.

       The  arx  tool captures the parameters of an execution and encodes them
       as an executable, making for easy, consistent transfer  and  repetition
       of  a  particular  run.  The generated executable ensures that each run
       occurs in a  freshly  allocated  temporary  directory,  with  only  the
       desired  files  in  scope;  it uses traps to ensure the cleanup of this
       directory; and its format is a simple POSIX shell  script,  relying  on
       just a few shell tools.

DEPENDENCIES
       The  arx  tool relies on the presence of sed, tr, date, head, tar, hex-
       dump and sh. When unpacking tar archives, it  may  use  the  -j  or  -z
       (bzip2 and gzip, respectively) options of tar. Scripts have been tested
       with dash and the GNU tools as well as the sh implementation and  user-
       land tools that are part of busybox.

APPLICATION
       The  tmpx  subcommand  of  arx offers a variety of options for bundling
       code and a task to run. The shdat subcommand  exposes  the  lower-level
       functionality  of  encoding  binary data in a shell script that outputs
       that binary data, using HERE documents and some odd  replacement  rules
       for nulls.

       Scripts generated by tmpx and shdat may be fed to sh over STDIN to exe-
       cute them. This can be helpful when using ssh and sudo  to  set  up  an
       execution context; for example:

       arx tmpx ... | ssh user@host.com sudo sh

       For  all subcommands, when options overlap in their effect -- for exam-
       ple, setting the output with -o -- the rightmost  option  takes  prece-
       dence.   Whenever -h, -? or --help is present on the command line, help
       is displayed and the program exits.

       When paths are specified on an arx command line, they  must  be  quali-
       fied, starting with /, ./ or ../. This simplifies the command line syn-
       tax, overall, without introducing troublesome ambiguities.

TMPX
       The tmpx subcommand bundles together archives, environment settings and
       an  executable  or  shell command in to a Bourne-compatible script that
       runs the command or executable in a temporary directory,  after  having
       unpacked the archives and set the environment.

       Any number of file path arguments may be specified; they will be inter-
       preted as tar archives to include in bundled script.  If  -  is  given,
       then  STDIN  will be included as an archive stream. If no arguments are
       given, it is assumed that no archives are desired and only the  command
       and environment are bundled.

       The  temporary  directory  created  by the script is different for each
       invocation, with a name of the  form  /tmp/tmpx.<timestamp>.<pid>.  The
       timestamp  used  is  a UTC, ISO 8601 format timestamp. One happy conse-
       quence of this is that earlier jobs sort  ASCIIbetically  before  later
       jobs.  After  execution,  the  temporary  directory is removed (or not,
       depending on the -rm[10!_] family of options).

          -rm0, -rm1, -rm_, -rm!
                 By default, the temporary directory  created  by  the  script
                 will be deleted no matter the exit status status of the task.
                 These options cause a script to be generated that deletes the
                 temporary  directory only on success, only on failure, always
                 (the default) or never.

          -b <size>
                 Please see the documentation for  this  option,  shared  with
                 shdat, below.

          -o <path>
                 By  default, the generated script is sent to STDOUT. With -o,
                 output is redirected to the given path.

          -e <path>
                 Causes the file specified to be packaged as the  task  to  be
                 run.  A  binary  executable, a Ruby script or a longish shell
                 script all fit here.

       In addition to these options, arguments of the form VAR=VALUE are  rec-
       ognized  as  environment  mappings and stored away in the script, to be
       sourced on execution.

       Without -e, the tmpx subcommand tries to find the task to be run  as  a
       sequence  of  arguments  delimited  by  a run of slashes. The following
       forms are all recognized:

       arx tmpx  ...some args... // ...command...
       arx tmpx  ...some args... // ...command... // ...more args...
       arx tmpx // ...command... // ...some args...

       The slash runs must have the same number of slashes  and  must  be  the
       longest  continuous  runs  of  slashes on the command line. The command
       will be included as is in a Bourne shell script.

SHDAT
       The shdat subcommand translates binary data in to a shell script  which
       outputs  the binary data. The data is encoded in HERE documents in such
       a way that data without NULs is not changed and that data with NULs  is
       minimally  expanded:  about  1% for randomish data like compressed tar-
       balls and about 10% in pathological cases.

       The shdat subcommand can be given any number of paths,  which  will  be
       concatenated in the order given. If no path is given, or if - is given,
       then STDIN will be read.

          -b <size>
                 The size of data chunks to place in each HERE  document.  The
                 argument  is  a positive integer followed by suffixes like B,
                 K, KiB, M and MiB, in the manner of dd, head and  many  other
                 tools.  The default is 4MiB.  This is unlikely to make a dif-
                 ference for you unless the generated script is intended to be
                 run on a memory-constrained system.

          -o <path>
                 By  default, the generated script is sent to STDOUT. With -o,
                 output is redirected to the given path.

EXAMPLES
       # Installer script that preserves failed builds.
       git archive HEAD | bzip2 | arx tmpx -rm0 // make install > go.sh
       # Now install as root; but don't log in as root.
       cat ./go.sh | ssh joey@hostname sudo /bin/sh

       # Variation of the above.
       git archive HEAD | bzip2 | arx tmpx -rm0 -e ./build-script.py

       # Bundle an instance of an application with DB credentials and run it.
       arx tmpx -rm! ./app.tbz ./stage-info.tgz // rake start | ssh ...

       # Get dump of linking info for build that works here but not there.
       arx tmpx ./server-build.tgz LD_DEBUG=files // ./bin/start | ssh ...

       # Test out Cabal source distribution of this package:
       arx tmpx // 'cd arx-* && cabal configure && cabal build' // \
                -rm0 ./dist/arx-0.0.0.tar.gz | sh

BUGS
       The command line parser offers no hints or help of any kind;  it  fails
       with  the simple message "argument error". The two most common mistakes
       I make are:

       o Not qualifying paths with /, ./ or ../.

       o Not specifying a subcommand (tmpx or shdat).