Hackage :: [Package]

pushme: Tool to synchronize directories with rsync, zfs or git-annex

[ bsd3, program, system ] [ Propose Tags ]

Script I use for synchronizing my data among machines.

[Skip to Readme]

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

  • No Candidates
Versions [RSS] 0.2.0, 1.0.0, 1.0.1, 1.1.0, 1.4.0, 1.5.0, 2.0.1, 2.0.2, 2.1.1, 2.1.3
Dependencies aeson (>=0.7 && <0.12), base (>=4.7 && <5.0), bytestring (>=0.10 && <0.11), containers (>=0.5 && <0.6), foldl, io-storage (>=0.3 && <0.4), lens (>=4.9 && <5.0), logging (>=3.0 && <3.1), monad-logger (>=0.3 && <0.4), old-locale (>=1.0 && <1.1), optparse-applicative (>=0.10 && <1.0), parallel-io (>=0.3 && <0.4), pipes, pipes-group, pipes-safe, pipes-text, regex-posix (>=0.95 && <1.0), safe (>=0.3 && <0.4), shelly (>=1.6 && <1.7), system-fileio (>=0.3 && <0.4), system-filepath (>=0.4 && <0.5), temporary (>=1.2 && <1.3), text (>=1.2 && <1.3), text-format (>=0.3 && <0.4), time (>=1.4 && <1.7), transformers (>=0.3 && <0.6), unix (>=2.6 && <2.8), unordered-containers (>=0.2 && <0.3), yaml (>=0.8 && <0.9) [details]
License BSD-3-Clause
Author John Wiegley
Maintainer John Wiegley <johnw@newartisans.com>
Category Development
Home page https://github.com/jwiegley/pushme
Source repo head: git clone https://github.com/jwiegley/pushme
Uploaded by JohnWiegley at 2016-09-29T19:02:41Z
Distributions
Reverse Dependencies 1 direct, 0 indirect [details]
Executables pushme
Downloads 6832 total (18 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs not available [build log]
All reported builds failed as of 2016-11-19 [all 3 reports]

Readme for pushme-2.0.1

[back to package description]

This is the script I use for synchronizing data between my machines (and also to directories on the same machine, externally connected drives, and to and between ZFS filesystems).

Here is an example set of files for synchronizing my home directory and /usr/local via rsync between two machines: foo and bar. First, ~/.pushme/stores.yml:

- Targets:
  - - bar
    - - home
      - local
  HostRe: [Ff]oo
  Name: Foo
  UserName: johnw
  SelfRe: [Ff]oo
  ZfsPool: null
  ZfsPath: null
  IsPrimary: true
  AnnexName: foo
  AnnexFlags: []
- Targets:
  - - foo
    - - home
      - local
  HostRe: [Bb]ar
  Name: Bar
  UserName: johnw
  SelfRe: [Bb]ar
  ZfsPool: null
  ZfsPath: null
  IsPrimary: false
  AnnexName: bar
  AnnexFlags:
  - - 'foo'
    - - - 'home'
        - - '-\('
          - '--not'
          - '--in'
          - 'foo'
          - '--and'
          - '--not'
          - '--in'
          - 'web'
          - '-\)'

Then, ~/.pushme/filesets.yml:

- Priority: 60
  Name: home
  Class: quick,main
  ReportMissing: true
- Priority: 80
  Name: local
  Class: system,main
  ReportMissing: false

And finally, ~/.pushme/containers.yml:

- Store: foo,bar
  Recurse: false
  Fileset: home
  PoolPath: null
  IsAnnex: false
  LastRev: null
  LastSync: null
  Path: ~/
- Store: foo,bar
  Recurse: false
  Fileset: local
  PoolPath: null
  IsAnnex: false
  LastRev: null
  LastSync: null
  Path: /usr/local/

Now I can run the following command:

foo $ pushme bar
15:18:44 - [NOTICE] Synchronizing Foo -> Bar
15:18:44 - [NOTICE] Sending Foo/home → Bar
15:18:52 - [NOTICE] Sent 151.0M in 131 files (out of 1.37G in 12,418)
15:20:26 - [NOTICE] Sending Foo/local → Bar
15:21:02 - [NOTICE] Sent 0b in 0 files (out of 6.45G in 207,453)

If you are wondering about the complexity of keeping filesets separate from containers, it's because I use pushme for several different scenarios: rsync'ing between machines, rsync'ing into ZFS filesystems, and sending ZFS filesystems as snapshot stream between servers. Pushme is able to handle all of these scenarios, but doing so required a bit more abstraction than a simple rsync-only tool would have needed.

Some common options include:

$ pushme -c quick bar       # only sync filesets with the "quick" class
$ pushme -f home bar        # only sync the "home" fileset
$ pushme -n bar             # don't do anything, just scan
$ pushme -N bar             # don't even scan
$ pushme -v bar             # show commands being executed
$ pushme -D bar             # show more info than you want to see