git-annex (10.20230126) upstream; urgency=medium * Change --metadata comparisons < > <= and >= to fall back to lexicographical comparisons when one or both values being compared are not numbers. * Improve handling of some .git/annex/ subdirectories being on other filesystems, in the bittorrent special remote, and youtube-dl integration, and git-annex addurl. * Added --anything (and --nothing). Eg, git-annex find --anything will list all annexed files whether or not the content is present. This is slightly faster and clearer than --include=* or --exclude=* * Speed up git-annex upgrade (from v5) and init in a repository that has submodules. * Added libgcc_s.so.1 to the linux standalone build so pthread_cancel will work. * Speed up initial scanning for annexed files when built with persistent-2.14.4.1 * Allow initremote of additional special remotes with type=web, in addition to the default web special remote. When --sameas=web is used, these provide additional names for the web special remote, and may also have their own additional configuration and cost. * web: Add urlinclude and urlexclude configuration settings. * Added an optional cost= configuration to all special remotes. * adb: Support the remote.name.cost and remote.name.cost-command configs. * findkeys: New command, very similar to git-annex find but operating on keys. * move, copy: Support combining --from and --to, which will send content from one remote across to another remote. -- Joey Hess <id@joeyh.name> Thu, 26 Jan 2023 15:26:22 -0400 git-annex (10.20221212) upstream; urgency=medium * Fix a hang that occasionally occurred during commands such as move, when operating on unlocked files. (A bug introduced in 10.20220927) * When youtube-dl is not available in PATH, use yt-dlp instead. * Support parsing yt-dpl output to display download progress. * init: Avoid scanning for annexed files, which can be lengthy in a large repository. Instead that scan is done on demand. * Sped up the initial scan for annexed files by 21%. * test: Add --test-debug option. * Support quettabyte and yottabyte. -- Joey Hess <id@joeyh.name> Mon, 12 Dec 2022 12:20:06 -0400 git-annex (10.20221104) upstream; urgency=medium * export: Fix a bug that left a file on a special remote when two files with the same content were both deleted in the exported tree. * S3: Support signature=anonymous to access a S3 bucket anonymously. This can be used, for example, with importtree=yes to import from a public bucket. This feature needs git-annex to be built with aws-0.23. -- Joey Hess <id@joeyh.name> Thu, 03 Nov 2022 14:07:31 -0400 git-annex (10.20221103) upstream; urgency=medium * Doubled the speed of git-annex drop when operating on many files, and of git-annex get when operating on many tiny files. * trust, untrust, semitrust, dead: Fix behavior when provided with multiple repositories to operate on. * trust, untrust, semitrust, dead: When provided with no parameters, do not operate on a repository that has an empty name. * move: Fix openFile crash with -J (Fixes a reversion in 8.20201103) * S3: Speed up importing from a large bucket when fileprefix= is set, by only asking for files under the prefix. * When importing from versioned remotes, fix tracking of the content of deleted files. * More robust handling of ErrorBusy when writing to sqlite databases. * Avoid hanging when a suspended git-annex process is keeping a sqlite database locked. * Make --batch mode handle unstaged annexed files consistently whether the file is unlocked or not. Note that this changes the behavior of --batch when it is provided with locked files that are in the process of being added to the repository, but have not yet been staged in git. * Make git-annex enable-tor work when using the linux standalone build. -- Joey Hess <id@joeyh.name> Thu, 03 Nov 2022 14:07:31 -0400 git-annex (10.20221003) upstream; urgency=medium * Avoid displaying warning about git-annex restage needing to be run in situations where it does not. * Fix the annex.adviceNoSshCaching config, which has never worked. Thanks, Reiko Asakura -- Joey Hess <id@joeyh.name> Mon, 03 Oct 2022 12:38:48 -0400 git-annex (10.20220927) upstream; urgency=medium * Fix a bug in the last release that caused v8 repositories to upgrade immediately to v10, rather than taking the scheduled 1 year to do so. * annex.diskreserve default increased from 1 mb to 100 mb. * Include the assistant and webapp when building with cabal 3.4.1.0. * Merged the webapp build flag into the assistant build flag. * Optimise linker in linux standalone tarballs. * Fix crash importing from a directory special remote that contains a broken symlink. * When accessing a git remote over http needs a git credential prompt for a password, cache it for the lifetime of the git-annex process, rather than repeatedly prompting. * Use curl for downloads from git remotes when annex.url-options is set. * Fix a reversion that made dead keys not be skipped when operating on all keys via --all or in a bare repo. (Introduced in version 8.20200720) * vicfg: Include mincopies configuration. * Improve handling of directory special remotes with importtree=yes whose ignoreinode setting has been changed. When getting a file from such a remote, accept the content that would have been accepted with the previous ignoreinode setting. * directory, adb: Fixed a bug with importtree=yes and multiple files in the special remote have the same content, that caused it to refuse to get a file from the special remote, incorrectly complaining that it had changed, due to only accepting the inode+mtime of one file (that was since modified or deleted) and not accepting the inode+mtime of other duplicate files. * Fix a reversion that prevented git-annex from working in a repository when --git-dir or GIT_DIR is specified to relocate the git directory to somewhere else. (Introduced in version 10.20220525) * Improved handling of --time-limit when combined with -J * Fix updating git index file after getting an unlocked file when annex.stalldetection is set. * restage: New git-annex command, handles restaging unlocked files. * test: Added --test-with-git-config option. * Run annex.freezecontent-command and annex.thawcontent-command when on a crippled filesystem. Thanks, Reiko Asakura * enable-tor: Fix breakage caused by git's fix for CVE-2022-24765. * Let GIT_DIR and --git-dir override git's protection against operating in a repository owned by another user. * p2p: Pass wormhole the --appid option before the receive/send command, as it does not accept that option after the command * Support "inbackend" in preferred content expressions. -- Joey Hess <id@joeyh.name> Tue, 27 Sep 2022 13:31:18 -0400 git-annex (10.20220822) upstream; urgency=medium * v8 repositories now automatically upgrade to v9, which will in turn automatically upgrade to v10 in a year's time. To avoid this upgrade, you can set annex.autoupgraderepository to false. * Now uses v10 by default for new repositories. * Fix a regression in 10.20220624 that caused git-annex add to crash when there was an unstaged deletion. * Added new matching options --want-get-by and --want-drop-by. * Allow find --branch to be used in a bare repository, the same as the deprecated findref can be. * add --dry-run: New option. * import: Avoid following symbolic links inside directories being imported. * Work around bug in git 2.37 that causes a segfault when core.untrackedCache is set, which broke git-annex init. * Added annex.dbdir config which can be used to move sqlite databases to a different filesystem than the git-annex repo, when the repo is on a filesystem that sqlite does not work well in. * Use curl when annex.security.allowed-url-schemes includes an url scheme not supported by git-annex internally, as long as annex.security.allowed-ip-addresses is configured to allow using curl. * Improve output when storing to bup. * When bup split fails, display its stderr. * Avoid running multiple bup split processes concurrently, since bup is not concurrency safe. * Avoid starting an unnecessary number of git hash-object processes when concurrency is enabled. -- Joey Hess <id@joeyh.name> Mon, 22 Aug 2022 11:56:06 -0400 git-annex (10.20220724) upstream; urgency=medium * filter-process: Fix a bug involving handling of empty files, that caused git to kill git-annex filter-process. * add: Fix reversion when adding an annex link that has been moved to another directory. (Introduced in version 10.20220624) * Added annex.alwayscompact setting which can be unset to speed up writes to the git-annex branch in some cases. See its documentation for important notes on when it's appropariate to use. * adb: Added configuration setting oldandroid=true to avoid using find -printf, which was first supported in Android around 2019-2020. This may need to be enabled for old android devices that used to work without it being set, since version 10.20220222 started using find -printf. * --backend is no longer a global option, and is only accepted by commands that actually need it. * Improve handling of parallelization with -J when copying content from/to a git remote that is a local path. * S3: Avoid writing or checking the uuid file in the S3 bucket when importtree=yes or exporttree=yes. * Fix a reversion that prevented --batch commands (and the assistant) from noticing data written to the journal by other commands. * Fix building with the Assistant build flag disabled but the Webapp build flag enabled. -- Joey Hess <id@joeyh.name> Mon, 25 Jul 2022 12:55:38 -0400 git-annex (10.20220624) upstream; urgency=medium * init: Added --no-autoenable option. * info: Added --autoenable option. * initremote: Improve handling of type=git special remotes. The location value no longer needs to match the url of an existing git remote, and locations not using ssh:// will work now, including both paths and host:/path * Fix retrival of an empty file that is stored in a special remote with chunking enabled. (Fixes a reversion in 8.20201103) * move: Improve resuming a move that succeeded in transferring the content, but where dropping failed due to eg a network problem, in cases where numcopies checks prevented the resumed move from dropping the object from the source repository. * add, fix, lock, rekey: When several files were being processed, replacing an annex symlink of a file that was already processed with a new large file could sometimes cause that large file to be added to git. These races have been fixed. * add: Also fix a similar race that could cause a large file be added to git when a small file was modified or overwritten while it was being added. * add --batch: Fix handling of a file that is skipped due to being gitignored. -- Joey Hess <id@joeyh.name> Thu, 23 Jun 2022 13:31:22 -0400 git-annex (10.20220525) upstream; urgency=medium * Special remotes with importtree=yes or exporttree=yes are once again treated as untrusted, since files stored in them can be deleted or modified at any time. (Fixes a reversion in 8.20201129) * Added support for "megabit" and related bandwidth units in annex.stalldetection and everywhere else that git-annex parses data units. Note that the short form is "Mbit" not "Mb" because that differs from "MB" only in case, and git-annex parses units case-insensitively. * Special remotes using exporttree=yes and/or importtree=yes now checksum content while it is being retrieved, instead of in a separate pass at the end. * fsck: Fix situations where the annex object file is stored in a directory structure other than where annex symlinks point to. * Deal with git's recent changes to fix CVE-2022-24765, which prevent using git in a repository owned by someone else. * Improve an error message displayed in that situation. * Prevent git-annex init incorrectly reinitializing the repository in that situation. * test: When limiting tests to run with -p, work around tasty limitation by automatically including dependent tests. -- Joey Hess <id@joeyh.name> Wed, 25 May 2022 13:44:46 -0400 git-annex (10.20220504) upstream; urgency=medium * Ignore annex.numcopies set to 0 in gitattributes or git config, or by git-annex numcopies or by --numcopies, since that configuration would make git-annex easily lose data. Same for mincopies. * assistant: When annex.autocommit is set, notice commits that the user makes manually, and push them out to remotes promptly. * multicast: Support uftp 5.0 by switching from aes256-cbc to aes256-gcm. * Fix test failure on NFS when cleaning up gpg temp directory. * Fix a build failure with ghc 9.2.2. Thanks, gnezdo for the patch. * rsync 3.2.4 broke backwards-compatability by preventing exposing filenames to the shell. Made the rsync and gcrypt special remotes detect this and disable shellescape. Closes: #1010397 * repair: Avoid treating refs/annex/last-index or other refs that are not commit objects as evidence of repository corruption. -- Joey Hess <id@joeyh.name> Wed, 04 May 2022 11:45:27 -0400 git-annex (10.20220322) upstream; urgency=medium * Directory special remotes with importtree=yes have changed to once more take inodes into account. This will cause extra work when importing from a directory on a FAT filesystem that changes inodes on every mount. To avoid that extra work, set ignoreinodes=yes when initializing a new directory special remote, or change the configuration of your existing remote: git-annex enableremote foo ignoreinodes=yes * add: Avoid unncessarily converting a newly unlocked file to be stored in git when it is not modified, even when annex.largefiles does not match it. * The above change to add fixes a reversion in version 10.20220222, where git-annex unlock followed by git-annex add, followed by git commit file could result in git thinking the file was modified after the commit. * Detect when an unlocked file whose content is not present has gotten some other content appended to it, and avoid treating it as a pointer file, so that appended content will not be checked into git, but will be annexed like any other file. * smudge: Warn when encountering a pointer file that has other content appended to it. * When annex.freezecontent-command is set, and the filesystem does not support removing write bits, avoid treating it as a crippled filesystem. * Run annex.thawcontent-command before deleting an object file, in case annex.freezecontent-command did something that would prevent deletion. * Fix propagation of nonzero exit status from git ls-files when a specified file does not exist, or a specified directory does not contain any files checked into git. * Fix build with aeson 2.0. Thanks, sternenseemann for the patch. * Avoid git-annex test being very slow when run from within the standalone linux tarball or OSX app. * test: Runs tests in parallel to speed up the test suite. * test: Added --jobs option. * Removed vendored copy of http-client-restricted, and removed the HttpClientRestricted build flag that avoided that dependency. * Removed the NetworkBSD build flag. -- Joey Hess <id@joeyh.name> Tue, 22 Mar 2022 13:56:12 -0400 git-annex (10.20220222) upstream; urgency=medium * annex.skipunknown now defaults to false, so commands like `git annex get foo*` will not silently skip over files/dirs that are not checked into git. * info: Allow using matching options in more situations. File matching options like --include will be rejected in situations where there is no filename to match against. * adb: Avoid find failing with "Argument list too long" * Fix git-annex forget propagation between repositories. (reversion introduced in version 7.20190122) * registerurl, unregisterurl: Improved output when reading from stdin to be more like other batch commands. * registerurl, unregisterurl: Added --json and --json-error-messages options. * Avoid git status taking a long time after git-annex unlock of many files. * Pass --no-textconv when running git diff internally. -- Joey Hess <id@joeyh.name> Tue, 22 Feb 2022 13:01:20 -0400 git-annex (10.20220127) upstream; urgency=medium * New v10 repository version (with v9 as a stepping-stone to it). v8 remains the default version for now. * In v10, object files are locked using separate lock files. This allows the object files to be kept non-writable even in repositories where core.sharedRepository is set. * The v10 upgrade will happen automatically, one year after the v9 upgrade, in order to allow time for any old git-annex processes that are not aware of the locking change to finish. Or git-annex upgrade can be used to upgrade to v10 immediately. * In v9 upgrade, set filter.annex.process. This makes git add/checkout faster when there are a lot of unlocked annexed files or non-annexed files, but can also make git add of large files to the annex somewhat slower. If this tradeoff does not work for your use case, you can still unset filter.annex.process. * export: When a non-annexed symlink is in the tree to be exported, skip it. * import: When the previously exported tree contained a non-annexed symlink, preserve it in the imported tree so it does not get deleted. * enableremote, renameremote: Better handling of the unusual case where multiple special remotes have been initialized with the same name. * Recover from corrupted content being received from a git remote, by deleting the temporary file when it fails to verify. This prevents a retry from failing again. (reversion introduced in version 8.20210903) * adb: Added ignorefinderror configuration parameter. * Avoid crashing when run in a bare git repo that somehow contains an index file. * Reject combinations of --batch (or --batch-keys) with options like --all or --key or with filenames. -- Joey Hess <id@joeyh.name> Thu, 27 Jan 2022 13:25:19 -0400 git-annex (8.20211231) upstream; urgency=medium * Improved support for using git-annex in a read-only repository, git-annex branch information from remotes that cannot be merged into the git-annex branch will now not crash it, but will be merged in memory. * addurl, youtube-dl: When --check-raw prevents downloading an url, still continue with any downloads that come after it, rather than erroring out. * Fix locking problems when annex.pidlock is set and concurrency is enabled eg with -J. * Improve error message display when autoinit fails due to eg, a permissions problem. * export: Avoid unncessarily re-exporting non-annexed files that were already exported. * Improve git command queue flushing so that eg, addurl of several large files that take time to download will update the index for each file, rather than deferring the index updates to the end. * sync: Better error message when unable to export to a remote because remote.name.annex-tracking-branch is configured to a ref that does not exist. * Fix build with ghc 9.0.1 * Fix build with old versions of feed library. -- Joey Hess <id@joeyh.name> Fri, 31 Dec 2021 15:03:36 -0400 git-annex (8.20211123) upstream; urgency=medium * Bugfix: When -J was enabled, getting files could leak an ever-growing number of git cat-file processes. * Support git's new "ort" resolver, which became the default in git 2.34.0, and broke the test suite and automatic merge resolution of a conflict between an annexed file and a non-annexed file. * importfeed: Display url before starting youtube-dl download. -- Joey Hess <id@joeyh.name> Tue, 23 Nov 2021 15:19:16 -0400 git-annex (8.20211117) upstream; urgency=medium * filter-process: New command that can make git add/checkout faster when there are a lot of unlocked annexed files or non-annexed files, but that also makes git add of large annexed files slower. Use it by running: git config filter.annex.process 'git-annex filter-process' * Fix a typo in the name of youtube-dl (reversion introduced in version 8.20210903) * git-lfs: Fix interoperability with gitlab's implementation of the git-lfs protocol, which requests Content-Encoding chunked. * importfeed: Fix a crash when used in a non-unicode locale. * migrate: New --remove-size option. * uninit: Avoid error message when no commits have been made to the repository yet. * uninit: Avoid error message when there is no git-annex branch. * metadata --batch: Avoid crashing when a non-annexed file is input, instead output a blank line like other batch commands do. * metadata --batch --json: Reject input whose "fields" does not consist of arrays of strings. Such invalid input used to be silently ignored. -- Joey Hess <id@joeyh.name> Wed, 17 Nov 2021 12:18:49 -0400 git-annex (8.20211028) upstream; urgency=medium * Removed support for accessing git remotes that use versions of git-annex older than 6.20180312. * git-annex-shell: Removed several commands that were only needed to support git-annex versions older than 6.20180312. * Negotiate P2P protocol version with tor remotes, allowing use of protocol version 1. This negotiation is not supported by versions of git-annex older than 6.20180312. * Fix bug that caused stale git-annex branch information to read when annex.private or remote.name.annex-private is set. * git-annex get when run as the first git-annex command in a new repo did not populate all unlocked files. (Reversion in version 8.20210621) * Fix using lookupkey and info inside a subdirectory of the git repository. (Reversion in version 8.20211011) * Avoid some sqlite crashes on Windows SubSystem for Linux (WSL). * Call annex.freezecontent-command on the annex object file after it has been moved into place in annex/objects/. This allows the hook to freeze the file in ways that prevent moving it, such as removing the Windows delete permission. Thanks, Reiko Asakura. * addurl: Support adding the same url to multiple files at the same time when using -J with --batch --with-files. * When retrieval from a chunked remote fails, display the error that occurred when downloading the chunk, rather than the error that occurred when trying to download the unchunked content, which is less likely to actually be stored in the remote. * Avoid crashing tilde expansion on user who does not exist. * test: Put gpg temp home directory in system temp directory, not filesystem being tested. -- Joey Hess <id@joeyh.name> Thu, 28 Oct 2021 11:58:29 -0400 git-annex (8.20211011) upstream; urgency=medium * Added annex.bwlimit and remote.name.annex-bwlimit config to limit the bandwidth of transfers. It works for git remotes and many but not all special remotes. * Bug fix: Git configs such as annex.verify were incorrectly overriding per-remote git configs such as remote.name.annex-verify. (Reversion in version 4.20130323) * borg: Significantly improved memory use when a borg repository contains many archives. * borg: Avoid trying to extract xattrs, ACLS, and bsdflags when retrieving from a borg repository. * Sped up git-annex smudge --clean by 25%. * Resume where it left off when copying a file to/from a local git remote was interrupted. * sync --content: Avoid a redundant checksum of a file that was incrementally verified, when used on NTFS and perhaps other filesystems. * reinject: Fix crash when reinjecting a file from outside the repository. (Reversion in version 8.20210621) * Avoid cursor jitter when updating progress display. -- Joey Hess <id@joeyh.name> Mon, 11 Oct 2021 12:52:14 -0400 git-annex (8.20210903) upstream; urgency=medium * Deal with clock skew, both forwards and backwards, when logging information to the git-annex branch. * GIT_ANNEX_VECTOR_CLOCK can now be set to a fixed value (eg 1) rather than needing to be advanced each time a new change is made. * Misuse of GIT_ANNEX_VECTOR_CLOCK will no longer confuse git-annex. * add: When adding a dotfile, avoid treating its name as an extension. * rsync special remote: Stop displaying rsync progress, and use git-annex's own progress display. * Many special remotes now checksum content while it is being retrieved, instead of in a separate pass at the end. This is supported for all special remotes on Linux (except for bittorrent), and for many on other OS's (except for adb, external, gcrypt, hook, and rsync). * unused: Skip the refs/annex/last-index ref that git-annex recently started creating. * Fix test suite failure on Windows. * New --batch-keys option added to these commands: get, drop, move, copy, whereis * Added annex.youtube-dl-command config. This can be used to run some forks of youtube-dl. * Run cp -a with --no-preserve=xattr, to avoid problems with copied xattrs, including them breaking permissions setting on some NFS servers. * add, import: Detect when xattrs or perhaps ACLs prevent removing write permissions from an annexed file, and fail with an informative message. * Fix support for readonly git remotes. (Reversion in version 8.20210621) * When downloading urls fail, explain which urls failed for which reasons. * web: Avoid displaying a warning when downloading one url failed but another url later succeeded. -- Joey Hess <id@joeyh.name> Fri, 03 Sep 2021 12:00:46 -0400 git-annex (8.20210803) upstream; urgency=medium * whereused: New command, finds what files use a key, or where a key was used historically. * Fix a bug that prevented getting content from a repository that started out as a bare repository, or had annex.crippledfilesystem set, and was converted to a non-bare repository. * Fix retrieval of content from borg repos accessed over ssh. * sync: When --quiet is used, run git commit, push, and pull without their ususual output. * merge: When --quiet is used, run git merge without its usual output. * sync, merge, post-receive: Avoid merging unrelated histories, which used to be allowed only to support direct mode repositories. (However, sync does still merge unrelated histories when importing trees from special remotes, and the assistant still merges unrelated histories.) * sync, merge: Added --allow-unrelated-histories option, which is the same as the git merge option. * Fix bug that caused some transfers to incorrectly fail with "content changed while it was being sent", when the content was not changed. * Fix bug that could prevent pointer files from being populated, in a repository that was upgraded from v7. * fsck: Detect and correct stale or missing inode caches. * Fix a rounding bug in display of data sizes. * git-annex get when run as the first git-annex command in a new repo did not populate unlocked files. (Reversion in version 8.20210621) -- Joey Hess <id@joeyh.name> Tue, 03 Aug 2021 12:20:09 -0400 git-annex (8.20210714) upstream; urgency=medium * assistant: Avoid unnecessary git repository repair in a situation where git fsck gets confused about a commit that is made while it's running. * addurl: Avoid crashing when used on beegfs. * --debug output goes to stderr again, not stdout. (Reversion in version 8.20210428) * init: Fix misbehavior when core.sharedRepository = group that caused it to enter an adjusted branch and set annex.crippledfilesystem (Reversion in version 8.20210630) * assistant: When adding non-large files to git, honor annex.delayadd configuration. Also, don't add non-large files to git when they are still being written to. -- Joey Hess <id@joeyh.name> Wed, 14 Jul 2021 12:22:55 -0400 git-annex (8.20210630) upstream; urgency=medium * Fixed bug that interrupting git-annex repair (or assistant) while it was fixing repository corruption would lose objects that were contained in pack files. Also the assistant might sometimes have crashed with the same result. * repair: Fix reversion in version 8.20200522 that prevented fetching missing objects from remotes. * sync: Partly work around github behavior that first branch to be pushed to a new repository is assumed to be the head branch, by not pushing synced/git-annex first. * Added annex.freezecontent-command and annex.thawcontent-command configs. * Improve display of errors when transfers fail. * Dropping an unused object with drop --unused or dropunused will mark it as dead, preventing fsck --all from complaining about it after it's been dropped from all repositories. * addurl, importfeed: Added --no-raw option that forces download with youtube-dl or a special remote. In particular this can avoid falling back to raw download when youtube-dl is blocked for some reason. -- Joey Hess <id@joeyh.name> Wed, 30 Jun 2021 11:48:16 -0400 git-annex (8.20210621) upstream; urgency=medium * New matching options --excludesamecontent and --includesamecontent * When two files have the same content, and a required content expression matches one but not the other, dropping the latter file will fail as it would also remove the content of the required file. * drop, move, mirror: When two files have the same content, and different numcopies or requiredcopies values, use the higher value. * drop --auto: When two files have the same content, and a preferred content expression matches one but not the other, do not drop the content. * sync --content, assistant: When two unlocked files have the same content, and a preferred content expression matches one but not the other, do not drop the content. (This was already the case for locked files.) * sync --content, assistant: Fix an edge case where a file that is not preferred content did not get dropped. * filter-branch: New command, useful to produce a filtered version of the git-annex branch, eg when splitting a repository. * fromkey: Create an unlocked file when used in an adjusted branch where the file should be unlocked, or when configured by annex.addunlocked. * Fix behavior of several commands, including reinject, addurl, and rmurl when given an absolute path to an unlocked file, or a relative path that leaves and re-enters the repository. * smudge: Fix a case where an unlocked annexed file that annex.largefiles does not match could get its unchanged content checked into git, due to git running the smudge filter unecessarily. * reinject: Error out when run on a file that is not annexed, rather than silently skipping it. * assistant: Fix a crash on startup by avoiding using forkProcess. * init: When annex.commitmessage is set, use that message for the commit that creates the git-annex branch. * Added annex.adviceNoSshCaching config. * Added --size-limit option. * Future proof activity log parsing. * Fix an exponential slowdown when large numbers of duplicate files are being added in unlocked form. -- Joey Hess <id@joeyh.name> Mon, 21 Jun 2021 12:17:24 -0400 git-annex (8.20210428) upstream; urgency=medium * New annex.private and remote.name.annex-private configs that can prevent storing information about a repository and remotes in the git-annex branch. * initremote: Added --private option to set up a private special remote. * importfeed: Made "checking known urls" phase run 12 times faster. * Added --debugfilter (and annex.debugfilter) * diffdriver: Support unlocked files. * forget: Preserve currently exported trees, avoiding problems with exporttree remotes in some unusual circumstances. * fsck: When downloading content from a remote, if the content is able to be verified during the transfer, skip checksumming it a second time. * directory: When cp supports reflinks, use it. * Avoid excess commits to the git-annex branch when stall detection is enabled. * git-annex-config: Allow setting annex.securehashesonly, which has otherwise been supported since 2019, but was missing from the list of allowed repo-global configs. * init: Fix a crash when the repo's was cloned from a repo that had an adjusted branch checked out, and the origin remote is not named "origin". * Fix some bugs that made git-annex not see recently recorded status information when configured with annex.alwayscommit=false. * When mincopies is set to a larger value than numcopies, make sure that mincopies is satisfied. Before, it assumed a sane configuration would have numcopies larger or equal to mincopies. It's still a good idea not to configure git-annex this way. * Avoid more than 1 gpg password prompt at the same time, which could happen occasionally before when concurrency is enabled. * Fix build with persistent-2.12.0.1 -- Joey Hess <id@joeyh.name> Wed, 28 Apr 2021 12:13:24 -0400 git-annex (8.20210330) upstream; urgency=medium * Behavior change: When autoenabling special remotes of type S3, weddav, or glacier, do not take login credentials from environment variables, as the user may not be expecting the autoenable to happen, or may have those set for other purposes and not intend git-annex to use them. * New annex.supportunlocked config that can be set to false to avoid some expensive things needed to support unlocked files, if you do not use them. * Fix bug importing from a special remote into a subdirectory more than one level deep, which generated unusual git trees that could confuse git merge. * borg: Fix a bug that prevented importing keys of type URL and WORM. * borg: Support importing files that are hard linked in the borg backup. * export: When a submodule is in the tree to be exported, skip it. * import: When the previously exported tree contained a submodule, preserve it in the imported tree so it does not get deleted. * export --json: Fill in the file field. * rmurl: When youtube-dl was used for an url, it no longer needs to be prefixed with "yt:" in order to be removed. * rmurl: If an url is both used by the web and also claimed by another special remote, fix a bug that caused the url to to not be removed. * unregisterurl: Fix a bug that caused an url to not be unregistered when it is claimed by a special remote other than the web. * whereis: Don't include yt: prefix when showing url to content retrieved with youtube-dl. * webdav: Work around some buggy webdav server behavior involving renaming files. * Make --debug also enable debugging in child git-annex processes. * fsck: When --from is used in combination with --all or similar options, do not verify required content, which can't be checked properly when operating on keys. * Sped up git-annex init in a clone of an existing repository. * Improved display of errors when accessing a git http remote fails. * Fix build with attoparsec-0.14. -- Joey Hess <id@joeyh.name> Tue, 30 Mar 2021 13:01:17 -0400 git-annex (8.20210310) upstream; urgency=medium * When non-annexed files in a tree are exported to a special remote, importing from the special remote keeps the files non-annexed, as long as their content has not changed, rather than converting them to annexed files. (Such a conversion will still happen when importing from a remote an old git-annex exported such a tree to before; export the tree with the new git-annex before importing to avoid that.) * Added support for git-remote-gcrypt's rsync URIs, which access a remote using rsync over ssh, and which git pushes to much more efficiently than ssh urls. * unregisterurl: New command. * registerurl: Allow it to be used in a bare repository. * Prevent combinations of options such as --all with --include. * Fixed handling of --mimetype or --mimeencoding combined with options like --all or --unused. * Fix handling of --branch combined with --unlocked or --locked. * Fix support for local gcrypt repositories with a space in their URI. * uninit: Fix a small bug that left a lock file in .git/annex * Windows: Correct the path to the html help file for 64 bit build. * OSX dmg: Updated bundled git to 2.30.2 which fixes CVE-2021-21300. -- Joey Hess <id@joeyh.name> Wed, 10 Mar 2021 13:58:15 -0400 git-annex (8.20210223) upstream; urgency=medium * annex.stalldetection can now be set to "true" to make git-annex do automatic stall detection when it detects a remote is updating its transfer progress consistently enough for stall detection to work. * When annex.stalldetection is not enabled and a likely stall is detected, display a suggestion to enable it. * Commands can be added to git-annex, by installing a program in PATH with a name starting with "git-annex-" * Fix a reversion that made import of a tree from a special remote result in a merge that deleted files that were not preferred content of that special remote. * Bugfix: fsck --from a ssh remote did not actually check that the content on the remote is not corrupted. * unannex, uninit: When an annexed file is modified, don't overwrite the modified version with an older version from the annex. * When adding files to an adjusted branch set up by --unlock-present, add them unlocked, not locked. * Fix an oddity in matching options and preferred content expressions such as "foo (bar or baz)", which was incorrectly handled as if it were "(foo or bar) and baz)" rather than the intended "foo and (bar or baz)" * Checksum as content is received from a remote git-annex repository, rather than doing it in a second pass. * Tahoe: Avoid verifying hash after download, since tahoe does sufficient verification itself. * unannex, uninit: Don't run git rm once per annexed file, for a large speedup. * When a git remote is configured with an absolute path, use that path, rather than making it relative. * get: Improve output when failing to get a file fails. * Fix build on openbsd. Thanks, James Cook for the patch. * Include libkqueue.h file needed to build the assistant on BSDs. -- Joey Hess <id@joeyh.name> Tue, 23 Feb 2021 14:40:14 -0400 git-annex (8.20210127) upstream; urgency=medium * Added mincopies configuration. This is like numcopies, but is enforced even more strictly. While numcopies can be violated in concurrent drop situations involving special remotes that do not support locking, mincopies cannot be. The default value has always been 1, but now it can be set to higher values if desired. * Behavior change: When numcopies is set to 0, git-annex used to drop content without requiring any copies. Now to get that (highly unsafe) behavior, mincopies also needs to be set to 0. * Behavior change: git-annex trust now needs --force, since unconsidered use of trusted repositories can lead to data loss. * Behavior change: --trust and --trust-glacier options no longer overrides trust, since that can lead to data loss, which should never be enabled by an option other than --force. * add: Significantly speed up adding lots of non-large files to git, by disabling the annex smudge filter when running git add. * add --force-small: Run git add rather than updating the index itself, so any other smudge filters than the annex one that may be enabled will be used. * Fix --time-limit, which got broken in several ways by some optimisations in version 8.20201007. * When syncing changes back from an adjusted branch to the basis branch, include deletions of submodules. Thanks, Kyle Meyer for the patch. * Bug fix: export with -J could fail when two files had the same content. * Bug fix: Fix tilde expansion in ssh urls when the tilde is the last character in the url. Thanks, Grond for the patch. * Avoid crashing when there are remotes using unparseable urls. Including the non-standard URI form that git-remote-gcrypt uses for rsync. * Directory special remotes with importtree=yes now avoid unnecessary hashing when inodes of files have changed, as happens whenever a FAT filesystem gets remounted. * Fix a bug that prevented git-annex init from working in a submodule. * Fix a bug in view filename generation when a metadata value ended with "/" (or ":" or "\" on Windows) * adjust: Fix some bad behavior when unlocked files use URL keys. * smudge: Fix some bad behavior when git add is run on an unlocked file that used an URL key. * Added GETGITREMOTENAME to external special remote protocol. * Windows: Work around win32 length limits when dealing with lock files. -- Joey Hess <id@joeyh.name> Wed, 27 Jan 2021 11:09:25 -0400 git-annex (8.20201129) upstream; urgency=medium * New borg special remote. This is a new kind of remote, that examines borg backups of git-annex repositories, learns what files have been backed up, and can restore files from the backup and so on. As well as backups, it can be useful for archival storage, since borg can efficiently store many related versions of files. * New config annex.stalldetection, remote.name.annex-stalldetection, which can be used to deal with remotes that stall during transfers, or are sometimes too slow to want to use. * Support special remotes that are configured with importtree=yes but without exporttree=yes. * Fix bug that made the next download after an empty file from a ssh or tor remote fail. * Avoid spurious "verification of content failed" message when downloading content from a ssh or tor remote fails due to the remote no longer having a copy of the content. * Fix bug that matched include= and exclude= in preferred/required content expressions relative to the current directory, rather than the path from the top of the repository. (Reversion introduced in version 8.20201116.) * Fix hang on shutdown of external special remote using ASYNC protocol extension. (Reversion introduced in version 8.20201007.) * Guard against running in a repo where annex.uuid is set but annex.version is not set, or vice-versa. * Avoid autoinit when a repo does not have annex.version or annex.uuid set, but has a git-annex objects directory, suggesting it was used by git-annex before, and the git config may have been lost. * importfeed: Avoid using youtube-dl when a feed does not contain an enclosure, but only a link to an url which youtube-dl does not support. * initremote: Prevent enabling encryption with exporttree=yes or importtree=yes. * Windows: include= and exclude= containing '/' will also match filenames that are written using '\'. (And vice-versa, but it's better to use '/' for portability.) * Fix a bug that could prevent getting files from an importtree=yes remote, because the imported tree was allowed to be garbage collected. * stack.yaml: Updated to lts-16.27. -- Joey Hess <id@joeyh.name> Tue, 29 Dec 2020 12:52:58 -0400 git-annex (8.20201127) upstream; urgency=medium * adjust: New --unlock-present mode which locks files whose content is not present (so the broken symlink is visible), while unlocking files whose content is present. * Added annex.adjustedbranchrefresh git config to update adjusted branches set up by git-annex adjust --unlock-present/--hide-missing. * Fix hang when an external special remote program exited but the stderr pipe to it was left open, due to a daemon having inherited the file descriptor. * Fix a bug that could make resuming a download from the web fail when the entire content of the file is actually already present locally. * examinekey: Added a "file" format variable for consistency with find, and for easier scripting. * init: When writing hook scripts, set all execute bits, not only the user execute bit. * upgrade: Support an edge case upgrading a v5 direct mode repo where nothing had ever been committed to the head branch. * Made the test suite significantly less noisy, only displaying command output when something failed. * Fix building without the torrent library. Thanks, Kyle Meyer. * Fix build on Windows. * Prevent windows assistant from trying (and failing) to upgrade itself, which has never been supported on windows. -- Joey Hess <id@joeyh.name> Fri, 27 Nov 2020 12:54:11 -0400 git-annex (8.20201116) upstream; urgency=medium * move: Fix a regression in the last release that made move --to not honor numcopies settings. * sync --content: Fix a bug where a file that was not preferred content could be transferred to a remote. This happened when the file got deleted after the sync started running. * Warn when adding a annex symlink or pointer file that uses a key that is not known to the repository, to prevent confusion if the user has copied it from some other repository. * examinekey: Added two new format variables: objectpath and objectpointer * examinekey: Added --migrate-to-backend -- Joey Hess <id@joeyh.name> Mon, 16 Nov 2020 09:36:35 -0400 git-annex (8.20201103) upstream; urgency=medium * move: Improve resuming a move that was interrupted after the object was transferred, in cases where numcopies checks prevented the resumed move from dropping the object from the source repository. * When a special remote has chunking enabled, but no chunk sizes are recorded (or the recorded ones are not found), speculatively try chunks using the configured chunk size. * Fixed some problems that prevented this command from working: git submodule foreach git annex init * Improve shutdown process for external special remotes and external backends. Make sure to relay any remaining stderr from the process, and potentially avoid the process getting a SIGPIPE if it writes to stderr too late. * Fix a bug that prevented linux standalone bundle from working on a fresh install. * Windows build changed to one done by the datalad-extensions project using Github actions. * Windows build now includes libmagic, so mimetype and mimeencoding will work. Thanks to John Thorvald Wodder II and Yaroslav Halchenko for their work on this. * view: Avoid using ':' from metadata when generating a view, because it's a special character on Windows ("c:") * Fix a memory leak introduced in the last release. * add, import: Fix a reversion in 7.20191009 that broke handling of --largerthan and --smallerthan. * view: Fix a reversion in 8.20200522 that broke entering or changing views. * Fix build on Windows with network-3. * testremote: Display exceptions when tests fail, to aid debugging. -- Joey Hess <id@joeyh.name> Tue, 03 Nov 2020 11:40:56 -0400 git-annex (8.20201007) upstream; urgency=medium * --json output now includes a new field "input" which is the input value (filename, url, etc) that caused a json object to be output. * --batch combined with -J now runs batch requests concurrently for many commands. Before, the combination was accepted, but did not enable concurrency. Since the output of batch requests can be in any order, --json with the new "input" field is recommended to be used, to determine which batch request each response corresponds to. * aws-0.22 improved its support for setting etags, which improves support for versioned S3 buckets. * Serialize use of C magic library, which is not thread safe. This fixes failures uploading to S3 when using -J. * add, addurl, importfeed, import: Added --no-check-gitignore option for finer grained control than using --force. * import: Check gitignores when importing trees from special remotes. * addunused: Don't check .gitignores when adding files. * Improve the "Try making some of these repositories available" message, with some hints for the user for what to do. * Improve --debug output to show pid of processes that are started and stopped. * sync --all: Sped up seeking to around twice as fast, by avoiding a pass over the worktree files when preferred content expressions of the local repo and remotes don't use include=/exclude=. * Sped up seeking for files to operate on, when using options like --copies or --in, by around 20% * import --no-content: Check annex.largefiles, and import small files into git, the same as is done when importing with content. If the largefiles expression needs the file content available (due to mimetype or mimeencoding being used), the import will fail. * sync: When run without --content, import without copying from importtree=yes directory special remotes. (Other special remotes may support this later as well.) * addurl: Avoid a redundant git ignores check for speed. * upgrade: Avoid an upgrade failure of a bare repo in unusual circumstances. * httpalso: Support being used with special remotes that do not have encryption= in their config. * Parse youtube-dl progress output, which lets progress be displayed when doing concurrent downloads. * Fix build with Benchmark build flag. * Enable building with git-annex benchmark by default, only turning it off when the criterion library is not installed. * runshell: Fix a edge case where rm errors were sent to stdout, which could confuse things parsing git-annex output. * runshell: Update files atomically when preparing to run git-annex. * Fix a build failure on Windows. -- Joey Hess <id@joeyh.name> Wed, 07 Oct 2020 12:19:05 -0400 git-annex (8.20200908) upstream; urgency=medium * Added httpalso special remote, which is useful for accessing content stored on other remotes that is published by http. * The external special remote protocol got an ASYNC extension. This can be used by an external special remote to let a single process perform concurrent actions, rather than multiple processes being started, when that is more efficient. * Retry transfers to exporttree=yes remotes same as for other remotes. * import: Retry downloads that fail, same as is done for downloads generally. * Limit retrying of failed transfers when forward progress is being made to 5, to avoid some unusual edge cases where too much retrying could result in far more data transfer than makes sense. * Exposed annex.forward-retry git config, to configure the forward retry behavior that git-annex has had for a long time. * sync, assistant, merge: When merge.directoryRenames is not set, default it it to "false", which works better with automatic merge conflict resolution than git's ususual default of "conflict". (This is not done when automatic merge conflict resolution is disabled.) * resolvemerge: Improve cleanup of cruft left in the working tree by a conflicted merge. * Support git remotes where .git is a file, not a directory, eg when --separate-git-dir was used. * Fixed several cases where files were created without file mode bits that the umask would usually set. This included exports to the directory special remote, torrent files used by the bittorrent special remote, hooks written by git-annex init, and some log files in .git/annex/ * Fix reversion in 7.20190322 that made addurl --file not be honored when youtube-dl was used to download media. * Fix reversion in 8.20200617 that made annex.pidlock being enabled result in some commands stalling, particularly those needing to autoinit. * Display warning when external special remote does not start up properly, or is not usable. * Display a message when git-annex has to wait for a pid lock file held by another process. * test: Stop gpg-agent daemons that are started for the test framework's gpg key. * Removed the S3 and WebDAV build flags so these special remotes are always supported. * stack.yaml: Updated to lts-16.10. -- Joey Hess <id@joeyh.name> Tue, 08 Sep 2020 14:20:11 -0400 git-annex (8.20200814) upstream; urgency=medium * Added support for external backend programs. So if you want a hash that git-annex doesn't support, or something stranger, you can write a small program to implement it. * Fix a bug in find --branch in the previous version. * importfeed: Fix reversion that caused some '.' in filenames to be replaced with '_' * Fix a lock file descriptor leak that could occur when running commands like git-annex add with -J. Bug was introduced as part of a different FD leak fix in version 6.20160318. * Fix a hang when using git-annex with an old openssh 7.2p2, which had some weird inheriting of ssh FDs by sshd. Bug was introduced in git-annex version 7.20200202.7. * move, copy --to: Sped up seeking files by 2x. * drop: Sped up seeking files to drop by 2x, and also some performance improvements to checking numcopies. * Deal with unusual IFS settings in the shell scripts for linux standalone and OSX app. Thanks, Yaroslav Halchenko * Avoid complaining that a file with "is beyond a symbolic link" when the filepath is absolute and the symlink in question is not actually inside the git repository. * Slightly sped up the linux standalone bundle. * Support building with dlist-1.0 -- Joey Hess <id@joeyh.name> Mon, 10 Aug 2020 13:04:04 -0400 git-annex (8.20200720.1) upstream; urgency=medium * Fix a bug in find --batch in the previous version. -- Joey Hess <id@joeyh.name> Mon, 20 Jul 2020 19:39:11 -0400 git-annex (8.20200720) upstream; urgency=medium * import: Added --no-content option, which avoids downloading files from a special remote. Currently only supported by the directory special remote. * Honor annex.largefiles when importing a tree from a special remote. (Except for when --no-content is used.) * Fix a deadlock that could occur after git-annex got an unlocked file, causing the command to hang indefinitely. Known to happen on vfat filesystems, possibly others. * Build with the http-client-restricted and git-lfs libraries when available, otherwise use the vendored copy as before. * testremote: Fix over-allocation of resources and bad caching, including starting up a large number of external special remote processes. (Regression introduced in version 8.20200501) * test: Fix some test cases that assumed git's default branch name. * importfeed: Added some additional --template variables: itempubyear, itempubmonth, itempubday, itempubhour, itempubminute, itempubsecond. * Made several special remotes support locking content on them, which allows dropping from other special remotes in some situations where it was not possible before. Supported special remotes: S3 (with versioning=yes), git-lfs, tahoe * Fix reversion that broke passing annex.* and remote.*.annex-* git configs with -c. (Since version 8.20200330.) * Bring back git-annex branch read cache. This speeds up some operations, eg git-annex sync --content --all gets 20% faster. * Fix a recently introduced bug that could cause a "fork: resource exhausted" after getting several thousand files. * Sped up the --all option by 2x to 16x by using git cat-file --buffer. Thanks to Lukey for finding this optimisation. * Sped up seeking for annexed files to operate on by a factor of nearly 2x. * Sped up sync --content by 2x and other commands like fsck --fast and whereis by around 50%, by using git cat-file --buffer. * importfeed: Made checking known urls step around 15% faster. * fsck: Detect if WORM keys contain a carriage return, and recommend upgrading the key. (git-annex could have maybe created such keys back in 2013). * When on an adjust --hide-missing branch, fix handling of files that have been deleted but the deletion is not yet staged. -- Joey Hess <id@joeyh.name> Mon, 20 Jul 2020 14:40:51 -0400 git-annex (8.20200617) upstream; urgency=medium * Added annex.skipunknown git config, that can be set to false to change the behavior of commands like `git annex get foo*`, to not skip over files/dirs that are not checked into git and are explicitly listed in the command line. * annex.skipunknown is planned to change to default to false in a git-annex release in early 2022. If you prefer the current behavior, you can explicitly set it to true. * Try to enable special remotes configured with autoenable=yes when git-annex auto-initialization happens in a new clone of an existing repo. Previously, git-annex init had to be explicitly run to enable them. Special remotes cannot display anything when autoenabled this way, to avoid interfering with the output of git-annex query commands. * export: Added options for json output. * import: Added --json-progress. * addurl: Make --preserve-filename also apply when eg a torrent contains multiple files. * Fix a crash or potentially not all files being exported when sync -J --content is used with an export remote. * export: Let concurrent transfers be done with -J or annex.jobs. * move --to, copy --to, mirror --to: When concurrency is enabled, run cleanup actions in separate job pool from uploads. * init: If lock probing stalls for a long time (eg a broken NFS server), display a message to let the user know what's taking so long. * init: When annex.pidlock is set, skip lock probing. * Fix file descriptor leak when importing from a directory special remote that is configured with exporttree=yes. * Note that external special remote programs should not block SIGINT or SIGTERM. * Avoid creating the keys database during init when there are no unlocked files, to prevent init failing when sqlite does not work in the filesystem. * import: Avoid using some strange names for temporary keys, which broke importing from a directory special remote onto a vfat filesystem. * S3: The REDUCED_REDUNDANCY storage class is no longer cheaper so stop documenting it, and stop offering it as a choice in the assistant. * Improve display of problems auto-initializing or upgrading local git remotes. * When a local git remote cannot be initialized because it has no git-annex branch or a .noannex file, avoid displaying a message about it. * checkpresentkey: When no remote is specified, try all remotes, not only ones that the location log says contain the key. This is what the documentation has always said it did. * Fix regression in external special remote handling: GETCONFIG did not return a value that was set with SETCONFIG immediately before. (Regression introduced in version 7.20200202.7) * Fix bug that made initremote of extrnal special remotes with embedcreds=yes or gpg encryption not store the creds in the git-annex branch. git-annex-remote-googledrive one was special remote affected by this bug. (Regression introduced in version 7.20200202.7) * Fix bug that made creds not be stored in git when a special remote was initialized with gpg encryption, but without an explicit embedcreds=yes. (Regression introduced in version 7.20200202.7) * Fix a annex.pidlock issue that made eg git-annex get of an unlocked file hang until the annex.pidlocktimeout and then fail. -- Joey Hess <id@joeyh.name> Wed, 17 Jun 2020 15:58:59 -0400 git-annex (8.20200522) upstream; urgency=medium * Fix bug that made enableremote of S3 and webdav remotes, that have embedcreds=yes, fail to set up the embedded creds, so accessing the remotes failed. (Regression introduced in version 7.20200202.7) * addurl, importfeed: Avoid adding filenames with leading '.', instead it will be replaced with '_'. * addurl, importfeed: Allow '-' in filenames, as long as it's not the first character. * addurl --preserve-filename: New option, uses server-provided filename without any sanitization, but will fail if the filename has an obvious security problem like using an escape sequence or trying to escape the current directory. * whereis: Added --format option. * S3: Support signature=v4, to use S3 Signature Version 4. Some S3 services seem to require v4, while others may only support v2, which remains the default. * upgrade: When upgrade fails due to an exception, display it. * repair: Improve fetching from a remote with an url in host:path format. * git-lfs repos that encrypt the annexed content but not the git repo only need --force passed to initremote, allow enableremote and autoenable of such remotes without forcing again. * When accessing a remote fails, always display a reason why. * Display a warning message when a remote uses a protocol, such as git://, that git-annex does not support. Silently skipping such a remote was confusing behavior. * Also display a warning message when a remote, without a known uuid, is located in a directory that does not currently exist, to avoid silently skipping such a remote. * sync: Avoid an ugly error message when nothing has been committed to master yet and there is a synced master branch to merge from. * Display a warning message when asked to operate on a file inside a directory that's a symbolic link to elsewhere. * Support building with tasty-1.3. -- Joey Hess <id@joeyh.name> Fri, 22 May 2020 12:05:01 -0400 git-annex (8.20200501) upstream; urgency=medium * Improve git-annex's ability to find the path to its program, especially when it needs to run itself in another repo to upgrade it. * adb: Better messages when the adb command is not installed. * Sped up query commands that read the git-annex branch by around 9%. * Various speed improvements gained by using ByteStrings for git refs and shas. * Fix a potential failure to parse git config. * Support boolean git configs that are represented by the name of the setting with no value, eg "core.bare" is the same as "core.bare = true". * When parsing git configs, support all the documented ways to write true and false, including "yes", "on", "1", etc. * Fix --batch commands (and git-annex info) to accept absolute filenames for unlocked files, which already worked for locked files. * Avoid repeatedly opening keys db when accessing a local git remote and -J is used. * Avoid running a large number of git cat-file child processes when run with a large -J value. * Avoid running with more git check-attr and check-ignore processes than there are CPU cores when run with a large -J value. * get --from, move --from: When used with a local git remote, these used to silently skip files that the location log thought were present on the remote, when the remote actually no longer contained them. Since that behavior could be surprising, now instead display a warning. * external special remotes: remote.name.annex-readonly=true no longer disables running the external special remote program. Instead, it just makes the remote operate in a readonly mode, same as any remote. To disable running the external special remote program, now need to set remote.name.annex-externaltype=readonly. That is done when git-annex enableremote is passed readonly=true. * Stop storing readonly=true in remote.log of external special remotes; it is a local setting only. * sync: When some remotes to sync with are specified, and --fast is too, pick the lowest cost of the specified remotes, do not sync with a faster remote that was not specified. * addurl: When run with --fast on an url that annex.security.allowed-ip-addresses prevents accessing, display a more useful message. * When the required content is set to "groupwanted", use whatever expression has been set in groupwanted as the required content of the repo, similar to how setting required content to "standard" already worked. * Avoid a test suite failure when the environment does not let gpg be tested due to eg, too long a path to the agent socket. * test: Include testremote tests, run on a directory special remote. -- Joey Hess <id@joeyh.name> Fri, 01 May 2020 13:09:24 -0400 git-annex (8.20200330) upstream; urgency=medium * fsck: Fix reversion in 8.20200226 that made it incorrectly warn that hashed keys with an extension should be upgraded. * add --force-small: Fix a bug that, when adding a symbolic link, checked in the content of the file the symlink pointed to. Thanks, Kyle Meyer for the patch. * add --force-small: Fix failure when passed a modified submodule. Thanks, Kyle Meyer for the patch. * When syncing changes back from an adjusted branch to the basis branch, include changes to submodules. Thanks, Kyle Meyer for the patch. * webdav: Made exporttree remotes faster by caching connection to the server. * Fix a minor bug that caused options provided with -c to be passed multiple times to git. -- Joey Hess <id@joeyh.name> Mon, 30 Mar 2020 12:14:50 -0400 git-annex (8.20200309) upstream; urgency=medium * Fix bug that caused unlocked annexed dotfiles to be added to git by the smudge filter when annex.dotfiles was not set. * Upgrade other repos than the current one by running git-annex upgrade inside them, which avoids problems with upgrade code making assumptions that the cwd will be inside the repo being upgraded. In particular, this fixes a problem where upgrading a v7 repo to v8 caused an ugly git error message. * Fix upgrade failure when a file has been deleted from the working tree. * Fix regression 1 month ago that prevented external special remotes from using GETCONFIG to query values like "name". * Improve behavior when a directory git-annex is writing to gets unmounted. Previously it could in some cases re-create the mount point and directory tree, and even write object contents to the wrong disk. * Don't ignore --debug when it is followed by -c. * whereis: If a remote fails to report on urls where a key is located, display a warning, rather than giving up and not displaying any information. * When external special remotes fail but neglect to provide an error message, say what request failed, which is better than displaying an empty error message to the user. * git-annex config: Only allow configs be set that are ones git-annex actually supports reading from repo-global config, to avoid confusion. * Avoid converting .git file in a worktree or submodule to a symlink when the repository is not a git-annex repository. * Linux standalone: Use md5sum to shorten paths in .cache/git-annex/locales * Fix build with ghc 8.8 (MonadFail) Thanks, Peter Simons * stack.yaml: Updated to lts-14.27. -- Joey Hess <id@joeyh.name> Mon, 09 Mar 2020 17:04:08 -0400 git-annex (8.20200226) upstream; urgency=medium * New v8 repository version. * v7 upgrades automatically to v8. The upgrade deletes old sqlite databases, which may cause git-annex to need to do extra work to regenerate the databases or due to not having the information from the old databases available. Two notable cases are interrupted incremental fscks and interrupted exports, both of which will restart from the beginning. * Improved serialization of filenames and keys to the sqlite databases, avoiding encoding problems and speeding up operations on them. * Add some missing indexes to sqlite databases. This will speed up some things involving export and import remotes, and git-annex smudge. Microbenchmarks show around 10-25% speedup of sqlite database operations. * add: When adding a whole directory, any dotfiles found in it will not be skipped, but will be added to git by default. This behavior can be configured with annex.dotfiles. * add: Removed special case for explicitly passing dotfiles, that no longer adds them to the annex, but to git. This behavior can be configured with annex.dotfiles. * add: Removed the --include-dotfiles option. * initremote, enableremote: Set remote.name.skipFetchAll when the remote cannot be fetched from by git, so git fetch --all will not try to use it. * Fix some cases where handling of keys with extensions varied depending on the locale. * annex.maxextensionlength used to be the number of characters, not bytes, when in a utf-8 locale. It's now always the number of bytes. * Extended annex.security.allowed-ip-addresses to let specific ports of an IP address to be used, while denying use of other ports. * init --version: When the version given is one that automatically upgrades to a newer version, use the newer version instead. * Auto upgrades from older repo versions, like v5, now jump right to v8. * Makefile: Support newer versions of cabal that use the new-build system. -- Joey Hess <id@joeyh.name> Wed, 26 Feb 2020 18:49:58 -0400 git-annex (7.20200226) upstream; urgency=high * Fix serious regression in gcrypt and encrypted git-lfs remotes. Since version 7.20200202.7, git-annex incorrectly stored content on those remotes without encrypting it. If your remotes are affected, you will want to make sure to delete any content that git-annex has stored on them that is not encrypted! * initremote: Fix regression in parsing the exporttree= parameter. * info: Fix display of the encryption value. (Some debugging junk had crept in.) * Bugfix to getting content from an export remote with -J, when the export database was not yet populated. * Bugfix: export --tracking (a deprecated option) set annex-annex-tracking-branch, instead of annex-tracking-branch. -- Joey Hess <id@joeyh.name> Wed, 26 Feb 2020 17:18:16 -0400 git-annex (7.20200219) upstream; urgency=medium * Added sync --only-annex, which syncs the git-annex branch and annexed content but leaves managing the other git branches up to you. * Added annex.synconlyannex git config setting, which can also be set with git-annex config to configure sync in all clones of the repo. * fsck --from remote: Fix a concurrency bug that could make it incorrectly detect that content in the remote is corrupt, and remove it, resulting in data loss. * When git-annex is built with a ssh that does not support ssh connection caching, default annex.sshcaching to false, but let the user override it. * Improve warning messages further when ssh connection caching cannot be used, to clearly state why. * Avoid throwing fatal errors when asked to write to a readonly git remote on http. * Fix support for repositories tuned with annex.tune.branchhash1=true, including --all not working and git-annex log not displaying anything for annexed files. -- Joey Hess <id@joeyh.name> Wed, 19 Feb 2020 12:44:43 -0400 git-annex (7.20200204) upstream; urgency=medium * Fix build with persistent-template 2.8.0. * Makefile: Really move the fish completion to the vendor_completions.d directory. -- Joey Hess <id@joeyh.name> Tue, 04 Feb 2020 14:30:55 -0400 git-annex (7.20200202.7) upstream; urgency=medium * add: --force-large/--force-small options make it easier to override annex.largefiles configuration (and potentially safer as it avoids bugs like the smudge bug fixed in the last release). * reinject --known: Fix bug that prevented it from working in a bare repo. * Support being used in a git repository that uses sha256 rather than sha1. * initremote, enableremote: Be stricter about rejecting invalid configuration parameters for remotes, particularly things like foo=true when foo=yes is expected. * initremote, enableremote: Reject unknown configuration parameters provided to these commands. * initremote: Added --whatelse option, to show additional configuration parameters you might want to set. Eg: git annex initremote type=directory encryption=none --whatelse * Added LISTCONFIGS to external special remote protocol. Special remote programs that use GETCONFIG/SETCONFIG are recommended to implement it. * init: Avoid an ugly error message when http remote has no git-annex uuid configured. * Support git remotes that need http basic auth to be accessed, using git credential to get the password. * Display a warning when concurrency is enabled but ssh connection caching is not enabled or won't work due to a crippled filesystem. * Makefile: Move the fish completion to the vendor_completions.d directory. * Fixed a test suite failure when run in the C locale. -- Joey Hess <id@joeyh.name> Sun, 02 Feb 2020 00:00:00 -0400 git-annex (7.20191230) upstream; urgency=medium * Optimised processing of many files, especially by commands like find and whereis that only report on the state of the repository. Commands like get also sped up in cases where they have to check a lot of files but only transfer a few files. Speedups range from 30-100%. * Added build dependency on the filepath-bytestring library. * Fixed an oversight that had always prevented annex.resolvemerge from being honored, when it was configured by git-annex config. * annex.largefiles can be configured by git-annex config, to more easily set a default that will also be used by clones, without needing to shoehorn the expression into the gitattributes file. The git config and gitattributes override that. * annex.addunlocked can be set to an expression with the same format used by annex.largefiles, when you want to default to unlocking some files but not others. * annex.addunlocked can be configured by git-annex config. * git-annex-config --set/--unset: No longer change the local git config setting, except for in the special case of annex.securehashesonly. * Improve file ordering behavior when one parameter is "." and other parameters are other directories. * smudge bugfix: When annex.largefiles=anything, files that were already stored in git, and have not been modified could sometimes be converted to being stored in the annex. Changes in 7.20191024 made this more of a problem. This case is now detected and prevented. -- Joey Hess <id@joeyh.name> Mon, 30 Dec 2019 12:43:30 -0400 git-annex (7.20191218) upstream; urgency=medium * git-lfs: The url provided to initremote/enableremote will now be stored in the git-annex branch, allowing enableremote to be used without an url. initremote --sameas can be used to add additional urls. * git-lfs: When there's a git remote with an url that's known to be used for git-lfs, automatically enable the special remote. * sync, assistant: Pull and push from git-lfs remotes. * Fix bug that made bare repos be treated as non-bare when --git-dir was used. * inprogress: Support --key. * Sped up many git-annex commands that operate on many files, by avoiding reserialization of keys. find is 7% faster; whereis is 3% faster; and git-annex get when all files are already present is 5% faster * Stop displaying rsync progress, and use git-annex's own progress display for local-to-local repo transfers. * benchmark: Changed --databases to take a parameter specifiying the size of the database to benchmark. * benchmark --databases: Display size of the populated database. * benchmark --databases: Improve the "addAssociatedFile (new)" benchmark to really add new values, not overwriting old values. * Windows: Fix handling of changes to time zone. (Used to work but was broken in version 7.20181031.) -- Joey Hess <id@joeyh.name> Wed, 18 Dec 2019 13:53:51 -0400 git-annex (7.20191114) upstream; urgency=medium * Added annex.allowsign option. * Make --json-error-messages capture more errors, particularly url download errors. * Fix a crash (STM deadlock) when -J is used with multiple files that point to the same key. * linuxstandalone: Fix a regression that broke git-remote-https. * OSX git-annex.app: Fix a problem that prevented using the bundled git-remote-https, git-remote-http, and git-shell. -- Joey Hess <id@joeyh.name> Thu, 14 Nov 2019 21:57:59 -0400 git-annex (7.20191106) upstream; urgency=medium * init: Fix bug that lost modifications to unlocked files when init is re-ran in an already initialized repo. * benchmark: Add --databases to benchmark sqlite databases. -- Joey Hess <id@joeyh.name> Wed, 06 Nov 2019 12:14:50 -0400 git-annex (7.20191024) upstream; urgency=medium * Changed git add/git commit -a default behavior back to what it was before v7; they add file contents to git, not to the annex. (However, if a file was annexed before, they will still add it to the annex, to avoid footgun.) * Configuring annex.largefiles overrides that; once git-annex has been told which files are large git add/git commit -a will annex them. * Added annex.gitaddtoannex configuration. Setting it to false prevents git add from adding files to the annex even when annex.largefiles is configured. (Unless the file was annexed before.) * smudge: Made git add smarter about renamed annexed files. It can tell when an annexed file was renamed, and will add it to the annex, and not to git, unless annex.largefiles tells it to do otherwise. * init: Fix a failure when used in a submodule on a crippled filesystem. * sync: Fix crash when there are submodules and an adjusted branch is checked out. * enable-tor: Deal with pkexec changing to root's home directory when running a command. -- Joey Hess <id@joeyh.name> Fri, 25 Oct 2019 13:04:59 -0400 git-annex (7.20191017) upstream; urgency=medium * initremote: Added --sameas option, allows for two special remotes that access the same data store. * Note that due to complications of the sameas feature, any external special remotes that try to send SETSTATE or GETSTATE during INITREMOTE or EXPORTSUPPORTED will now get back an ERROR. That would be a very hackish thing for an external special remote to do, needing some kind of hard-coded key value to be used, so probably nothing will be affected. * forget --drop-dead: Remove several classes of git-annex log files when they become empty, further reducing the size of the git-annex branch. * OSX: Deal with symbolic link problem that caused git to not be included in the git-annex.dmg. * Fix build with persistent-2.10. -- Joey Hess <id@joeyh.name> Thu, 17 Oct 2019 18:19:12 -0400 git-annex (7.20191009) upstream; urgency=medium * Fix bug in handling of annex.largefiles that use largerthan/smallerthan. When adding a modified file, it incorrectly used the file size of the old version of the file, not the current size. * Added --mimetype and --mimeencoding file matching options. * Added --unlocked and --locked file matching options. * Added adjust --lock, to enter an adjusted branch where files are locked. * git-lfs: Added support for http basic auth. * git-lfs: Only do endpoint discovery once when concurrency is enabled. * fsck --incremental/--more: Fix bug that prevented the incremental fsck information from being updated every 5 minutes as it was supposed to be; it was only updated after 1000 files were checked, which may be more files that are possible to fsck in a given fsck time window. Thanks to Peter Simons for help with analysis of this bug. * Test: Use more robust directory removal when built with directory-1.2.7. * Close sqlite databases more robustly. * remotedaemon: Don't list --stop in help since it's not supported. * enable-tor: Run kdesu with -c option. * enable-tor: Use pkexec to run command as root when gksu and kdesu are not available. * When dropping an unlocked file, preserve its mtime, which avoids git status unncessarily running the clean filter on the file. * uninit: Remove several git hooks that git-annex init sets up. * uninit: Remove the smudge and clean filters that git-annex init sets up. * Work around git cat-file --batch's odd stripping of carriage return from the end of the line (some windows infection), avoiding crashing when the repo contains a filename ending in a carriage return. * git-annex-standalone.rpm: Fix the git-annex-shell symlink. -- Joey Hess <id@joeyh.name> Wed, 09 Oct 2019 12:31:31 -0400 git-annex (7.20190912) upstream; urgency=medium * Default to v7 for new repositories. * Automatically upgrade v5 repositories to v7. * Automatically convert direct mode repositories to v7 with adjusted unlocked branches and set annex.thin. * Added annex.autoupgraderepository configuration that can be set to false to prevent any automatic repository upgrades. * Refuse to upgrade direct mode repositories when git is older than 2.22, which fixed a memory leak that could cause an OOM during the upgrade. * Removed support for git versions older than 2.1. * assistant: When creating a new repository, no longer use direct mode, instead use v7 adjusted branches with annex.thin. * init: When run on a crippled filesystem with --version=5, will error out, since version 7 is needed for adjusted unlocked branch. * direct: This command always errors out as direct mode is no longer supported. * indirect: This command has become a deprecated noop. * proxy: This command is deprecated because it was only needed in direct mode. (But it continues to work.) * info: Removed the "repository mode" from its output (including the --json output) since with the removal of direct mode, there is no repository mode. * info: When file matching options are specified when getting info of something other than a directory, they won't have any effect, so error out to avoid confusion. * info: Display trust level when getting info on a uuid, same as a remote. * When upgrading a direct mode repo to v7 with adjusted unlocked branches, fix a bug that prevented annex.thin from taking effect for the files in working tree. * Avoid making a commit when upgrading from direct mode to v7. * init: Catch more exceptions when testing locking. * init: Fix a reversion that broke initialization on systems that need to use pid locking. * A git-annex-standalone rpm is now distributed along with the standalone tarball. * Added standalone/rpm/rpmbuild-from-standalone-tarball script. -- Joey Hess <id@joeyh.name> Fri, 13 Sep 2019 12:53:06 -0400 git-annex (7.20190819) upstream; urgency=medium * New git-lfs special remote, which can be used to store data on any git-lfs server, including github, gitlab, and gogs. * Support fully encrypting all data sent to a git-lfs special remote, using a combination of gcrypt to encrypt the git data, and git-annex's encryption of its data. * Use the same optimisation for --in=here as has always been used for --in=. rather than the slow code path that unncessarily queries the git-annex branch. * Allow setting up a gcrypt special remote with encryption=shared. * Fix querying git for object type when operating on a file containing newlines. * merge: When run with a branch parameter, merges from that branch. This is especially useful when using an adjusted branch, because it applies the same adjustment to the branch before merging it. * init: Install working hook scripts when run on a crippled filesystem and on Windows. If your repository was set up by an old version of git-annex that omitted the hooks, you can simply re-run git-annex init to install them. * init: When the repo is already initialized, and --version requests a different version, error out rather than silently not changing the version. * S3: Fix encoding when generating public urls of S3 objects. * Make git-annex-standalone.deb include the shell completions again. * Makefile: Changed default zsh completion location to zsh default fpath. Systems such as Debian that have overridden the default fpath will need to set ZSH_COMPLETIONS_PATH. * test: Add pass using adjusted unlocked branch. * Fix intermittent failure of the test suite, where its repeated opening and writing to the sqlite database somehow caused inode cache information to occasionally be lost. * Fix several test suite failures on Windows. * Windows installer: Always install to 64 bit program files directory, since it needs 64 bit git now. * Build with silently-1.2.5.1 on Windows; the old one used "NUL" device which is not supported with recent versions of ghc. -- Joey Hess <id@joeyh.name> Mon, 19 Aug 2019 12:23:16 -0400 git-annex (7.20190730) upstream; urgency=medium * Improved probing when CoW copies can be made between files on the same drive. Now supports CoW between BTRFS subvolumes. And, falls back to rsync instead of using cp when CoW won't work, eg copies between repos on the same EXT4 filesystem. * Add BLAKE2BP512 and BLAKE2BP512E backends, using a blake2 variant optimised for 4-way CPUs. * Support running v7 upgrade in a repo where there is no branch checked out, but HEAD is set directly to some other ref. * Windows build no longer ships with a copy of rsync, since that is only used any more to access rsync special remotes or remotes with a very old version of git-annex-shell. * Windows build is now 64 bit, and using it with the 64 bit git for Windows is fully supported. * Windows problems with long filenames should be fixed now, since the Windows build is made with a newer ghc version that works around the problems. * stack.yaml: Build with http-client-0.5.14 to get a bug fix to http header parsing. * Drop support for building with ghc older than 8.4.4, and with older versions of serveral haskell libraries. * Support building with socks-0.6 and persistant-template-2.7. * Corrected some license statements. Thanks, Sean Whitton. -- Joey Hess <id@joeyh.name> Tue, 30 Jul 2019 12:22:25 -0400 git-annex (7.20190708) upstream; urgency=medium * Fix find --json to output json once more. -- Joey Hess <id@joeyh.name> Mon, 08 Jul 2019 08:58:16 -0400 git-annex (7.20190626) upstream; urgency=medium * get, move, copy, sync: When -J or annex.jobs has enabled concurrency, checksum verification uses a separate job pool than is used for downloads, to keep bandwidth saturated. * Other commands also run their cleanup phase using a separate job pool than their perform phase, which may make some of them somewhat faster when running concurrently as well. * When downloading an url and the destination file exists but is empty, avoid using http range to resume, since a range "bytes=0-" is an unusual edge case that it's best to avoid relying on working. This is known to fix a case where importfeed downloaded a partial feed from such a server. * importfeed: When there's a problem parsing the feed, --debug will output the feed content that was downloaded. * init: Fix a reversion in the last release that prevented automatically generating and setting a description for the repository. * add: Display progress meter when hashing files. * add: Support --json-progress option. * The Linux standalone arm build now works again on CPU versions below arm6. Thanks to Emanuele Olivetti, Ilias Tsitsimpis, Bernhard Übelacker, and Adrian Bunk for fixing ghc in Debian (bug #928882). * OSX dmg: Put git-annex's version in the Info.plist file. -- Joey Hess <id@joeyh.name> Wed, 26 Jun 2019 12:29:46 -0400 git-annex (7.20190615) upstream; urgency=medium * Fixed bug that caused git-annex to fail to add a file when another git-annex process cleaned up the temp directory it was using. * Makefile: Added install-completions to install target. * Added the ability to run one job per CPU (core), by setting annex.jobs=cpus, or using option --jobs=cpus or -Jcpus. * Honor preferred content of a special remote when exporting trees to it; unwanted files are filtered out of the tree that is exported. * Importing from a special remote honors its preferred content too; unwanted files are not imported. But, some preferred content expressions can't be checked before files are imported, and trying to import with such an expression will fail. * Don't try to import .git directories from special remotes, because git does not support storing git repositories inside a git repository. * Improve shape of commit tree when importing from unversioned special remotes. * init: When the repository already has a description, don't change it. * describe: When run with no description parameter it used to set the description to "", now it will error out. * Android: Improve installation process when the user's login shell is not bash. * When a remote is configured to be readonly, don't allow changing what's exported to it. * Renamed annex.security.allowed-http-addresses to annex.security.allowed-ip-addresses because it is not really specific to the http protocol, also limiting eg, git-annex's use of ftp. The old name for the config will still work. * Add back support for ftp urls, which was disabled as part of the fix for security hole CVE-2018-10857 (except for configurations which enabled curl and bypassed public IP address restrictions). Now it will work if allowed by annex.security.allowed-ip-addresses. * Avoid a delay at startup when concurrency is enabled and there are rsync or gcrypt special remotes, which was caused by git-annex opening a ssh connection to the remote too early. -- Joey Hess <id@joeyh.name> Sat, 15 Jun 2019 12:38:25 -0400 git-annex (7.20190507) upstream; urgency=medium * Fix reversion in last release that caused wrong tree to be written to remote tracking branch after an export of a subtree. * Improved locking when multiple git-annex processes are writing to the .git/index file -- Joey Hess <id@joeyh.name> Tue, 07 May 2019 13:05:33 -0400 git-annex (7.20190503) upstream; urgency=medium * adb special remote supports being configured with importtree=yes, to allow git-annex import of files from an Android device. This can be combined with exporttree=yes and git-annex export used to send changes back to the Android device. * S3 special remote supports being configured with importtree=yes, to allow git-annex import of files from a S3 bucket. This can be combined with exporttree=yes and git-annex export used to send changes back to the S3 bucket. * S3: When versioning is enabled on a bucket, importing from it will import old versions of files that were written to the bucket as well as the current versions. A git history is synthesized to reflect the way the bucket changed over time. * Fix bug that caused importing from a special remote to repeatedly download unchanged files when multiple files in the remote have the same content. * Made git-annex sync --content much faster when all the remotes it's syncing with are export/import remotes. * sync: When listing contents on an import remote fails, proceed with other syncing instead of aborting. * renameremote: New command, changes the name that is used to enable a special remote. Especially useful when you want to reuse the name of an old remote for something new. * Drop support for building with aws older than 0.14. * info: Show when a remote is configured with importtree. * Added mimeencoding= term to annex.largefiles expressions. This is probably mostly useful to match non-text files with eg "mimeencoding=binary" * git-annex matchexpression: Added --mimeencoding option. -- Joey Hess <id@joeyh.name> Fri, 03 May 2019 12:47:41 -0400 git-annex (7.20190322) upstream; urgency=medium * New feature allows importing from special remotes, using git annex import branch:subdir --from remote * Directory special remote supports being configured with importree=yes, to allow git-annex import of files from the directory. This can be combined with exporttree=yes and git-annex export used to send changes back to the same directory. * Remote tracking branches are updated when importing and exporting to special remotes, in ways analagous to how git fetch and git push do. * export: Deprecated the --tracking option. Instead, users can configure remote.<name>.annex-tracking-branch themselves. * sync --content: When remote.<name>.annex-tracking-branch is configured, import from special remotes. * sync, assistant: --no-push and remote.<name>.annex-push prevent exporting trees to special remotes. * Fix storage of metadata values containing newlines. (Reversion introduced in version 7.20190122.) * Sped up git-annex export in repositories with lots of keys. * S3: Support enabling bucket versioning when built with aws-0.21.1. * stack.yaml: Build with aws-0.21.1 * Fix cleanup of git-annex:export.log after git-annex forget --drop-dead. * Makefile: Added install-home target which installs git-annex into the HOME directory. * addurl --file: Fix a bug that made youtube-dl be used unneccessarily when adding an html url that does not contain any media. * Add -- before %f in the smudge/clean filter configuration, to support filenames starting with dashes. (To update the config of existing repositories, you can re-run git-annex init.) * fsck: Detect situations where annex.thin has caused data loss to the content of locked files. * Removed bundled gpg from the Linux standalone build and OSX dmg, because gpg now always wants to use gpg-agent, and shipping such a daemon in those is not a good idea. * import: Let --force overwrite symlinks, not only regular files. * Android: Fix typo of name of armv7l in installation script. Thanks, 4omecha. * S3: Added protocol= initremote setting, to allow https to be used on a non-standard port. -- Joey Hess <id@joeyh.name> Fri, 22 Mar 2019 13:56:51 -0400 git-annex (7.20190219) upstream; urgency=medium * init: Fix bug when direct mode needs to be enabled on a crippled filesystem, that left the repository in indirect mode. * Fix false positive in export conflict detection, that occurred when the same tree was exported by multiple clones. Previous fix was incomplete. * When key-based retrieval from a S3 remote with exporttree=yes appendonly=yes fails, fall back to trying to retrieve from the exported tree. This allows downloads of files that were exported to such a remote before versioning was enabled on it. * Fix path separator bug on Windows that completely broke git-annex since version 7.20190122. * Improved speed of S3 remote by only loading S3 creds once. * Display progress bar when getting files from export remotes. * Fix race in cleanup of othertmp directory that could result in a failure attempting to access it. * fromkey: Made idempotent. * fromkey: Added --json. * fromkey --batch output changed to support using it with --json. The old output was not parseable for any useful information, so this is not expected to break anything. * Avoid performing repository fixups for submodules and git-worktrees when there's a .noannex file that will prevent git-annex from being used in the repository. * init: Don't let --force be used to override a .noannex file, instead the user can just delete the file. * webdav: Exporting files with '#' or '?' in their name won't work because urls get truncated on those. Fail in a better way in this case, and avoid failing when removing such files from the export, so after the user has renamed the problem files the export will succeed. * On Windows, avoid using rsync for local copies, since rsync is not always available there. * Added NetworkBSD build flag to deal with Network.BSD moving to a new package. -- Joey Hess <id@joeyh.name> Tue, 19 Feb 2019 11:12:45 -0400 git-annex (7.20190129) upstream; urgency=medium * initremote S3: When configured with versioning=yes, either ask the user to enable bucket versioning, or auto-enable it when built with aws-0.22. * enableremote S3: Do not let versioning=yes be set on existing remote, because when git-annex lacks S3 version IDs for files stored in the bucket, deleting them would cause data loss. * S3: Detect when version=yes but an exported file lacks a S3 version ID, and refuse to delete it, to avoid data loss. * S3: Send a Content-Type header when storing objects in S3, so exports to public buckets can be linked to from web pages. (When git-annex is built with MagicMime support.) -- Joey Hess <id@joeyh.name> Tue, 29 Jan 2019 15:09:20 -0400 git-annex (7.20190122) upstream; urgency=medium * sync --content: Fix dropping unwanted content from the local repository. * sync --content: Support dropping local content that has reached an exporttree remote that is not untrusted (currently only S3 remotes with versioning). * init: When --version=5 is passed on a crippled filesystem, use a v5 direct mode repo as requested, rather than upgrading to v7 adjusted unlocked. (Fixes test suite on crippled filesystems.) * Some optimisations, including a 10x faster timestamp parser, a 7x faster key parser, and improved parsing and serialization of git-annex branch data. Many commands will run 5-15% faster. * Stricter parser for keys doesn't allow doubled fields or out of order fields. * The benchmark command, which only had some old benchmarking of the sqlite databases before, now allows benchmarking any other git-annex commands. * Support being built with ghc 8.6.3 (MonadFail). * Removed old code that cleaned up after a bug in git-annex versions 3.20111105-3.20111110. In the unlikely event that a repo was last touched by that ancient git-annex version, the descriptions of remotes would appear missing when used with this version of git-annex. * Improve uuid.log parser to preserve whitespace in repo descriptions. * Improve activity.log parser to not remove unknown values, allowing for future expansion. * addunused, merge, assistant: Avoid creating work tree files in subdirectories in an edge case where the key contains "/". * testremote: Support testing readonly remotes with the --test-readonly option. * Switch to using .git/annex/othertmp for tmp files other than partial downloads, and make stale files left in that directory when git-annex is interrupted be cleaned up promptly by subsequent git-annex processes. * The .git/annex/misctmp directory is no longer used and git-annex will delete anything lingering in there after it's 1 week old. * Estimated time to completion display shortened from eg "1h1m1s" to "1h1m". * Fix doubled progress display when downloading an url when -J is used. * unused: Update suggested git log message to see where data was previously used so it will also work with v7 unlocked pointer files. * importfeed: Better error message when downloading the feed fails. * Make test suite work better when the temp directory is on NFS. * webapp: Remove configurator for box.com repository, since their webdav support is going away at the end of this January. * webapp: Remove configurator for gitlab, which stopped supporting git-annex some time ago. * Android: For armv71 architecture, use the armel build. * Windows: If 64 bit git is installed, use it when installing git-annex. (However, rsync still won't work and this is still not the documented way to install it.) -- Joey Hess <id@joeyh.name> Tue, 22 Jan 2019 12:25:26 -0400 git-annex (7.20181211) upstream; urgency=medium * S3: Improve diagnostics when a remote is configured with exporttree and versioning, but no S3 version id has been recorded for a key. * findref: Support file matching options: --include, --exclude, --want-get, --want-drop, --largerthan, --smallerthan, --accessedwithin * Commands supporting --branch now apply file matching options --include, --exclude, --want-get, --want-drop to filenames from the branch. Previously, combining --branch with those would fail to match anything. * add, import, findref: Support --time-limit. * Add --branch option to git-annex find and mildly deprecate findref in favor of it. * webdav: When initializing, avoid trying to make a directory at the top of the webdav server, which could never accomplish anything and failed on nextcloud servers. (Reversion introduced in version 6.20170925.) * Fix a case where upgrade to v7 caused git to think that unlocked files were modified. * Fix bug upgrading from direct mode to v7: when files in the repository were already committed as v7 unlocked files elsewhere, and the content was present in the direct mode repository, the annexed files got their full content checked into git. * Fix bug that caused v7 unlocked files in a direct mode repository to get locked when committing. -- Joey Hess <id@joeyh.name> Tue, 11 Dec 2018 16:33:10 -0400 git-annex (7.20181205) upstream; urgency=medium * Make bittorrent special remote work w/o btshowmetainfo installed when it was build with torrentparser. Thanks, Robert Schütz * When running youtube-dl to get a filename, pass --no-playlist. * Fix build without concurrent-output. * init: When a crippled filesystem causes an adjusted unlocked branch to be used, set repo version to 7, which it neglected to do before. * init: When on a crippled filesystem, and the git version is too old to use an adjusted unlocked branch, fall back to using direct mode. * info: When used with an exporttree remote, includes an "exportedtree" info, which is the tree last exported to the remote. During an export conflict, multiple values will be listed. * dropunused: When an unused object file has gotten modified, eg due to annex.thin being set, don't silently skip it, but display a warning and let --force drop it. * annex.cachecreds: New config to allow disabling of credentials caching for special remotes. -- Joey Hess <id@joeyh.name> Wed, 05 Dec 2018 14:02:27 -0400 git-annex (7.20181121) upstream; urgency=medium * git-annex-shell: Fix hang when transferring the same objects to two different clients at the same time. (Or when annex.pidlock is used, two different objects.) * Fixed some other potential hangs in the P2P protocol. * Fix bash completion of "git annex" to propertly handle files with spaces and other problem characters. (Completion of "git-annex" already did.) * Fix resume of download of url when the whole file content is already actually downloaded. * When an export conflict prevents accessing a special remote, be clearer about what the problem is and how to resolve it. * export, sync --content: Avoid unnecessarily trying to upload files to an exporttree remote that already contains the files. * smudge: When passed a file located outside the working tree, eg by git diff, avoid erroring out. * drop -J: Avoid processing the same key twice at the same time when multiple annexed files use it. * When a command is operating on multiple files and there's an error with one, try harder to continue to the rest. (As was already done for many types of errors including IO errors.) * Fixed a crash when using -J with ssh password prompts in --quiet/--json mode. Thanks to Yaroslav Halchenko and the DataLad&ReproNim team for helping to track down this bug. * Remove esqueleto dependency to allow upgrading other dependencies to newer versions. Thanks Sean Parsons. * Fix build with persistent-sqlite older than 2.6.3. * Updated stack.yaml to lts-12.19; added stack-lts-9.9.yaml to support old versions of stack. -- Joey Hess <id@joeyh.name> Wed, 21 Nov 2018 14:22:47 -0400 git-annex (7.20181105) upstream; urgency=medium * Fix test suite failure when git-annex test is not run inside a git repository. * Fix a P2P protocol hang. * importfeed: Avoid erroring out when a feed has been repeatedly broken, as that can leave other imported files not checked into git. * Increase minimum QuickCheck version. -- Joey Hess <id@joeyh.name> Mon, 05 Nov 2018 13:31:09 -0400 git-annex (7.20181031) upstream; urgency=medium * Added v7 repository mode. v6 upgrades automatically to v7, but v5 is still the default for now. While v6 was always experimental to some degree, its successor v7 is ready for production use! http://git-annex.branchable.com/tips/unlocked_files/ * Direct mode repositories are deprecated; they have many problems that v7 fixes, so upgrading them now is recommended (but not yet required): git annex upgrade --version=7 * init: When in a crippled filesystem, initialize a v7 repository using an adjusted unlocked branch, instead of a direct mode repository. * At long last there's a way to hide annexed files whose content is missing from the working tree: git-annex adjust --hide-missing See https://git-annex.branchable.com/tips/hiding_missing_files/ * When already in an adjusted branch, running git-annex adjust again will update the branch as needed. This is mostly useful with --hide-missing to hide/unhide files after their content has been dropped or received. * git-annex sync --content supports --hide-missing; it can be used to get the content of hidden files, and it updates the adjusted branch to hide/unhide files as necessary. * smudge: The smudge filter no longer provides git with annexed file content, to avoid a git memory leak, and because that did not honor annex.thin. Now git annex smudge --update has to be run after a checkout to update unlocked files in the working tree with annexed file contents. * v7 init, upgrade: Install git post-checkout and post-merge hooks that run git annex smudge --update. * precommit: Run git annex smudge --update, because the post-merge hook is not run when there is a merge conflict. So the work tree will be updated when a commit is made to resolve the merge conflict. * Note that git has no hooks run after git stash or git cherry-pick, so the user will have to manually run git annex smudge --update after such commands. * Removed the old Android app. * Removed support for building with very old ghc < 8.0.1, and with yesod < 1.4.3, and without concurrent-output, which were only being used for the Android cross build. * Webapp: Fix termux detection. * runshell: Use system locales when built with GIT_ANNEX_PACKAGE_INSTALL set. (For Neurodebian packages.) * Fix database inconsistency that could cause git-annex to get confused about whether a locked file's content was present. * Fix concurrency bug that occurred on the first download from an exporttree remote. * init --version=6 will still work, but the repository is auto-upgraded immediately to v7. * When annex.thin is set, allow hard links to be made between executable work tree files and annex objects. * addurl: Removed undocumented special case in handling of a CHECKURL-MULTI response with only a single file listed. Rather than ignoring the url that was in the response, use it. * webapp: Fixed a crash when adding a git remote. (Reversion introduced in version 6.20180112) * migrate: Fix failure to migrate from URL keys. (Reversion introduced in version 6.20180926) * Cache high-resolution mtimes for improved detection of modified files in v7 (and direct mode). -- Joey Hess <id@joeyh.name> Wed, 31 Oct 2018 09:21:50 -0400 git-annex (6.20181011) upstream; urgency=medium * sync: Warn when a remote's export is not updated to the current tree because export tracking is not configured. * Improve display when git config download from a http remote fails. * Added annex.jobs setting, which is like using the -J option. * Fix reversion in support of annex.web-options. * rmurl: Fix a case where removing the last url left git-annex thinking content was still present in the web special remote. * SETURLPRESENT, SETURIPRESENT, SETURLMISSING, and SETURIMISSING used to update the presence information of the external special remote that called them; this was not documented behavior and is no longer done. * export: Fix false positive in export conflict detection, that occurred when the same tree was exported by multiple clones. * Fix potential crash in exporttree database due to failure to honor uniqueness constraint. * Fix crash when exporttree is set to a bad value. * Linux standalone: Avoid using bundled cp before envionment is fully set up. * Added arm64 Linux standalone build. * Improved termux installation process. -- Joey Hess <id@joeyh.name> Thu, 11 Oct 2018 13:41:10 -0400 git-annex (6.20180926) upstream; urgency=medium [ Joey Hess ] * Fixes a reversion in the last release that broke interoperation with older versions of git-annex-shell. * init: Improve generated post-receive hook, so it won't fail when run on a system whose git-annex is too old to support git-annex post-receive * init: Update the post-receive hook when re-run in an existing repository. * S3: Fix url construction bug when the publicurl has been set to an url that does not end with a slash. * --debug shows urls accessed by git-annex, like it used to do when git-annex used wget and curl. * Fix support for filenames containing newlines when querying git cat-file, though less efficiently than other filenames. This should make git-annex fully support filenames containing newlines as the rest of git's interface is used in newline-safe ways. * Added -z option to git-annex commands that use --batch, useful for supporting filenames containing newlines. * Added annex.maxextensionlength for use cases where extensions longer than 4 characters are needed. * Added remote.name.annex-security-allow-unverified-downloads, a per-remote setting for annex.security.allow-unverified-downloads. * More FreeBSD build fixes. [ Yaroslav Halchenko ] * debian/control + add netbase to Depends: since required for basic tcp interactions (see e.g. https://github.com/nipy/heudiconv/issues/260) -- Joey Hess <id@joeyh.name> Wed, 26 Sep 2018 12:56:49 -0400 git-annex (6.20180913) upstream; urgency=medium * When --batch is used with matching options like --in, --metadata, etc, only operate on the provided files when they match those options. Otherwise, a blank line is output in the batch protocol. Affected commands: find, add, whereis, drop, copy, move, get * Make metadata --batch combined with matching options refuse to run, since it does not seem worth supporting that combination. * v6 add: Take advantage of improved SIGPIPE handler in git 2.5 to speed up the clean filter by not reading the file content from the pipe. This also avoids git buffering the whole file content in memory. * v6: After updating the worktree for an add/drop, update git's index, so git status will not show the files as modified. * v6: When annex.largefiles is not configured for a file, running git add or git commit, or otherwise using git to stage a file will add it to the annex if the file was in the annex before, and to git otherwise. This is to avoid accidental conversion. Note that git-annex add's behavior has not changed. * v6: Update associated files database when git has staged changes to pointer files. * v6: Fix some race conditions. * v6: Fix annex object file permissions when git-annex add is run on a modified unlocked file, and in some related cases. * v6: When a file is unlocked but has not been modified, and the unlocking is only staged, git-annex add did not lock it. Now it will, for consistency with how modified files are handled and with v5. * Fix git command queue to be concurrency safe. * linux standalone: When LOCPATH is already set, use it instead of the bundled locales. It can be set to an empty string to use the system locales too. * Stop using external hash programs, since cryptonite is faster. * Fix build on FreeBSD. * S3: Support buckets with versioning enabled. When a remote is configured with exporttree=yes versioning=yes, git-annex can download past versions of exported files from it. * S3: Multipart uploads are now only supported when git-annex is built with aws-0.16.0 or later, as earlier versions of the library don't support versioning with multipart uploads. * S3: Support AWS_SESSION_TOKEN. * Don't use GIT_PREFIX when GIT_WORK_TREE=. because it seems git does not intend GIT_WORK_TREE to be relative to GIT_PREFIX in that case, despite GIT_WORK_TREE=.. being relative to GIT_PREFIX. * Don't use GIT_PREFIX to fix up a relative GIT_DIR, because git 2.11 sets GIT_PREFIX to a path it's not relative to. and apparently GIT_DIR is never relative to GIT_PREFIX. * git-annex.cabal: Fix build without assistant, and some other refinements. Thanks fftehnik. -- Joey Hess <id@joeyh.name> Thu, 13 Sep 2018 15:50:38 -0400 git-annex (6.20180807) upstream; urgency=medium * S3: Support credential-less download from remotes configured with public=yes exporttree=yes. * Fix reversion in display of http 404 errors. * Added remote.name.annex-speculate-present config that can be used to make cache remotes. * Added --accessedwithin matching option. * Added annex.commitmessage config that can specify a commit message for the git-annex branch instead of the usual "update". * Fix wrong sorting of remotes when using -J, it was sorting by uuid, rather than cost. * addurl: Include filename in --json-progress output. * Fix git-annex branch data loss that could occur after git-annex forget --drop-dead. -- Joey Hess <id@joeyh.name> Tue, 07 Aug 2018 16:22:05 -0400 git-annex (6.20180719) upstream; urgency=medium * Support working trees set up by git-worktree. * Improve support for repositories created with --separate-git-dir. * Support configuring remote.web.annex-cost and remote.bittorrent.annex-cost * addurl: When security configuration prevents downloads with youtube-dl, still check if the url is one that it supports, and fail downloading it, instead of downloading the raw web page. * Send User-Agent and any configured annex.http-headers when downloading with http, fixes reversion introduced when switching to http-client. * Fix reversion introduced in version 6.20180316 that caused git-annex to stop processing files when unable to contact a ssh remote. * v6: Work around git bug that runs smudge/clean filters at the top of the repository while passing them a relative GIT_WORK_TREE that may point outside of the repository, by using GIT_PREFIX to get back to the subdirectory where a relative GIT_WORK_TREE is valid. * p2p --pair: Fix interception of the magic-wormhole pairing code, which since 0.8.2 it has sent to stderr rather than stdout. * info: Display uuid and description when a repository is identified by uuid, and for "here". * unused --from: Allow specifiying a repository by uuid or description. * linux standalone: Generate locale files in ~/.cache/git-annex/locales/ so they're available even when the standalone tarball is installed in a directory owned by root. Note that this prevents using the standalone bundle in environments where HOME is not writable. * Include uname command in standalone builds since git-annex uses it. * git-annex.cabal: Fix network version. -- Joey Hess <id@joeyh.name> Thu, 19 Jul 2018 13:53:45 -0400 git-annex (6.20180626) upstream; urgency=high Security fix release for CVE-2018-10857 and CVE-2018-10859 https://git-annex.branchable.com/security/CVE-2018-10857_and_CVE-2018-10859/ * Refuse to download content, that cannot be verified with a hash, from encrypted special remotes (for CVE-2018-10859), and from all external special remotes and glacier (for CVE-2018-10857). In particular, URL and WORM keys stored on such remotes won't be downloaded. If this affects your files, you can run `git-annex migrate` on the affected files, to convert them to use a hash. * Added annex.security.allow-unverified-downloads, which can override the above. * Added annex.security.allowed-url-schemes setting, which defaults to only allowing http, https, and ftp URLs. Note especially that file:/ is no longer enabled by default. * Removed annex.web-download-command, since its interface does not allow supporting annex.security.allowed-url-schemes across redirects. If you used this setting, you may want to instead use annex.web-options to pass options to curl. * git-annex will refuse to download content from http servers on localhost, or any private IP addresses, to prevent accidental exposure of internal data. This can be overridden with the annex.security.allowed-http-addresses setting. * Local http proxies will not be used unless allowed by the annex.security.allowed-http-addresses setting. * Since the interfaces to curl and youtube-dl do not have a way to prevent them from accessing localhost or private IP addresses, they default to not being used for url downloads. Only when annex.security.allowed-http-addresses=all will curl and youtube-dl be used. Non-security fix changes: * Fix build with ghc 8.4+, which broke due to the Semigroup Monoid change. * version: Show operating system and repository version list when run outside a git repo too. * Fix annex-checkuuid implementation, so that remotes configured that way can be used. * Fix problems accessing repositories over http when annex.tune.* is configured. * External special remotes can now add info to `git annex info $remote`, by replying to the GETINFO message. * adb: Android serial numbers are not all 16 characters long, so accept other lengths. * Display error messages that come from git-annex-shell when the p2p protocol is used, so that diskreserve messages, IO errors, etc from the remote side are visible again. * When content has been lost from an export remote and git-annex fsck --from remote has noticed it's gone, re-running git-annex export or git-annex sync --content will re-upload it. -- Joey Hess <id@joeyh.name> Fri, 22 Jun 2018 10:36:22 -0400 git-annex (6.20180529) upstream; urgency=medium * Prevent haskell http-client from decompressing gzip files, so downloads of such files works the same as it used to with wget and curl. * Workaround for bug in an old version of cryptonite that broke https downloads, by using curl for downloads when git-annex is built with it. * view, vadd: Fix crash when a git submodule has a name starting with a dot. * Don't allow entering a view with staged or unstaged changes. * move: --force was accidentially enabling two unrelated behaviors since 6.20180427. The older behavior, which has never been well documented and seems almost entirely useless, has been removed. * copy: --force no longer does anything. * migrate: Fix bug in migration between eg SHA256 and SHA256E, that caused the extension to be included in SHA256 keys, and omitted from SHA256E keys. (Bug introduced in version 6.20170214) * migrate: Check for above bug when migrating from SHA256 to SHA256 (and same for SHA1 to SHA1 etc), and remove the extension that should not be in the SHA256 key. * fsck: Detect and warn when keys need an upgrade, either to fix up from the above migrate bug, or to add missing size information (a long ago transition), or because of a few other past key related bugs. * git-annex-shell: GIT_ANNEX_SHELL_APPENDONLY makes it allow writes, but not deletion of annexed content. Note that securing pushes to the git repository is left up to the user. * setpresentkey: Added --batch support. -- Joey Hess <id@joeyh.name> Tue, 29 May 2018 13:05:26 -0400 git-annex (6.20180509) upstream; urgency=medium * The old git-annex Android app is now deprecated in favor of running git-annex in termux. * runshell: Use proot when running on Android, to work around Android 8's ill-advised seccomp filtering of system calls, including ones crucial for reliable thread locking. (This will only work with termux's version of proot.) * Fix bug in last release that crashes when using --all or running git-annex in a bare repository. May have also affected git-annex unused and git-annex info. * Fix bug in last release that prevented the webapp opening on non-Linux systems. * Support building with hinotify-0.3.10. * Display error message when http download fails. * Avoid forward retry when 0 bytes were received. -- Joey Hess <id@joeyh.name> Wed, 09 May 2018 16:20:26 -0400 git-annex (6.20180427) upstream; urgency=medium * move: Now takes numcopies configuration, and required content configuration into account, and refuses to reduce the current number of copies of a file, or remove content that a repository requires. --force can override these checks. Note that it's still allowed to move the content of a file from one repository to another when numcopies is not satisfied, as long as the move does not result in there being fewer copies. * Fix mangling of --json output of utf-8 characters when not running in a utf-8 locale. * Fix build with yesod 1.6. * Clean up some build warnings with newer versions of ghc and haskell libraries. * runshell: Unset LD_PRELOAD since preloaded libraries from the host system may not get along with the bundled linker. * runshell: Added some tweaks to make git-annex work in termux on Android. The regular arm standalone tarball now works in termux. * Webapp: Support being run inside termux on Android, and offer to set up a repository on the sdcard. * Assistant: Integrate with Termux:Boot, so when it's installed, the assistant is autostarted on boot. * Assistant: Fix installation of menus, icons, etc when run from within runshell. * import: Avoid buffering all filenames to be imported in memory. * Improve memory use and speed of --all and git-annex info remote, by not buffering list of all keys. -- Joey Hess <id@joeyh.name> Fri, 27 Apr 2018 12:36:20 -0400 git-annex (6.20180409) upstream; urgency=medium * Added adb special remote which allows exporting files to Android devices. * For url downloads, git-annex now defaults to using a http library, rather than wget or curl. But, if annex.web-options is set, it will use curl. To use the .netrc file, run: git config annex.web-options --netrc * git-annex no longer uses wget (and wget is no longer shipped with git-annex builds). * Enable HTTP connection reuse across multiple files for improved speed. * Fix calculation of estimated completion for progress meter. * OSX app: Work around libz/libPng/ImageIO.framework version skew by not bundling libz, assuming OSX includes a suitable libz.1.dylib. * Added annex.retry, annex.retry-delay, and per-remote versions to configure transfer retries. * Also do forward retrying in cases where no exception is thrown, but the transfer failed. * When adding a new version of a file, and annex.genmetadata is enabled, don't copy the data metadata from the old version of the file, instead use the mtime of the file. * Avoid running annex.http-headers-command more than once. * info: Added "combined size of repositories containing these files" stat when run on a directory. * info: Changed sorting of numcopies stats table, so it's ordered by the variance from the desired number of copies. * Fix resuming a download when using curl. -- Joey Hess <id@joeyh.name> Mon, 09 Apr 2018 13:03:15 -0400 git-annex (6.20180316) upstream; urgency=medium * New protocol for communicating with git-annex-shell increases speed of operations involving ssh remotes. When not transferring large files, git-annex is between 200% and 400% faster using the new protocol, and it's just as fast as before when transferring large files. (When the remote has an old git-annex-shell, git-annex falls back to the old slower code. This fallback is planned to be removed after 5 years or so.) * Note that, due to not using rsync to transfer files over ssh any longer, permissions and other file metadata of annexed files will no longer be preserved when copying them to and from ssh remotes. Other remotes never supported preserving that information, so this is not considered a regression. * Fix data loss bug in content locking over tor, when the remote repository is in direct mode, it neglected to check that the content was actually present when locking it. This could cause git annex drop to remove the only copy of a file when it thought the tor remote had a copy. * Fix data loss bug when the local repository uses direct mode, and a locally modified file is dropped from a remote repsitory. The bug caused the modified file to be counted as a copy of the original file. (This is not a severe bug because in such a situation, dropping from the remote and then modifying the file is allowed and has the same end result.) * Some downloads will be verified, even when annex.verify=false. This is done in some edge cases where there's a likelyhood than an object was downloaded incorrectly. * Support exporttree=yes for rsync special remotes. * Added backends for the BLAKE2 family of hashes, when built with a new enough version of cryptonite. * Improve SHA*E extension extraction code to not treat parts of the filename that contain punctuation or other non-alphanumeric characters as extensions. Before, such characters were filtered out. * Better ssh connection warmup when using -J for concurrency. Avoids ugly messages when forced ssh command is not git-annex-shell. * Fix race condition in ssh warmup that caused git-annex to get stuck and never process some files when run with high levels of concurrency. * Fix reversion introduced in 6.20171214 that caused concurrent transfers to incorrectly fail with "transfer already in progress". * Note that Remote/Git.hs now contains AGPL licensed code, thus the license of git-annex as a whole is AGPL. This was already the case when git-annex was built with the webapp enabled. * Include amount of data transferred in progress display. * Dial back optimisation when building on arm, which prevents ghc and llc from running out of memory when optimising some files. (Unfortunately this fix is incomplete due to a ghc bug.) -- Joey Hess <id@joeyh.name> Fri, 16 Mar 2018 12:10:40 -0400 git-annex (6.20180227) upstream; urgency=medium * inprogress: Avoid showing failures for files not in progress. * Added INFO to external special remote protocol. * Added EXTENSIONS to external special remote protocol. * datalad < 0.9.1 had a problem in its special remote protocol handling which is broken by EXTENSIONS. Make the debian git-annex package conflict with the problem version of datalad. * fsck: Warn when required content is not present in the repository that requires it. * Add gpg-agent to Build-Depends. Needed to run the test suite. * --json: When there are multiple lines of notes about a file, make the note field multiline, rather than the old behavior of only including the last line. * git-annex.cabal: Once more try to not build the assistant on the hurd, hopefully hackage finally recognises that OS. * Split Test.hs and avoid optimising it much, to need less memory to compile. * Fix behavior of --json-progress followed by --json, the latter option used to disable the former. * Added --json-error-messages option, which makes messages that would normally be output to standard error be included in the json output. * Remove temporary code added in 6.20160619 to prime the mergedrefs log. * importfeed: Fix a failure when downloading with youtube-dl and the destination subdirectory does not exist yet. * Added annex.merge-annex-branches config setting which can be used to disable automatic merge of git-annex branches. * tips/automatically_adding_metadata/pre-commit-annex: Fix to not silently skip filenames containing non-ascii characters. * sync: Fix bug that prevented pulling changes into direct mode repositories that were committed to remotes using git commit rather than git-annex sync. * Makefile: Remove chrpath workaround for bug in cabal, which is no longer needed. -- Joey Hess <id@joeyh.name> Tue, 27 Feb 2018 12:04:52 -0400 git-annex (6.20180112) upstream; urgency=medium * Added inprogress command for accessing files as they are being downloaded. * Fix bug introduced in version 6.20171018 that caused some commands to print out "ok" twice after processing a file. * addurl: When the file youtube-dl will download is already an annexed file, don't download it again and fail to overwrite it, instead just do nothing, like it used to when quvi was used. * addurl: Fix encoding of filename queried from youtube-dl when in --fast mode. * Fix several places where files in .git/annex/ were written with modes that did not take the core.sharedRepository config into account. * Improve startup time for commands that do not operate on remotes, and for tab completion, by not unnessessarily statting paths to remotes, which used to cause eg, spin-up of removable drives. * Added remote.<name>.annex-checkuuid config, which can be set to false to disable the default checking of the uuid of remotes that point to directories. This can be useful to avoid unnecessary drive spin-ups and automounting. * git-annex.cabal: Add back custom-setup stanza, so cabal new-build works. * git-annex.cabal: Removed the testsuite build flag; test suite is always included. -- Joey Hess <id@joeyh.name> Fri, 12 Jan 2018 15:45:48 -0400 git-annex (6.20171214) upstream; urgency=medium * Use youtube-dl rather than quvi to download media from web pages, since quvi is not being actively developed and youtube-dl supports many more sites. * addurl --relaxed got slower, since youtube-dl has to hit the network to check for embedded media. If you relied on --relaxed not hitting the network for speed reasons, using --relaxed --raw will get the old level of speed, but can't be used for urls with embedded videos. * importfeed now downloads things linked to by feeds, even when they are not media files. * Removed no longer needed dependency on yesod-default. * Allow exporttree remotes to be marked as dead. * initremote, enableremote: Really support gpg subkeys suffixed with an exclamation mark, which forces gpg to use a specific subkey. (Previous try had a bug.) * lookupkey: Support being given an absolute filename to a file within the current git repository. * A top-level .noannex file will prevent git-annex init from being used in a repository. This is useful for repositories that have a policy reason not to use git-annex. The content of the file will be displayed to the user who tries to run git-annex init. -- Joey Hess <id@joeyh.name> Thu, 14 Dec 2017 11:50:48 -0400 git-annex (6.20171124) unstable; urgency=medium * Display progress meter when uploading a key without size information, getting the size by statting the content file. * Fix build with dns-3.0. -- Joey Hess <id@joeyh.name> Fri, 24 Nov 2017 10:49:36 -0400 git-annex (6.20171109) unstable; urgency=medium * Fix export of subdir of a branch. * Fix exporting of non-annexed files to external special remotes. * unlock, lock: Support --json. * When there are multiple urls for a file, still treat it as being present in the web when some urls don't work, as long as at least one url does work. * Makefile improvement for sudo make install. Thanks, Eric Siegerman * Makefile improvement for BUILDER=stack, use stack to run ghc. * testremote: Test exporttree. * Fix directory special remote's cleanup of empty export directories. -- Joey Hess <id@joeyh.name> Thu, 09 Nov 2017 12:21:49 -0400 git-annex (6.20171026) unstable; urgency=medium * Windows: Fix reversion that caused the path used to link to annexed content to include the drive letter and full path, rather than being relative. (`git annex fix` will fix up after this problem). * Windows build fixed, and changed to use stack for more reliable build environment. * Windows: Remove wget from bundle; it needs libraries that are not included, and git for windows includes curl which git-annex will use instead. * Add day to metadata when annex.genmetadata is enabled. Thanks, Sean T Parsons * stack.yaml: Added nix packages section. Thanks, Sean T Parsons -- Joey Hess <id@joeyh.name> Thu, 26 Oct 2017 13:56:18 -0400 git-annex (6.20171018) unstable; urgency=medium * add: Replace work tree file atomically on systems supporting hard links. Avoids a window where interrupting an add could result in the file being moved into the annex, with no symlink yet created. * webdav: Avoid unncessisarily creating the collection at the top of the repository when storing files there, since that collection is created by initremote. (This seems to work around some brokenness of the box.com webdav server, which caused uploads to be very slow or sometimes fail.) * webdav: Make --debug show all webdav operations. * get -J/move -J/copy -J/mirror -J/sync -J: Avoid "transfer already in progress" errors when two files use the same key. * Konqueror desktop file location changed to one used by plasma 5. Thanks, Félix Sipma. * Avoid repeated checking that files passed on the command line exist. * Fix build with aws-0.17. * stack.yaml: Update to lts-9.9. -- Joey Hess <id@joeyh.name> Wed, 18 Oct 2017 15:40:06 -0400 git-annex (6.20171003) unstable; urgency=medium * webdav: Improve error message for failed request to include the request method and path. * metadata: Added --remove-all. * Warn when metadata is inherited from a previous version of a file, to avoid the user being surprised in cases where that behavior is not desired or expected. * sync: Added --cleanup, which removes local and remote synced/ branches. * external: When the external special remote program crashed, a newline could be output, which messed up the expected output for --batch mode. * external: Avoid checking EXPORTSUPPORTED for special remotes that are not configured to use exports. * test: Fix reversion that made it only run inside a git repository. * copy, move: Behave same with --fast when sending to remotes located on a local disk as when sending to other remotes. * Fix process and file descriptor leak that was exposed when git-annex was built with ghc 8.2.1. Broke git-annex test on OSX due to running out of FDs, and may have also leaked in other situations. * info: Improve cleanup of stale transfer info files. -- Joey Hess <id@joeyh.name> Tue, 03 Oct 2017 13:18:15 -0400 git-annex (6.20170925) unstable; urgency=medium * git-annex export: New command, can create and efficiently update exports of trees to special remotes. * Use git-annex initremote with exporttree=yes to set up a special remote for use by git-annex export. * Implemented export to directory, S3, and webdav special remotes. * External special remote protocol extended to support export. Developers of external special remotes should consider if export makes sense for them and add support. * sync, assistant: Update tracking exports. * Support building with feed-1.0, while still supporting older versions. * init: Display an additional message when it detects a filesystem that allows writing to files whose write bit is not set. * S3: Allow removing files from IA. * webdav: Checking if a non-existent file is present on Box.com triggered a bug in its webdav support that generates an infinite series of redirects. Deal with such problems by assuming such behavior means the file is not present. * webdav: Fix lack of url-escaping of filenames. Mostly impacted exports of filenames containing eg spaces. * webdav: Changed path used on webdav server for temporary files. -- Joey Hess <id@joeyh.name> Mon, 25 Sep 2017 11:13:58 -0400 git-annex (6.20170818) unstable; urgency=high * Security fix: Disallow hostname starting with a dash, which would get passed to ssh and be treated an option. This could be used by an attacker who provides a crafted repository url to cause the victim to execute arbitrary code via -oProxyCommand. CVE-2017-12976 (The same class of security hole recently affected git itself.) * git-annex.cabal: Deal with breaking changes in Cabal 2.0. * Fix build with QuickCheck 2.10. * fsck: Support --json. * move, copy: Support --batch. * Added GIT_ANNEX_VECTOR_CLOCK environment variable, which can be used to override the default timestamps used in log files in the git-annex branch. This is a dangerous environment variable; use with caution. * Fix a git-annex test failure when run on NFS due to NFS lock files preventing directory removal. * test: Avoid most situations involving failure to delete test directories, by forking a worker process and only deleting the test directory once it exits. * Disable http-client's default 30 second response timeout when HEADing an url to check if it exists. Some web servers take quite a long time to answer a HEAD request. * Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. * Prevent spaces from being embedded in the name of new WORM keys, as that handing spaces in keys would complicate things like the external special remote protocol. * migrate: WORM keys containing spaces will be migrated to not contain spaces anymore. * External special remotes will refuse to operate on keys with spaces in their names. That has never worked correctly due to the design of the external special remote protocol. Display an error message suggesting migration. * Fix incorrect external special remote documentation, which said that the filename parameter to the TRANSFER command could not contain spaces. It can in fact contain spaces. Special remotes implementors that relied on that may need to fix bugs in their special remotes. * Fix the external special remotes git-annex-remote-ipfs, git-annex-remote-torrent and the example.sh template to correctly support filenames with spaces. * Windows: Win32 package has subsumed Win32-extras; update dependency. -- Joey Hess <id@joeyh.name> Fri, 18 Aug 2017 11:19:06 -0400 git-annex (6.20170520) unstable; urgency=medium * move --to=here moves from all reachable remotes to the local repository. * initremote, enableremote: Support gpg subkeys suffixed with an exclamation mark, which forces gpg to use a specific subkey. * Improve progress display when watching file size, in cases where a transfer does not resume. * Fix transfer log file locking problem when running concurrent transfers. * Avoid concurrent git-config setting problem when running concurrent threads. * metadata: When setting metadata of a file that did not exist, no error message was displayed, unlike getting metadata and most other git-annex commands. Fixed this oversight. * Added annex.resolvemerge configuration, which can be set to false to disable the usual automatic merge conflict resolution done by git-annex sync and the assistant. * sync: Added --no-resolvemerge option. * Avoid error about git-annex-shell not being found when syncing with -J with a git remote where git-annex-shell is not installed. * Fix bug that prevented transfer locks from working when run on SMB or other filesystem that does not support fcntl locks and hard links. * assistant: Merge changes from refs/remotes/foo/master into master. Previously, only sync branches were merged. This makes regular git push into a repository watched by the assistant auto-merge. * Makefile: Install completions for the fish and zsh shells when git-annex is built with optparse-applicative-0.14. * assistant: Don't trust OSX FSEvents's eventFlagItemModified to be called when the last writer of a file closes it; apparently that sometimes does not happen, which prevented files from being quickly added. -- Joey Hess <id@joeyh.name> Mon, 12 Jun 2017 13:37:16 -0400 git-annex (6.20170519) unstable; urgency=medium * Ssh password prompting improved when using -J for concurrency. When ssh connection caching is enabled (and when GIT_ANNEX_USE_GIT_SSH is not set), only one ssh password prompt will be made per host, and only one ssh password prompt will be made at a time. * When built with concurrent-output 1.9, ssh password prompts will no longer interfere with the -J display. * Removed dependency on MissingH, instead depending on the split library. * Progress is displayed for transfers of files of unknown size. * Work around bug in git 2.13.0 involving GIT_COMMON_DIR that broke merging changes into adjusted branches. -- Joey Hess <id@joeyh.name> Fri, 19 May 2017 10:37:57 -0400 git-annex (6.20170510) unstable; urgency=medium * When a http remote does not expose an annex.uuid config, only warn about it once, not every time git-annex is run. * multicast: New command, uses uftp to multicast annexed files, for eg a classroom setting. * Added remote.<name>.annex-push and remote.<name>.annex-pull which can be useful to make remotes that don't get fully synced with local changes. * Disable git-annex's support for GIT_SSH and GIT_SSH_COMMAND, unless GIT_ANNEX_USE_GIT_SSH=1 is also set in the environment. This is necessary because as feared, the extra -n parameter that git-annex passes breaks uses of these environment variables that expect exactly the parameters that git passes. * enableremote: When enabling a non-special remote, param=value parameters can't be used, so error out if any are provided. * enableremote: Fix re-enabling of special remotes that have a git url, so that eg, encryption key changes take effect. They were silently ignored, a reversion introduced in 6.20160527. * gcrypt: Support re-enabling to change eg, encryption parameters. This was never supported before. * git annex add -u now supported, analagous to git add -u * version: Added "dependency versions" line. * Keys marked as dead are now skipped by --all. * annex.backend is the new name for what was annex.backends, and takes a single key-value backend, rather than the unnecessary and confusing list. The old option still works if set. -- Joey Hess <id@joeyh.name> Wed, 10 May 2017 15:05:22 -0400 git-annex (6.20170321) unstable; urgency=medium * Bugfix: Passing a command a filename that does not exist sometimes did not display an error, when a path to a directory was also passed. * status: Propigate nonzero exit code from git status. * Linux standalone builds put the bundled ssh last in PATH, so any system ssh will be preferred over it. * assistant: Add 1/200th second delay between checking each file in the full transfer scan, to avoid using too much CPU. * get -J: Improve distribution of jobs amoung remotes when there are more jobs than remotes. * fsck -q: When a file has bad content, include the name of the file in the warning message. * Windows: Improve handling of shebang in external special remote program, searching for the program in the PATH. * Drop support for building with old versions of dns, http-conduit, directory, feed, and http-types. * Windows: Fix bug in shell script shebang lookup code that caused a "delayed read on closed handle" error. * git-annex-shell: Fix bug when used with a recently cloned repository, where "merging" messages were included in the output of configlist (and perhaps other commands) and caused a "Failed to get annex.uuid configuration" error. * Support GIT_SSH and GIT_SSH_COMMAND, which are handled close the same as they are by git. However, unlike git, git-annex sometimes needs to pass the -n parameter when using these. * sync --content-of=path (-C path) added for when you want to sync only some files' contents, not the whole working tree. -- Joey Hess <id@joeyh.name> Tue, 21 Mar 2017 11:27:38 -0400 git-annex (6.20170301.1) unstable; urgency=medium * Fix reversion in yesterday's release that made SHA1E and MD5E backends not work. -- Joey Hess <id@joeyh.name> Wed, 01 Mar 2017 12:46:03 -0400 git-annex (6.20170301) unstable; urgency=medium * No changes from 6.20170228; a new version number was needed due to a problem with Hackage. -- Joey Hess <id@joeyh.name> Wed, 01 Mar 2017 12:06:02 -0400 git-annex (6.20170228) unstable; urgency=medium * Cryptographically secure hashes can be forced to be used in a repository, by setting annex.securehashesonly. This does not prevent the git repository from containing links to insecure hashes, but it does prevent the content of such files from being added to .git/annex/objects by any method. * Tighten key parser to prevent SHA1 collision attacks generating two keys that have the same SHA1. (Only done for keys that contain a hash). This ensures that signed git commits of annexed files will remain secure, as long as git-annex is using a secure hashing backend. * fsck: Warn about any files whose content is present, that don't use secure hashes, when annex.securehashesonly is set. * init: When annex.securehashesonly has been set with git-annex config, copy that value to the annex.securehashesonly git config. * Added --securehash option to match files using a secure hash function, and corresponding securehash preferred content expression. * sync, merge: Fail when the current branch has no commits yet, instead of not merging in anything from remotes and appearing to succeed. * Run ssh with -n whenever input is not being piped into it, to avoid it consuming stdin that it shouldn't. This fixes git-annex-checkpresentkey --batch remote, which didn't output results for all keys passed into it. Other git-annex commands that communicate with a remote over ssh may also have been consuming stdin that they shouldn't have, which could have impacted using them in eg, shell scripts. * sync: Improve integration with receive.denyCurrentBranch=updateInstead, displaying error messages from the remote then it fails to update its checked out branch. * Added post-recieve hook, which makes updateInstead work with direct mode and adjusted branches. * init: Set up the post-receive hook. * sync: When syncing with a local repository located on a crippled filesystem, run the post-receive hook there, since it wouldn't get run otherwise. This makes pushing to repos on FAT-formatted removable drives update them when receive.denyCurrentBranch=updateInstead. * config group groupwanted numcopies schedule wanted required: Avoid displaying extraneous messages about repository auto-init, git-annex branch merging, etc, when being used to get information. * adjust: Fix behavior when used in a repository that contains submodules. * Run wget with -nv instead of -q, so it will display HTTP errors. * Run curl with -S, so HTTP errors are displayed, even when it's otherwise silent. * When downloading in --json or --quiet mode, use curl in preference to wget, since curl is able to display only errors to stderr, unlike wget. * status: Pass --ignore-submodules=when option on to git status. * config --set: As well as setting value in git-annex branch, set local gitconfig. This is needed especially for annex.securehashesonly, which is read only from local gitconfig and not the git-annex branch. * Removed support for building with the old cryptohash library. Building with that library made git-annex not support SHA3; it's time for that to always be supported in case SHA2 dominoes. * git-annex.cabal: Make crypto-api a dependency even when built w/o webapp and test suite. -- Joey Hess <id@joeyh.name> Tue, 28 Feb 2017 14:39:47 -0400 git-annex (6.20170214) unstable; urgency=medium * Increase default cost for p2p remotes from 200 to 1000. This makes git-annex prefer transferring data from special remotes when possible. * Remove -j short option for --json-progress; that option was already taken for --json. * vicfg: Include the numcopies configuation. * config: New command for storing configuration in the git-annex branch. * annex.autocommit can be configured via git-annex config, to control the default behavior in all clones of a repository. * New annex.synccontent config setting, which can be set to true to make git annex sync default to --content. This may become the default at some point in the future. As well as being configuable by git config, it can be configured by git-annex config to control the default behavior in all clones of a repository. * stack.yaml: Update to lts-7.18. * Some optimisations to string splitting code. * unused: When large files are checked right into git, avoid buffering their contents in memory. * unused: Improved memory use significantly when there are a lot of differences between branches. * Wormhole pairing will start to provide an appid to wormhole on 2021-12-31. An appid can't be provided now because Debian stable is going to ship a older version of git-annex that does not provide an appid. Assumption is that by 2021-12-31, this version of git-annex will be shipped in a Debian stable release. If that turns out to not be the case, this change will need to be cherry-picked into the git-annex in Debian stable, or its wormhole pairing will break. * Fix build with aws 0.16. Thanks, aristidb. * assistant: Make --autostart --foreground wait for the children it starts. Before, the --foreground was ignored when autostarting. * initremote: When a uuid= parameter is passed, use the specified UUID for the new special remote, instead of generating a UUID. This can be useful in some situations, eg when the same data can be accessed via two different special remote backends. * import: Changed how --deduplicate, --skip-duplicates, and --clean-duplicates determine if a file is a duplicate. Before, only content known to be present somewhere was considered a duplicate. Now, any content that has been annexed before will be considered a duplicate, even if all annexed copies of the data have been lost. Note that --clean-duplicates and --deduplicate still check numcopies, so won't delete duplicate files unless there's an annexed copy. * import: --deduplicate and --skip-duplicates were implemented inneficiently; they unncessarily hashed each file twice. They have been improved to only hash once. * import: Added --reinject-duplicates. * Added git template directory to Linux standalone tarball and OSX app bundle. * Improve pid locking code to work on filesystems that don't support hard links. * S3: Fix check of uuid file stored in bucket, which was not working. * Work around sqlite's incorrect handling of umask when creating databases. -- Joey Hess <id@joeyh.name> Tue, 14 Feb 2017 14:22:00 -0400 git-annex (6.20170101) unstable; urgency=medium * XMPP support has been removed from the assistant in this release. If your repositories used XMPP to keep in sync, that will no longer work, and you should enable some other remote to keep them in sync. A ssh server is one way, or use the new Tor pairing feature. * p2p --pair makes it easy to pair repositories, over Tor, using Magic Wormhole codes to find the other repository. See http://git-annex.branchable.com/tips/peer_to_peer_network_with_tor/ * webapp: The "Share with a friend" and "Share with your other devices" pages have been changed to pair repositories using Tor and Magic Wormhole. * metadata --batch: Fix bug when conflicting metadata changes were made in the same batch run. * Pass annex.web-options to wget and curl after other options, so that eg --no-show-progress can be set by the user to disable the default --show-progress. * Revert ServerAliveInterval change in 6.20161111, which caused problems with too many old versions of ssh and unusual ssh configurations. It should have not been needed anyway since ssh is supposted to have TCPKeepAlive enabled by default. * Make all --batch input, as well as fromkey and registerurl stdin be processed without requiring it to be in the current encoding. * p2p: --link no longer takes a remote name, instead the --name option can be used. * Linux standalone: Improve generation of locale definition files, supporting locales such as en_GB.UTF-8. * rekey --force: Incorrectly marked the new key's content as being present in the local repo even when it was not. * enable-tor: Put tor sockets in /var/lib/tor-annex/, rather than in /etc/tor/hidden_service/. * enable-tor: No longer needs to be run as root. * enable-tor: When run as a regular user, also tests a connection back to the hidden service over tor. * Support all common locations of the torrc file. * Always use filesystem encoding for all file and handle reads and writes. * Fix build with directory-1.3. * Debian: Suggest tor and magic-wormhole. * Debian: Build webapp on armel. -- Joey Hess <id@joeyh.name> Sat, 31 Dec 2016 15:11:04 -0400 git-annex (6.20161210) unstable; urgency=medium * Linux standalone: Updated ghc to fix its "unable to decommit memory" bug, which may have resulted in data loss when these builds were used with Linux kernels older than 4.5. * enable-tor: New command, enables tor hidden service for P2P syncing. * p2p: New command, allows linking repositories using a P2P network. * remotedaemon: Serve tor hidden service. * Added git-remote-tor-annex, which allows git pull and push to the tor hidden service. * remotedaemon: Fork to background by default. Added --foreground switch to enable old behavior. * addurl: Fix bug in checking annex.largefiles expressions using largerthan, mimetype, and smallerthan; the first two always failed to match, and the latter always matched. * Relicense 5 source files that are not part of the webapp from AGPL to GPL. * map: Run xdot if it's available in PATH. On OSX, the dot command does not support graphical display, while xdot does. * Debian: xdot is a better interactive viewer than dot, so Suggest xdot, rather than graphviz. * rmurl: Multiple pairs of files and urls can be provided on the command line. * rmurl: Added --batch mode. * fromkey: Accept multiple pairs of files and keys. Thanks, Daniel Brooks. * rekey: Added --batch mode. * add: Stage modified non-large files when running in indirect mode. (This was already done in v6 mode and direct mode.) * git-annex-shell, remotedaemon, git remote: Fix some memory DOS attacks. * Fix build with http-client 0.5. Thanks, Alper Nebi Yasak. -- Joey Hess <id@joeyh.name> Sat, 10 Dec 2016 11:56:25 -0400 git-annex (6.20161118) unstable; urgency=medium * git-annex.cabal: Loosen bounds on persistent to allow 2.5, which on Debian has been patched to work with esqueleto. This may break cabal's resolver on non-Debian systems; if so, either use stack to build, or run cabal with --constraint='persistent ==2.2.4.1' Hopefully this mess with esqueleto will be resolved soon. * sync: Pass --allow-unrelated-histories to git merge when used with git git 2.9.0 or newer. This makes merging a remote into a freshly created direct mode repository work the same as it works in indirect mode. * Avoid backtraces on expected failures when built with ghc 8; only use backtraces for unexpected errors. * fsck --all --from was checking the existence and content of files in the local repository, rather than on the special remote. Oops. * Linux arm standalone: Build with a 32kb page size, which is needed on several ARM NAS devices, including Drobo 5N, and WD NAS. -- Joey Hess <id@joeyh.name> Fri, 18 Nov 2016 11:43:14 -0400 git-annex (6.20161111) unstable; urgency=medium * Restarting a crashing git process could result in filename encoding issues when not in a unicode locale, as the restarted processes's handles were not read in raw mode. * Make .git/annex/ssh.config file work with versions of ssh older than 7.3, which don't support Include. When used with an older version of ssh, any ServerAliveInterval in ~/.ssh/config will be overridden by .git/annex/ssh.config. * S3: Support the special case endpoint needed for the cn-north-1 region. * Webapp: Don't list the Frankfurt S3 region, as this (and some other new regions) need V4 authorization which the aws library does not yet use. * reinject --known: Avoid second, unnecessary checksum of file. * OSX: Remove RPATHs from git-annex binary, which are not needed, slow down startup, and break the OSX Sierra linker. * webapp: Explicitly avoid checking for auth in static subsite requests. Yesod didn't used to do auth checks for that, but this may have changed. * Linux standalone: Avoid using hard links in the tarball so it can be untarred on eg, afs which does not support them. -- Joey Hess <id@joeyh.name> Fri, 11 Nov 2016 14:46:39 -0400 git-annex (6.20161031) unstable; urgency=medium * Assistant, repair: Fix ignoring of git fsck errors due to duplicate file entries in tree objects. * Linux standalone: Fix location of locale files in the bundle. * Fix reversion in 6.20161012 that prevented adding files with a space in their name. -- Joey Hess <id@joeyh.name> Mon, 31 Oct 2016 18:55:59 -0400 git-annex (6.20161027) unstable; urgency=medium * lock, smudge: Fix edge cases where data loss could occur in v6 mode when the keys database was not populated. * upgrade: Handle upgrade to v6 when the repository already contains v6 unlocked files whose content is already present. * Improve style of offline html build of website. * importfeed: Drop URL parameters from file extension. Thanks, James MacMahon. * Assistant, repair: Improved filtering out of git fsck lines about duplicate file entries in tree objects. * test: Deal with gpg-agent behavior change that broke the test suite. * Improve ssh socket cleanup code to skip over the cruft that NFS sometimes puts in a directory when a file is being deleted. * If a transfer fails for some reason, but some data managed to be sent, the transfer will be retried. (The assistant already did this.) * Run ssh with ServerAliveInterval 60, so that stalled transfers will be noticed within about 3 minutes. (Any setting in your ~/.ssh/config or /etc/ssh/ssh_config overrides this.) -- Joey Hess <id@joeyh.name> Thu, 27 Oct 2016 15:21:58 -0400 git-annex (6.20161012) unstable; urgency=medium * Optimisations to time it takes git-annex to walk working tree and find files to work on. Sped up by around 18%. * Optimisations to git-annex branch query and setting, avoiding repeated copies of the environment. Speeds up commands like "git-annex find --in remote" by over 50%. * Optimised git-annex branch log file timestamp parsing. * Add "total-size" field to --json-progress output. * Make --json-progress output be shown even when the size of a object is not known. * Multiple external special remote processes for the same remote will be started as needed when using -J. This should not beak any existing external special remotes, because running multiple git-annex commands at the same time could already start multiple processes for the same external special remotes. * Linux standalone: Include locale files in the bundle, and generate locale definition files for the locales in use when starting runshell. (Currently only done for utf-8 locales.) * Avoid using a lot of memory when large objects are present in the git repository and have to be checked to see if they are a pointed to an annexed file. Cases where such memory use could occur included, but were not limited to: - git commit -a of a large unlocked file (in v5 mode) - git-annex adjust when a large file was checked into git directly * When auto-upgrading a v3 remote, avoid upgrading to version 6, instead keep it at version 5. * Support using v3 repositories without upgrading them to v5. * sync: Fix bug in adjusted branch merging that could cause recently added files to be lost when updating the adjusted branch. -- Joey Hess <id@joeyh.name> Wed, 12 Oct 2016 09:37:41 -0400 git-annex (6.20160923) unstable; urgency=medium * Rate limit console progress display updates to 10 per second. Was updating as frequently as changes were reported, up to hundreds of times per second, which used unnecessary bandwidth when running git-annex over ssh etc. * Make --json and --quiet work when used with -J. Previously, -J override the other options. * addurl, get: Added --json-progress option, which adds progress objects to the json output. * Remove key:null from git-annex add --json output. * copy, move, mirror: Support --json and --json-progress. * Improve gpg secret key list parser to deal with changes in gpg 2.1.15. Fixes key name display in webapp. * info: Support being passed a treeish, and show info about the annexed files in it similar to how a directory is handled. * sync: Previously, when run in a branch with a slash in its name, such as "foo/bar", the sync branch was "synced/bar". That conflicted with the sync branch used for branch "bar", so has been changed to "synced/foo/bar". * Note that if you're using an old version of git-annex to sync with a branch with a slash in its name, it won't see some changes synced by this version, and this version won't see some changes synced by the older version. This is not a problem if there's a central bare repository, but may impact other configurations until git-annex is upgraded to this version. * adjust: Previously, when adjusting a branch with a slash in its name, such as "foo/bar", the adjusted branch was "adjusted/bar(unlocked)". That conflicted with the adjusted branch used for branch "bar", so has been changed to "adjusted/foo/bar(unlocked)" * Also, running sync in an adjusted branch did not correctly sync changes back to the parent branch when it had a slash in its name. This bug has been fixed. * addurl, importfeed: Improve behavior when file being added is gitignored. -- Joey Hess <id@joeyh.name> Fri, 23 Sep 2016 09:43:26 -0400 git-annex (6.20160907) unstable; urgency=medium * Windows: Handle shebang in external special remote program. * Fix formatting of git-annex-smudge man page, and improve mdwn2man. Thanks, Jim Paris. * examimekey: Allow being run in a git repo that is not initialized by git-annex yet. * Android: Fix disabling use of cp --reflink=auto, curl, sha224, and sha384. * Make --json and --quiet suppress automatic init messages, and any other messages that might be output before a command starts. Fixes a reversion introduced in version 5.20150727. * Assistant, repair: Filter out git fsck lines about duplicate file entries in tree objects. * get -J, sync --content -J: Download different files from different remotes when the remotes have the same costs. -- Joey Hess <id@joeyh.name> Wed, 07 Sep 2016 11:12:11 -0400 git-annex (6.20160808) unstable; urgency=medium * metadata --json output format has changed, adding a inner json object named "fields" which contains only the fields and their values. This should be easier to parse than the old format, which mixed up metadata fields with other keys in the json object. Any consumers of the old format will need to be updated. * Added metadata --batch option, which allows getting, setting, deleting, and modifying metadata for multiple files/keys. * Added --branch option to copy, drop, fsck, get, metadata, mirror, move, and whereis commands. This option makes git-annex operate on files that are included in a specified branch (or other treeish). * git-annex.cabal: Temporarily limit to http-conduit <2.2.0 since aws 0.14.0 is not compatible with the newer version. * git-annex.cabal: Temporarily limit to persistent <2.5 since esqueleto 2.4.3 is not compatible with the newer version. * Removed dependency on json library; all JSON is now handled by aeson. * When built with uuid-1.3.12, generate more random UUIDs than before. (However, this did not impact git-annex much, so a hard depedency has not been added on uuid-1.3.12.) * info: When run on a file now includes an indication of whether the content is present locally. * get, move, copy, mirror: Added --failed switch which retries failed copies/moves. * Re-enable accumulating transfer failure log files for command-line actions (disabled in 5.20150522), and remove the log files after successful transfers. -- Joey Hess <id@joeyh.name> Mon, 08 Aug 2016 11:42:17 -0400 git-annex (6.20160619) unstable; urgency=medium * get, drop: Add --batch and --json options. * testremote: Fix crash when testing a freshly made external special remote. * Remove unnecessary rpaths in the git-annex binary, but only when it's built using make, not cabal. This speeds up git-annex startup time by around 50%. * Speed up startup time by caching the refs that have been merged into the git-annex branch. This can speed up git-annex commands by as much as a second, depending on the number of remotes. * fsck: Fix a reversion in direct mode fsck of a file that is present when the location log thinks it is not. Reversion introduced in version 5.20151208. * uninit: Fix crash due to trying to write to deleted keys db. Reversion introduced by v6 mode support, affects v5 too. * Fix a similar crash when the webapp is used to delete a repository. * Support checking presence of content at a http url that redirects to a ftp url. * log: Added --all option. * New url for git-remote-gcrypt, now maintained by spwhitton. * webapp: Don't allow deleting a remote that has syncing disabled, as such a deletion will never finish. Thanks, Farhan Kathawala. * webapp: Escape unusual characters in ssh hostnames when generating mangled hostnames. This allows IPv6 addresses to be used on filesystems not supporting : in filenames. * Avoid any access to keys database in v5 mode repositories, which are not supposed to use that database. * Remove the EKG build flag, since Gentoo for some reason decided to enable this flag, depsite it not being intended for production use and so disabled by default. -- Joey Hess <id@joeyh.name> Tue, 19 Jul 2016 14:17:54 -0400 git-annex (6.20160613) unstable; urgency=medium * Improve SHA*E extension extraction code. * Windows: Avoid terminating git-annex branch lines with \r\n when union merging and performing transitions. * Remove Makefile from cabal tarball; man page building is now handled by a small haskell program. * sync --content: Fix bug that caused transfers of files to be made to a git remote that does not have a UUID. This particularly impacted clones from gcrypt repositories. * Pass -S to git commit-tree when commit.gpgsign is set and when making a non-automatic commit, in order to preserve current behavior when used with git 2.9, which has stopped doing this itself. * remotedaemon: Fixed support for notifications of changes to gcrypt remotes, which was never tested and didn't quite work before. * list: Do not include dead repositories. * move --to: Better behavior when system is completely out of disk space; drop content from disk before writing location log. * Avoid a crash if getpwuid does not work, when querying the user's full name. * Automatically enable v6 mode when initializing in a clone from a repo that has an adjusted branch checked out. * v6: Fix initialization of a bare clone of a repo that has an adjusted branch checked out. * v6: Fix bad automatic merge conflict resolution between an annexed file and a directory with the same name when in an adjusted branch. * v6: Fix bad merge in an adjusted branch that resulted in an empty tree. * v6: Fix bug in initialization of clone from a repo with an adjusted branch that had not been synced back to master. (This bug caused broken tree objects to get built by a later git annex sync.) * v6: Make lock and unlock work on files whose content is not present. * v6: Fix update of associated files db when unlocking a file. * v6: Make git clean filter preserve the backend that was used for a file. -- Joey Hess <id@joeyh.name> Mon, 13 Jun 2016 14:57:38 -0400 git-annex (6.20160527) unstable; urgency=medium * Split lines in the git-annex branch on \r as well as \n, to deal with \r\n terminated lines written by some versions of git-annex on Windows. This fixes strange displays in some cases. * assistant: Fix bug that caused v6 pointer files to be annexed by the assistant. * assistant: Fix race in v6 mode that caused downloaded file content to sometimes not replace pointer files. * add: Adding a v6 pointer file used to annex it; now the pointer file is added to git as-is. (git add of a pointer file already did the right thing) * enableremote: Can now be used to explicitly enable git-annex to use git remotes. Using the command this way prevents other git-annex commands from probing new git remotes to auto-enable them. * enableremote: Remove annex-ignore configuration from a remote. * Change git annex info remote encryption description to use wording closer to what's used in initremote. * Pass the various gnupg-options configs to gpg in several cases where they were not before. Most notably, gnupg-decrypt-options is now passed when decrypting an encrypted cipher. * adjust: Add --fix adjustment, which is useful when the git directory is in a nonstandard place. * adjust: If the adjusted branch already exists, avoid overwriting it, since it might contain changes that have not yet been propigated to the original branch. * Work around git weirdness in handling of relative path to GIT_INDEX_FILE when in a subdirectory of the repository. This affected git annex view. * Fix crash when entering/changing view in a subdirectory of a repo that has a dotfile in its root. * webapp: Avoid confusing display of dead remotes. * Support building with ghc 8.0.1. * Updated cabal file explicitly lists source files. The tarball on hackage will include only the files needed for cabal install; it is NOT the full git-annex source tree. * debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead. -- Joey Hess <id@joeyh.name> Fri, 27 May 2016 11:48:36 -0400 git-annex (6.20160511) unstable; urgency=medium * Fix bug that sometimes prevented git-annex smudge --clean from consuming all its input, which resulted in git add bypassing git-annex. * Fix build with directory-1.2.6.2. * Improve behavior when a just added http remote is not available during uuid probe. Do not mark it as annex-ignore, so it will be tried again later. * Android: Icon refresh. Thanks, freewheelinfranks. * Added DIRHASH-LOWER to external special remote protocol. * git-annex.cabal: Add Setup-Depends. * stack.yaml: Enable explicit-setup-deps. * Windows: Fix several bugs in propigation of changes from the adjusted branch back to the master branch. * Windows: Fix an over-long temp directory name. * map: Hide dead repositories that are not connected to the graph. * map: Changed colors; red is used for untrusted repositories and grey for dead. * version: Display OS version and architecture too. * Propigate GIT_DIR and GIT_WORK_TREE environment to external special remotes. * Added annex.gnupg-decrypt-options and remote.<name>.annex-gnupg-decrypt-options, which are passed to gpg when it's decrypting data. * fsck: When a key is not previously known in the location log, record something so that reinject --known will work. * In the unusual configuration where annex.crippledfilesystem=true but core.symlinks=true, store object contents in mixed case hash directories so that symlinks will point to them. * Added new encryption=sharedpubkey mode for special remotes. This is useful for makking a special remote that anyone with a clone of the repo and your public keys can upload files to, but only you can decrypt the files stored in it. -- Joey Hess <id@joeyh.name> Wed, 11 May 2016 12:41:42 -0400 git-annex (6.20160419) unstable; urgency=medium * Fix bug that prevented resuming of uploads to encrypted special remotes that used chunking. * That bug could also expose the names of keys to such remotes, so it is a minor security issue. * Fix duplicate progress meter display when downloading from a git remote over http with -J. * reinject: When src file's content cannot be verified, leave it alone, instead of deleting it. * reinject: Added new mode which can reinject known files into the annex. For example: git-annex reinject --known /mnt/backup/* * calckey: New plumbing command, calculates the key that would be used to refer to a file. * Fix bug that prevented annex.sshcaching=false configuration from taking effect when on a crippled filesystem. Thanks, divergentdave. * git 2.9.0 is going to prevent git merge from merging in unrelated branches. Since the webapp's pairing etc features often combine together repositories with unrelated histories, work around this behavior change when the assistant merges, by passing --allow-unrelated-histories. Note though that this is not done for git annex sync's merges, so it will follow git's default or configured behavior. * When git-annex is used with a git version older than 2.2.0, disable support for adjusted branches, since GIT_COMMON_DIR is needed to update them and was first added in that version of git. * Avoid setting LOCPATH in linux standalone builds that are built with a ghc that has been fixed to not hang when it cannot find locale files. * Isolate test suite from global git config settings. -- Joey Hess <id@joeyh.name> Thu, 28 Apr 2016 09:31:14 -0400 git-annex (6.20160418) unstable; urgency=medium * smudge: Print a warning when annex.thin is set, as git's smudge interface does not allow honoring that configuration. * webapp: When $HOME is a git repository, and has been initialized for use by git-annex, opening the webapp went ahead and ran the assistant there, annexing all files. Since this is almost certianly not desirable, especially when the user is just opening the webapp from a dekstop menu which happens to run it in $HOME, the webapp will now not treat such a $HOME git repository as a git-annex repository. * webapp: Update url to add gitlab.com ssh key. * Fix bug in v6 mode that prevented treating unlocked executable files as annexed. If you have such files, run git annex init --version=6 to update the cache after upgrading to this version of git-annex. * Preserve execute bits of unlocked files in v6 mode. * fsck: Warn when core.sharedRepository is set and an annex object file's write bit is not set and cannot be set due to the file being owned by a different user. * Fix hang when dropping content needs to lock the content on a ssh remote, which occurred when the remote has git-annex version 5.20151019 or newer. (The bug was in the client side; the remote git-annex-shell does not need to be upgraded.) -- Joey Hess <id@joeyh.name> Mon, 18 Apr 2016 18:33:52 -0400 git-annex (6.20160412) unstable; urgency=medium * adjust --unlock: Enters an adjusted branch in which all annexed files are unlocked. The v6 equivilant of direct mode, but much cleaner! * Upgrading a direct mode repository to v6 has changed to enter an adjusted unlocked branch. This makes the direct mode to v6 upgrade able to be performed in one clone of a repository without affecting other clones, which can continue using v5 and direct mode. * init --version=6: Automatically enter the adjusted unlocked branch when filesystem doesn't support symlinks. * ddar remote: fix ssh calls Thanks, Robie Basak * log: Display time with time zone. * log --raw-date: Use to display seconds from unix epoch. * v6: Close pointer file handles more quickly, to avoid problems on Windows. * sync: Show output of git commit. * annex.thin and annex.hardlink are now supported on Windows. * unannex --fast now makes hard links on Windows. * Fix bug in annex.largefiles mimetype= matching when git-annex is run in a subdirectory of the repository. * Fix build with ghc v7.11. Thanks, Gabor Greif. -- Joey Hess <id@joeyh.name> Tue, 12 Apr 2016 14:53:22 -0400 git-annex (6.20160318) unstable; urgency=medium * metadata: Added -r to remove all current values of a field. * Fix data loss that can occur when annex.pidlock is set in a repository. * Fix bug preventing moving files to/from a repository with annex.pidlock set. * Fix shared lock file FD leak. * Fix metadata hook behavior when multiple files are added at once. Thanks, Klaus Ethgen. * Added dependencies on haskell mountpoints and disk-free-space libraries, removing FFI code from git-annex. * dropkey: Add --batch and --json. * Fix OSX dmg to include libraries needed by bundled gpg, lost in last release. * Always try to thaw content, even when annex.crippledfilesystem is set. * Correct git-annex info to include unlocked files in v6 repository. * Sped up git-annex add in direct mode and v6 by using git hash-object --stdin-paths. * Sped up git-annex merge by using git hash-object --stdin-paths. -- Joey Hess <id@joeyh.name> Fri, 18 Mar 2016 11:30:36 -0400 git-annex (6.20160229) unstable; urgency=medium * Update perlmagick build dependency. Closes: #789225 * Fix memory leak in last release, which affected commands like git-annex status when a large non-annexed file is present in the work tree. * fsck: When the only copy of a file is in a dead repository, mention the repository. * info: Mention when run in a dead repository. * Linux and OSX standalone builds put the bundled gpg last in PATH, so any system gpg will be preferred over it. * Avoid crashing when built with MagicMime support, but when the magic database cannot be loaded. * Include magic database in the linux and OSX standalone builds. * Fix memory leak when hashing files, which triggered during fsck when an external hash program was not used. (This leak was introduced in version 6.20160114.) * Support --metadata field<number, --metadata field>number etc to match ranges of numeric values. * Similarly, support preferred content expressions like metadata=field<number and metadata=field>number * The pre-commit-annex hook script that automatically extracts metadata has been updated to also use exiftool. Thanks, Klaus Ethgen. -- Joey Hess <id@joeyh.name> Mon, 29 Feb 2016 12:41:49 -0400 git-annex (6.20160217) unstable; urgency=medium * Support getting files from read-only repositories. * checkpresentkey: Allow to be run without an explicit remote. * checkpresentkey: Added --batch. * Work around problem with concurrent-output when in a non-unicode locale by avoiding use of it in such a locale. Instead -J will behave as if it was built without concurrent-output support in this situation. * Fix storing of filenames of v6 unlocked files when the filename is not representable in the current locale. * fsck: Detect and fix missing associated file mappings in v6 repositories. * fsck: Populate unlocked files in v6 repositories whose content is present in annex/objects but didn't reach the work tree. * When initializing a v6 repo on a crippled filesystem, don't force it into direct mode. * Windows: Fix v6 unlocked files to actually work. * add, addurl, import, importfeed: When in a v6 repository on a crippled filesystem, add files unlocked. * annex.addunlocked: New configuration setting, makes files always be added unlocked. (v6 only) * Improve format of v6 unlocked pointer files to support keys containing slashes. -- Joey Hess <id@joeyh.name> Wed, 17 Feb 2016 14:48:51 -0400 git-annex (6.20160211) unstable; urgency=medium * annex.addsmallfiles: New option controlling what is done when adding files not matching annex.largefiles. * Fix reversion in lookupkey, contentlocation, and examinekey which caused them to sometimes output side messages. * webapp: Fix deletion of current repository directory. * Added "nothing" to preferred content expression syntax. * annex.largefiles can be configured in .gitattributes too; this is particulary useful for v6 repositories, since the .gitattributes configuration will apply in all clones of the repository. * Limit annex.largefiles parsing to the subset of preferred content expressions that make sense in its context. So, not "standard" or "lackingcopies", etc. * annex.largefiles: Add support for mimetype=text/* etc, when git-annex is linked with libmagic. * matchexpression: Added --largefiles option to parse an annex.largefiles expression. * Brought back the dbus and xmpp build flags, so build from source can be done without C libraries that may be hard to install. * init: Fix bugs in submodule .git symlink fixup, that occurred when initializing in a subdirectory of a submodule and a submodule of a submodule. * WebDAV: Set depth 1 in PROPFIND request, for better compatibility with some servers. Thanks, wzhd. * WebDAV: Remove a bogus trailing slash from the end of the url to the temporary store location for a key. Thanks, wzhd. * S3: Allow configuring with requeststyle=path to use path-style bucket access instead of the default DNS-style access. -- Joey Hess <id@joeyh.name> Thu, 11 Feb 2016 11:42:19 -0400 git-annex (6.20160126) unstable; urgency=medium * Fix nasty reversion in the last release that broke sync --content's handling of many preferred content expressions. * whereis --json: Urls are now listed inside the remote that claims them, rather than all together at the end. * info, add, whereis, find: Support --batch mode. * Force output to be line-buffered, even when it's not connected to the terminal. This is particuarly important for commands with --batch output, which was not always being flushed at an appropriate time. * add, import: Support --json output. * addurl --json: Include field for added key (unless the file was added directly to git due to annex.largefiles configuration.) (Also done by add --json and import --json) * registerurl: Check if a remote claims the url, same as addurl does. * Bug fix: Git config settings passed to git-annex -c did not always take effect. * assistant: Use udisks2 dbus events to detect when disks are mounted, instead of relying on gnome/kde stuff that is not stable. * Fix build with QuickCheck 2.8.2 * matchexpression: New plumbing command to check if a preferred content expression matches some data. * Removed the webapp-secure build flag, rolling it into the webapp build flag. * Removed the quvi, tahoe, feed, and tfds build flags, adding aeson feed and regex-tdfa to the core dependencies. * Roll the dns build flag into the assistant build flag. * Debian: Avoid building debug package, since gdb is not often useful to debug haskell programs. -- Joey Hess <id@joeyh.name> Tue, 26 Jan 2016 14:57:42 -0400 git-annex (6.20160114) unstable; urgency=medium "hexapodia as the key insight" * Added v6 repository mode, but v5 is still the default for now. * unlock, lock: In v6 mode, unlocking a file changes it from a symlink to a pointer file, and this change can be committed to the git repository. For details, see http://git-annex.branchable.com/tips/unlocked_files/ * The upgrade to version 6 is not done fully automatically yet, because upgrading a direct mode repository to version 6 will prevent old versions of git-annex from working in other clones of that repository. For details, see http://git-annex.branchable.com/upgrades/ * init: --version parameter added to control which supported repository version to use. * init, upgrade: Configure .git/info/attributes to use git-annex as a smudge filter. In v6 repository mode, this makes git add add files to the annex in unlocked mode, unless overridden by annex.largefiles configuration. * assistant: In v6 mode, adds files in unlocked mode, so they can continue to be modified. * Added annex.thin setting, which makes unlocked files in v6 repositories be hard linked to their content, instead of a copy. This saves disk space but means any modification of an unlocked file will lose the local (and possibly only) copy of the old version. * Enable annex.thin by default on upgrade from direct mode to v6, since direct mode made the same tradeoff. * fix: Adjusts unlocked files as configured by annex.thin. * persistent-sqlite is now a hard build dependency, since v6 repository mode needs it. * status: On crippled filesystems, was displaying M for all annexed files that were present. Probably caused by a change to what git status displays in this situation. Fixed by treating files git thinks are modified the same as typechanged files. * addurl: Added --batch and --with-files options. * addurl: Support --json, particularly useful in --batch mode. * addurl: Refuse to overwrite any existing, non-annexed file. * Debian: Adjust build dependencies for webapp, DAV. Now available on mips, mipsel, but temporarily removed armel since build is failing there. * info: Fix "backend usage" numbers, which were counting present keys twice. * info --json: Improve json for "backend usage", using a nested object with fields for each backend instead of the previous weird nested lists. This may break existing parsers of this json output, if there were any. * whereis --json: Make url list be included in machine-parseable form. * test: Added --keep-failures option. * unused: Bug fix when a new file was added to the annex, and then removed (but not git rmed). git still has the add staged in this case, so the content should not be unused and was wrongly treated as such. * migrate: Copy over metadata to new key. * rekey: No longer copies over urls from the old to the new key. It makes sense for migrate to do that, but not for this low-level (and little used) plumbing command to. * view: Fix crash in non-unicode capable locale when entering a view of metadata containing a slash or backslash. * When annex.http-headers is used to set the User-Agent header, avoid sending User-Agent: git-annex * Windows: Fix rsync cross-drive hack to work with msys2 rsync. Thanks, Pieter Kitslaar. -- Joey Hess <id@joeyh.name> Thu, 14 Jan 2016 10:14:19 -0400 git-annex (5.20151218) unstable; urgency=medium * Add S3 features to git-annex version output. * webdav: When testing the WebDAV server, send a file with content. The empty file it was sending tickled bugs in some php WebDAV server. * fsck: Failed to honor annex.diskreserve when checking a remote. * Debian: Build depend on concurrent-output. * Fix insecure temporary permissions when git-annex repair is used in in a corrupted git repository. * Fix potential denial of service attack when creating temp dirs. -- Joey Hess <id@joeyh.name> Fri, 18 Dec 2015 12:09:33 -0400 git-annex (5.20151208) unstable; urgency=medium * Build with -j1 again to get reproducible build. * Display progress meter in -J mode when copying from a local git repo, to a local git repo, and from a remote git repo. * Display progress meter in -J mode when downloading from the web. * map: Improve display of git remotes with non-ssh urls, including http and gcrypt. * When core.sharedRepository is set, annex object files are not made mode 444, since that prevents a user other than the file owner from locking them. Instead, a mode such as 664 is used in this case. * tahoe: Include tahoe capabilities in whereis display. * import: Changed to honor annex.largefiles settings. * addurl, importfeed: Changed to honor annex.largefiles settings, when the content of the url is downloaded. (Not when using --fast or --relaxed.) * webapp: Fix bugs that could result in a relative path such as "." being written to ~/.config/git-annex/autostart, and ignore any such relative paths in the file. This was a reversion caused by the relative path changes in 5.20150113. * dropunused: Make more robust when trying to drop an object that has already been dropped. * Fix reversion in handling of long filenames, particularly when using addurl/importfeed, which was introduced in the previous release. -- Joey Hess <id@joeyh.name> Tue, 08 Dec 2015 11:14:03 -0400 git-annex (5.20151116) unstable; urgency=medium * Use concurrent-output library when configured with -fConcurrentOutput. This allows nicely displayed messages when using the -J flag. * Additional commands now support the -J flag: fsck, drop, add, addurl, import * import: Avoid very ugly error messages when the directory files are imported to is not a directort, but perhaps an annexed file. * Concurrent progress bars are now displayed when using -J with a command that moves file contents around. * Fix race that could result in an annexed file's symlink not being created, when eg, running concurrent git-annex adds. * add: Fix error recovery rollback to not move the injested file content out of the annex back to the file, because other files may point to that same content. Instead, copy the injected file content out to recover. * quvi may output utf-8 encoded data when the conifigured locale doesn't support that; avoid crashing on such invalid encoding. * runshell: Avoid failing when $HOME/.ssh does not exist and cannot be created. * Make the git-annex-standalone.deb prevent runshell from installing wrappers into $HOME/.ssh * Make git-annex-standalone.deb include the git-annex html documentation, desktop file, and base completion file, same as the regular git-annex.deb. * fsck: When fscking a dead repo, avoid incorrect "fixing location log" message, and display a warning about it being dead, since it's unusual to have access to a dead repo. * assistant: Pass ssh-options through 3 more git pull/push calls that were missed before. * Added annex.pidlock and annex.pidlocktimeout configuration to support filesystems where POSIX fcntl locks cannot be used. * init: Automatically enable annex.pidlock when necessary. -- Joey Hess <id@joeyh.name> Mon, 16 Nov 2015 14:17:40 -0400 git-annex (5.20151102.1) unstable; urgency=medium * Avoid installing desktop file and program file if cabal install git-annex is run as root, since that is not a systemwide install, but to /root, and so generating a systemwide desktop file is not right. * When cabal install is run with the desktop file location not writable, display a warning, but continue successfully. -- Joey Hess <id@joeyh.name> Tue, 03 Nov 2015 12:08:38 -0400 git-annex (5.20151102) unstable; urgency=medium * Use statvfs on OSX. * Symlink timestamp preservation code uses functions from unix-2.7.0 when available, which should be more portable. * enableremote: List uuids and descriptions of remotes that can be enabled, and accept either the uuid or the description in leu if the name. * Catch up with current git behavior when both repo and repo.git exist; it seems it now prefers repo in this case, although historically it may have preferred repo.git. * Fix failure to build with aws-0.13.0. * When built with aws-0.13.0, the S3 special remote can be used to create google nearline buckets, by setting storageclass=NEARLINE. -- Joey Hess <id@joeyh.name> Mon, 02 Nov 2015 12:41:20 -0400 git-annex (5.20151019) unstable; urgency=medium * Fix a longstanding, but unlikely to occur bug, where dropping a file from a remote could race with other drops of the same file, and result in all copies of its content being lost. * git-annex-shell: Added lockcontent command, to prevent dropping of a key's content. This is necessary due to the above bugfix. * In some cases, the above bugfix changes what git-annex allows you to drop: - When a file is present in several special remotes, but not in any accessible git repositories, dropping it from one of the special remotes will now fail. Instead, the file has to be moved from one of the special remotes to the git repository, and can then safely be dropped from the git repository. - If a git remote has too old a version of git-annex-shell installed, git-annex won't trust it to hold onto a copy of a file when dropping that file from the local git repository. * Changed drop ordering when using git annex sync --content or the assistant, to drop from remotes first and from the local repo last. This works better with the behavior changes to drop in many cases. * Do verification of checksums of annex objects downloaded from remotes. * When annex objects are received into git repositories from other git repos, their checksums are verified then too. * To get the old, faster, behavior of not verifying checksums, set annex.verify=false, or remote.<name>.annex-verify=false. * setkey, rekey: These commands also now verify that the provided file matches the expected checksum of the key, unless annex.verify=false. * reinject: Already verified content; this can now be disabled by setting annex.verify=false. * sync, merge, assistant: When git merge failed for a reason other than a conflicted merge, such as a crippled filesystem not allowing particular characters in filenames, git-annex would make a merge commit that could omit such files or otherwise be bad. Fixed by aborting the whole merge process when git merge fails for any reason other than a merge conflict. * Allow building with S3 disabled again. * Ported disk free space checking code to work on Solaris. * Windows webapp: Fix support for entering password when setting up a ssh remote. * copy --auto was checking the wrong repo's preferred content. (--from was checking what --to should, and vice-versa.) Fixed this bug, which was introduced in version 5.20150727. * Avoid unnecessary write to the location log when a file is unlocked and then added back with unchanged content. * S3: Fix support for using https. * Avoid displaying network transport warning when a ssh remote does not yet have an annex.uuid set. * Debian: Add torrent library to build-depends as it's packaged now, and stop recommending bittornado | bittorrent. * Debian: Remove build dependency on transformers library, as it is now included in ghc. * Debian: Remove menu file, since a desktop file is provided and lintian says there can be only one. -- Joey Hess <id@joeyh.name> Mon, 19 Oct 2015 13:59:01 -0400 git-annex (5.20150930) unstable; urgency=medium * Added new linux standalone "ancient" build to support kernels like 2.6.32. * info: Don't allow use in a non-git-annex repository, since it uses the git-annex branch and would create it if it were missing. * assistant: When updating ~/.ssh/config, preserve any symlinks. * webapp: Remove the "disable remote" feature from the UI. * S3: When built with aws-0.13.0, supports using more storage classes. In particular, storageclass=STANDARD_IA to use Amazon's new Infrequently Accessed storage, and storageclass=NEARLINE to use Google's NearLine storage. * Improve ~/.ssh/config modification code to not add trailing spaces to lines it cannot parse. * Fix a crash at direct mode merge time when .git/index doesn't exist yet. Triggered by eg, git-annex sync --no-commit in a fresh clone of a repository. * status: Show added but not yet committed files. * Added stack.yaml to support easy builds from source with stack. -- Joey Hess <id@joeyh.name> Wed, 30 Sep 2015 14:31:52 -0400 git-annex (5.20150916) unstable; urgency=medium * Fix Windows build to work with ghc 7.10. * init: Fix reversion in detection of repo made with git clone --shared * info: Support querying info of individual files in direct mode. * unused: Fix reversion in 5.20150727 that broke parsing of the --unused-refspec option. Thanks, Øyvind A. Holm. * Make full option parsing be done when not in a git repo, so --help can be displayed for commands that require a git repo, etc. * fsck: Work around bug in persistent that broke display of problematically encoded filenames on stderr when using --incremental. * When gpg.program is configured, it's used to get the command to run for gpg. Useful on systems that have only a gpg2 command or want to use it instead of the gpg command. * Windows: Switched to using git for Windows, rather than msysgit. Using msysgit with git-annex is no longer supported. * Windows: Even when the user neglects to tell the git installer to add git to PATH, git-annex will still work from within the git bash shell, and the webapp can be used too. * sync: Add --no-commit, --no-pull, --no-push options to turn off parts of the sync process, as well as supporting --commit, --pull, --push, and --no-content options to specify the (current) default behavior. * annex.hardlink extended to also try to use hard links when copying from the repository to a remote. * Improve bash completion, so it completes names of remotes and backends in appropriate places. * Special remotes configured with autoenable=true will be automatically enabled when git-annex init is run. * Fix bug in combination of preferred and required content settings. When one was set to the empty string and the other set to some expression, this bug caused all files to be wanted, instead of only files matching the expression. -- Joey Hess <id@joeyh.name> Wed, 16 Sep 2015 10:31:24 -0400 git-annex (5.20150824) unstable; urgency=medium * Sped up downloads of files from ssh remotes, reducing the non-data-transfer overhead 6x. * sync: Support --jobs * sync --content: Avoid unnecessary second pull from remotes when no file transfers are made. * External special remotes can now be built that can be used in readonly mode, where git-annex downloads content from the remote using regular http. * Added WHEREIS to external special remote protocol. * importfeed --relaxed: Avoid hitting the urls of items in the feed. * Fix reversion in init when ran as root, introduced in version 5.20150731. * Reorder declaration to fix build with yesod-core > 1.4.13. Thanks, Michael Alan Dorman. * Fix building without quvi and without database. Thanks, Ben Boeckel. * Avoid building the assistant on the hurd, since an inotify equivalent is not yet implemented in git-annex for the hurd. * --debug log messages are now timestamped with fractional seconds. * --debug is passed along to git-annex-shell when git-annex is in debug mode. * Makefile: Pass LDFLAGS, CFLAGS, and CPPFLAGS through ghc and on to ld, cc, and cpp. * As a result of the Makefile changes, the Debian package is built with various hardening options. Although their benefit to a largely haskell program is unknown. -- Joey Hess <id@joeyh.name> Mon, 24 Aug 2015 14:11:05 -0700 git-annex (5.20150812) unstable; urgency=medium * Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. * metadata: Fix reversion introduced in 5.20150727 that caused recursive display of metadata to not work. * Windows: Fix bug that caused git-annex sync to fail due to missing environment variable. * Fix setting/setting/viewing metadata that contains unicode or other special characters, when in a non-unicode locale. * Simplify setup process for a ssh remote. Now it suffices to run git remote add, followed by git-annex sync. Now the remote is automatically initialized for use by git-annex, where before the git-annex branch had to manually be pushed before using git-annex sync. Note that this involved changes to git-annex-shell, so if the remote is using an old version, the manual push is still needed. * git-annex-shell: Don't let configlist auto-init repository when in readonly mode. * Perform a clean shutdown when --time-limit is reached. This includes running queued git commands, and cleanup actions normally run when a command is finished. * fsck: Commit incremental fsck database when --time-limit is reached. Previously, some of the last files fscked did not make it into the database when using --time-limit. * fsck: Commit incremental fsck database after every 1000 files fscked, or every 5 minutes, whichever comes first. Previously, commits were made every 1000 files fscked. * Linux standalone: Work around problem that prevented it from working properly if unpacked into a directory that contains ":" or ";" in its name. * proxy: Fix proxy git commit of non-annexed files in direct mode. * proxy: If a non-proxied git command, such as git revert would normally fail because of unstaged files in the work tree, make the proxied command fail the same way. * proxy: Fix removal of files deleted by the proxied command. * proxy: Fix behavior when run in subdirectory of git repo. * Improve Setup.hs file so that cabal copy --destdir works. Thanks, Magnus Therning. * Tighten dependency on optparse-applicative to 0.11.0. * Added back debian/cabal-wrapper, since it still seems needed after all. -- Joey Hess <id@joeyh.name> Wed, 12 Aug 2015 11:14:58 -0400 git-annex (5.20150731) unstable; urgency=medium * webapp: Support enabling known gitlab.com remotes. * Fix rsync special remote to work when -Jn is used for concurrent uploads. * The last release accidentally removed a number of options from the copy command. (-J, file matching options, etc). These have been added back. * init: Detect when the filesystem is crippled such that it ignores attempts to remove the write bit from a file, and enable direct mode. Seen with eg, NTFS fuse on linux. * Fix man page installation by cabal install; all the new man pages are now installed. -- Joey Hess <id@joeyh.name> Fri, 31 Jul 2015 11:34:36 -0400 git-annex (5.20150727) unstable; urgency=medium * Fix bug that prevented uploads to remotes using new-style chunking from resuming after the last successfully uploaded chunk. * Switched option parsing to use optparse-applicative. This was a very large and invasive change, and may have caused some minor behavior changes to edge cases of option parsing. (For example, the metadata command no longer accepts the combination of --get and --set, which never actually worked.) * Bash completion file is now included in the git-annex source tree, and installed into Debian package (and any other packages built using make install). This bash completion is generated by the option parser, so it covers all commands, all options, and will never go out of date! * As well as tab completing "git-annex" commands, "git annex" will also tab complete. However, git's bash completion script needs a patch, which I've submitted, for this to work prefectly. * version --raw now works when run outside a git repository. * assistant --startdelay now works when run outside a git repository. * dead now accepts multiple --key options. * addurl now accepts --prefix and --suffix options to adjust the filenames used. * sync --content: Fix bug that caused files to be uploaded to eg, more archive remotes than wanted copies, only to later be dropped to satisfy the preferred content settings. * importfeed: Improve detection of known items whose url has changed, and avoid adding redundant files. Where before this only looked at permalinks in rss feeds, it now also looks at guids. * importfeed: Look at not only permalinks, but now also guids to identify previously downloaded files. * Webapp: Now features easy setup of git-annex repositories on gitlab.com. * Adjust debian build deps: The webapp can now build on arm64, s390x and hurd-i386. WebDAV support is also available on those architectures. * Debian package now maintained by Richard Hartmann. * Support building without persistent database on for systems that lack TH. This removes support for incremental fsck. -- Joey Hess <id@joeyh.name> Mon, 27 Jul 2015 12:24:49 -0400 git-annex (5.20150710) unstable; urgency=medium * add: Stage symlinks the same as git add would, even if they are not a link to annexed content. * sync: When annex.autocommit=false, avoid making any commit of local changes, while still merging with remote to the extent possible. * unused: --used-refspec can now be configured to look at refs in the reflog. This provides a way to not consider old versions of files to be unused after they have reached a specified age, when the old refs in the reflog expire. * log: Fix reversion introduced in version 5.20150528 that broke this command. * assistant --autostart: First stop any daemons that are already running, which might be left over from a previous login session and so unable to use the ssh agent of a new login session. * assistant: Fix local pairing to not include newline in ssh pubkey, which is rejected on the other end for security reasons. * assistant: Fix ANNEX_SHELL_DIR written to ~/.ssh/authorized_keys in local pairing to be the absolute path to the repository, not "." This was a reversion caused by the relative path changes in 5.20150113. * Brought back the setkey plumbing command that was removed in 2011, since we found a use case for it. Note that the command's syntax was changed for consistency. * bugfix: Pass --full-tree when using git ls-files to get a list of files on the git-annex branch, so it works when run in a subdirectory. This bug affected git-annex unused, and potentially also transitions running code and other things. * Support git's undocumented core.sharedRepository=2 value, which is equivalent to "world", and is set when a repo was created using git init --shared=world. * When building on linux, pass --as-needed to linker to avoid linking with unused shared libraries including libyaml. * import: Fix failure of cross-device import on Windows. * merge: Avoid creating the synced/master branch. * Removed support for optparse-applicative versions older than 0.10. -- Joey Hess <id@joeyh.name> Fri, 10 Jul 2015 16:36:42 -0400 git-annex (5.20150617) unstable; urgency=medium * Now supports git annex sync --all --content to sync all versions of all files with all repos that want them. * Added new "anything" preferred content expression, which matches all versions of all files. * Standard preferred content for client, backup, incremental backup, and unwanted groups have been adjusted to work better when used with git annex sync --all --content. * fromkey, registerurl: Improve handling of urls that happen to also be parsable as strange keys. * sync, remotedaemon: Pass configured ssh-options even when annex.sshcaching is disabled. * assistant: Consume systemd-networkd dbus events to learn about changes to network connections, as was already done with network-manager and wicd. Thanks to Sebastian Reuße for the patches. * get --incomplete: New option to resume any interrupted downloads. * dead --key: Can be used to mark a key as dead. * fsck: Ignore keys that are known to be dead when running in --all/--unused/--key mode or a in a bare repo. Closes: #753888 Otherwise, still reports files with lost contents, even if the content is dead. * S3: Special remotes can be configured with public=yes to allow the public to access the bucket's content. * S3: Publically accessible buckets can be used without creds. * import --clean-duplicates: Fix bug that didn't count local or trusted repo's copy of a file as one of the necessary copies to allow removing it from the import location. * tahoe: Use ~/.tahoe-git-annex/ rather than ~/.tahoe/git-annex/ when setting up a tahoe special remote to avoid old versions of tahoe create-client choking. * Fix bug that prevented enumerating locally present objects in repos tuned with annex.tune.objecthash1=true. Fixes: unused, object count in info, unannex. * Improve url parsing to handle some urls containing illegal [] characters in their paths. * info: Added json output for "backend usage", "numcopies stats", "repositories containing these files", and "transfers in progress". * Fix incremental backup standard preferred content expression to match its documentation, which says it does not want files that have reached a backup repository. * Increased the default annex.bloomaccuracy from 1000 to 10000000. This makes git annex unused use up to 16 mb more memory than it did before, but the massive increase in accuracy makes this worthwhile for all but the smallest systems. * Build documentation with deterministic=1 for reproducible builds. (A new ikiwiki feature.) Closes: #785736 * Re-remove dependency on obsolete hamlet package. Closes: #786659 * debian/cabal-wrapper: Removed this hack which should not be needed anymore. -- Joey Hess <id@joeyh.name> Wed, 17 Jun 2015 13:50:35 -0400 git-annex (5.20150528) unstable; urgency=medium * fromkey, registerurl: Allow urls to be specified instead of keys, and generate URL keys. * Linux standalone, OSX app: Improve runshell script to always quote shell vars, so that it will work when eg, untarred into a directory path with spaces in its name. * Revert removal dependency on obsolete hamlet package, since the autobuilders are not ready for this change yet and it prevented them from building the webapp. Reopens: #786659 * fsck: When checksumming a file fails due to a hardware fault, the file is now moved to the bad directory, and the fsck proceeds. Before, the fsck immediately failed. * Linux standalone: The webapp was not built in the previous release, this release fixes that oversight. -- Joey Hess <id@joeyh.name> Thu, 28 May 2015 10:48:03 -0400 git-annex (5.20150522) unstable; urgency=medium * import: Refuse to import files that are within the work tree, as that does not make sense and could cause data loss. * drop: Now supports --all, --unused, and --key. * drop: Now defaults to --all when run in a bare repository. (Previously, did nothing when run in a bare repository.) * get, move, copy, mirror: Concurrent transfers are now supported! For example: git-annex get -J10 However, progress bars are not yet displayed for concurrent transfers, pending an updated version of the ascii-progress library. * --quiet now makes progress output by rsync, wget, etc be quiet too. * Take space that will be used by other running downloads into account when checking annex.diskreserve. * Avoid accumulating transfer failure log files unless the assistant is being used. * Fix an unlikely race that could result in two transfers of the same key running at once. * Stale transfer lock and info files will be cleaned up automatically when get/unused/info commands are run. * unused: Add --used-refspec option and annex.used-refspec, which can specify a set of refs to consider used, rather than the default of considering all refs used. * webapp: Fix zombie xdg-open process left when opening file browser. Closes: #785498 * Safer posix fctnl locking implementation, using lock pools and STM. * Build documentation with TZ=UTC for reproducible builds. See #785736. * OSX: Corrected the location of trustedkeys.gpg, so the built-in upgrade code will find it. Fixes OSX upgrade going forward, but older versions won't upgrade themselves due to this problem. * Remove dependency on obsolete hamlet package. Closes: #786659 -- Joey Hess <id@joeyh.name> Fri, 22 May 2015 14:20:18 -0400 git-annex (5.20150508.1) unstable; urgency=medium * Now builds cleanly using ghc 7.10 (as well as ghc back to 7.6). * Imrovements to the git-annex-standalone.deb build process. (Thanks, Yaroslav Halchenko) -- Joey Hess <id@joeyh.name> Mon, 11 May 2015 12:08:58 -0400 git-annex (5.20150508) unstable; urgency=medium * Improve behavior when a git-annex command is told to operate on a file that doesn't exist. It will now continue to other files specified after that on the command line, and only error out at the end. * S3: Enable debug logging when annex.debug or --debug is set. * S3: git annex info will show additional information about a S3 remote (endpoint, port, storage class) * S3: Let git annex enableremote be used, without trying to recreate a bucket that should already exist. * S3: Fix incompatibility with bucket names used by hS3; the aws library cannot handle upper-case bucket names. git-annex now converts them to lower case automatically. * import: Check for gitignored files before moving them into the tree. (Needs git 1.8.4 or newer.) * import: Don't stop entire import when one file fails due to being gitignored or conflicting with something in the work tree. * import: Before removing a duplicate file in --deduplicate or --clean-duplicates mode, verify that enough copies of its content still exist. * Improve integration with KDE's file manager to work with dolphin version 14.12.3 while still being compatable with 4.14.2. Thanks, silvio. * assistant: Added --autostop to complement --autostart. * Work around wget bug #784348 which could cause it to clobber git-annex symlinks when downloading from ftp. * Support checking ftp urls for file presence. * Fix bogus failure of fsck --fast. * fsck: Ignore error recording the fsck in the activity log, which can happen when running fsck in a read-only repository. Closes: #698559 (fsck can still need to write to the repository if it find problems, but a successful fsck can be done read-only) * Improve quvi 0.4 output parsing to handle cases wher there is no known filename extension. This is currently the case when using quvi with youtube. In this case, the extension ".m" will be used. * Dropped support for older versions of yesod, warp, and dbus than the ones in Debian Jessie. * Switch from the obsolete dataenc library for base64 encoding to sandi. (Thanks, Magnus Therning) * Debian's ghc now supports TH on arm! Adjust build dependencies to build the webapp on arm, and enable DAV support on arm. \o/ * Adjust some other arch specific build dependencies that are now available on more architectures in Devian unstable. * Windows: Remove cygwin ssh, the newer version of which has stopped honoring the setting of HOME. Instead, copy msysgit's ssh into PATH. Note that setting up a remote ssh server using password authentication is known to be broken in this release on Windows. * Windows: Roll back to an older version of rsync from cygwin. The newer version has some dependency on a newer ssh from cygwin. -- Joey Hess <id@joeyh.name> Fri, 08 May 2015 13:42:30 -0400 git-annex (5.20150420) unstable; urgency=medium * Fix activity log parsing, which caused the log to not retain activity from other uuids. * Union merge could fall over if there was a file in the repository with the same name as a git ref. Now fixed. * info dir: Added information about repositories that contain files in the specified directory. * info: Added --bytes option. * bittorrent: Fix handling of magnet links. * When a key's size is unknown, still check the annex.diskreserve, and avoid getting content if the disk is too full. * Fix fsck --from a git remote in a local directory, and from a directory special remote. This was a reversion caused by the relative path changes in 5.20150113. * fsck --from remote: When bad content is found in the remote, and the local repo does not have a copy of the content, preserve the bad content in .git/annex/bad/ to avoid further data loss. * fsck --from remote: Avoid downloading a key if it would go over the annex.diskreserve limit. * required: New command, like wanted, but for required content. * Removed dependency on haskell SHA library, instead using cryptohash >= 0.11.0. * Make repo init more robust. * New debian/rules build-standalone target, which generates a git-annex-standalone.deb that should work on many old Debian etc systems. Thanks, Yaroslav Halchenko. * Windows: Renamed start menu file to avoid loop in some versions of Windows where the menu file is treated as a git-annex program. * Windows: Fixed support of remotes on other drives. (A reversion introduced in version 5.20150113.) * Windows: Bundled versions of rsync, wget, ssh, and gpg from cygwin all updated. Thanks, Yury V. Zaytsev. -- Joey Hess <id@joeyh.name> Mon, 20 Apr 2015 14:44:04 -0400 git-annex (5.20150409) unstable; urgency=medium * This fixes a bug in the assistant introduced by the literal pathspec changes in version 5.20150406. * --quiet now suppresses progress displays from eg, rsync. (Second time's the charm..) * fromkey, registerurl: When reading from stdin, allow the filename and url, respectively, to contain whitespace. * add: If annex.largefiles is set and does not match a file that's being added, the file will be checked into git rather than being added to the annex. Previously, git annex add skipped over such files; this new behavior is more useful in direct mode. * proxy: Made it work when run in a new repository before initial commit. * info: Display repository mode: bare when in a bare (non-direct mode) repo. * importfeed: Fix feed download when curl is used. * importfeed: Error out when passed a non-url. * webapp: When adding another local repository, and combining it with the current repository, the new repository's remote path was set to "." rather than the path to the current repository. This was a reversion caused by the relative path changes in 5.20150113. * contentlocationn: New plumbing command. -- Joey Hess <id@joeyh.name> Thu, 09 Apr 2015 15:06:38 -0400 git-annex (5.20150406.1) unstable; urgency=medium * Fixes a bug in the last release that caused rsync and possibly other commands to hang at the end of a file transfer. (--quiet is back to not blocking progress displays until that code can be fixed properly.) -- Joey Hess <id@joeyh.name> Mon, 06 Apr 2015 17:13:13 -0400 git-annex (5.20150406) unstable; urgency=medium * Prevent git-ls-files from double-expanding wildcards when an unexpanded wildcard is passed to a git-annex command like add or find. * Fix make build target. Thanks, Justin Geibel. * Fix GETURLS in external special remote protocol to strip downloader prefix from logged url info before checking for the specified prefix. * importfeed: Avoid downloading a redundant item from a feed whose permalink has been seen before, even when the url has changed. * importfeed: Always store itemid in metadata; before this was only done when annex.genmetadata was set. * Relax debian package dependencies to git >= 1:1.8.1 rather than needing >= 1:2.0. * test: Fix --list-tests * addurl --file: When used with a special remote that claims urls and checks their contents, don't override the user's provided filename with filenames that the special remote suggests. Also, don't allow adding the url if the special remote says it contains multiple files. * import: --deduplicate and --cleanduplicates now output the keys corresponding to duplicated files they process. * expire: New command, for expiring inactive repositories. * fsck: Record fsck activity for use by expire command. * Fix truncation of parameters that could occur when using xargs git-annex. * Significantly sped up processing of large numbers of directories passed to a single git-annex command. * version: Add --raw * init: Improve fifo test to detect NFS systems that support fifos but not well enough for sshcaching. * --quiet now suppresses progress displays from eg, rsync. (The option already suppressed git-annex's own built-in progress displays.) -- Joey Hess <id@joeyh.name> Mon, 06 Apr 2015 12:48:48 -0400 git-annex (5.20150327) unstable; urgency=medium * readpresentkey: New plumbing command for checking location log. * checkpresentkey: New plumbing command to check if a key can be verified to be present on a remote. * Added a post-update-annex hook, which is run after the git-annex branch is updated. Needed for git update-server-info. * migrate: --force will force migration of keys already using the destination backend. Useful in rare cases. * Man pages for individual commands now available, and can be opened using "git annex help <command>" * --auto is no longer a global option; only get, drop, and copy accept it. (Not a behavior change unless you were passing it to a command that ignored it.) * Improve error message when --in @date is used and there is no reflog for the git-annex branch. * assistant: Committing a whole lot of files at once could overflow command-line length limits and cause the commit to fail. This only happened when using the assistant in an indirect mode repository. * Work around curl bug when asked to download an empty url to a file. * Fix bug introduced in the last release that broke git-annex sync when git-annex was installed from the standalone tarball. -- Joey Hess <id@joeyh.name> Fri, 27 Mar 2015 13:10:59 -0400 git-annex (5.20150317) unstable; urgency=medium * fsck: Incremental fsck uses sqlite to store its records, instead of abusing the sticky bit. Existing sticky bits are ignored; incremental fscks started by old versions won't be resumed by this version. * fsck: Multiple incremental fscks of different repos (including remotes) can now be running at the same time in the same repo without it getting confused about which files have been checked for which remotes. * unannex: Refuse to unannex when repo is too new to have a HEAD, since in this case there must be staged changes in the index (if there is anything to unannex), and the unannex code path needs to run with a clean index. * Linux standalone: Set LOCPATH=/dev/null to work around https://ghc.haskell.org/trac/ghc/ticket/7695 This prevents localization from working, but git-annex is not localized anyway. * sync: As well as the synced/git-annex push, attempt a git-annex:git-annex push, as long as the remote branch is an ancestor of the local branch, to better support bare git repos. (This used to be done, but it forgot to do it since version 4.20130909.) * When re-execing git-annex, use current program location, rather than ~/.config/git-annex/program, when possible. * Submodules are now supported by git-annex! * metadata: Fix encoding problem that led to mojibake when storing metadata strings that contained both unicode characters and a space (or '!') character. * Also potentially fixes encoding problem when embedding credentials that contain unicode characters. * sync: Fix committing when in a direct mode repo that has no HEAD ref. (For example, a newly checked out git submodule.) * Added SETURIPRESENT and SETURIMISSING to external special remote protocol, useful for things like ipfs that don't use regular urls. * addurl: Added --raw option, which bypasses special handling of quvi, bittorrent etc urls. * git-annex-shell: Improve error message when the specified repository doesn't exist or git config fails for some reason. * fromkey --force: Skip test that the key has its content in the annex. * fromkey: Add stdin mode. * registerurl: New plumbing command for mass-adding urls to keys. * remotedaemon: Fixed support for notifications of changes to gcrypt remotes, which was never tested and didn't quite work before. -- Joey Hess <id@joeyh.name> Tue, 17 Mar 2015 13:02:36 -0400 git-annex (5.20150219) unstable; urgency=medium * glacier: Detect when the glacier command in PATH is the wrong one, from boto, rather than from glacier-cli, and refuse to use it, since the boto program fails to fail when passed parameters it does not understand. * groupwanted: New command to set the groupwanted preferred content expression. * import: Support file matching options such as --exclude, --include, --smallerthan, --largerthan * The file matching options are now only accepted by commands that can actually use them, instead of by all commands. * import: Avoid checksumming file twice when run in the default or --duplicate mode. * Windows: Fix bug in dropping an annexed file, which caused a symlink to be staged that contained backslashes. * webapp: Fix reversion in opening webapp when starting it manually inside a repository. * assistant: Improve sanity check for control characters when pairing. * Improve race recovery code when committing to git-annex branch. * addurl: Avoid crash if quvi is not installed, when git-annex was built with process-1.2 * bittorrent: Fix mojibake introduced in parsing arai2c progress output. * fsck --from: If a download from a remote fails, propagate the failure. * metadata: When setting metadata, do not recurse into directories by default, since that can be surprising behavior and difficult to recover from. The old behavior is available by using --force. * sync, assistant: Include repository name in head branch commit message. * The ssh-options git config is now used by gcrypt, rsync, and ddar special remotes that use ssh as a transport. * sync, assistant: Use the ssh-options git config when doing git pull and push. * remotedaemon: Use the ssh-options git config. * Linux standalone: Improved process names of linker shimmed programs. -- Joey Hess <id@joeyh.name> Thu, 19 Feb 2015 14:16:03 -0400 git-annex (5.20150205) unstable; urgency=medium * info: Can now display info about a given uuid. * Added to remote/uuid info: Count of the number of keys present on the remote, and their size. This is rather expensive to calculate, so comes last and --fast will disable it. * info remote: Include the date of the last sync with the remote. * sync: Added --message/-m option like git commit. * remotedaemon: Fix problem that could prevent ssh connections being made after two LOSTNET messages were received in a row (perhaps due to two different network interfaces being brought down). * Fix build failure when wget is not installed. * Fix wording of message displayed when unable to get a file that is available in untrusted repositories. * addurl: When a Content-Disposition header suggests a filename to use, addurl will consider using it, if it's reasonable and doesn't conflict with an existing file. (--file overrides this) * Fix default repository description created by git annex init, which got broken by the relative path changes in the last release. * init: Repository tuning parameters can now be passed when initializing a repository for the first time. For details, see http://git-annex.branchable.com/tuning/ * merge: Refuse to merge changes from a git-annex branch of a repo that has been tuned in incompatible ways. * Support annex.tune.objecthash1, annex.tune.objecthashlower, and annex.tune.branchhash1. * Remove support for building without cryptohash. * Added MD5 and MD5E backends. * assistant: Fix local pairing when ssh pubkey comment contains spaces. * Avoid using fileSize which maxes out at just 2 gb on Windows. Instead, use hFileSize, which doesn't have a bounded size. Fixes support for files > 2 gb on Windows. * Windows: Fix running of the pre-commit-annex hook. * Windows: Fix S3 special remote; need to call withSocketsDo. Thanks, Trent. -- Joey Hess <id@joeyh.name> Thu, 05 Feb 2015 14:08:33 -0400 git-annex (5.20150113) unstable; urgency=medium * unlock: Don't allow unlocking files that have never been committed to git before, to avoid an intractable problem that prevents the pre-commit hook from telling if such a file is intended to be an annexed file or not. * Avoid re-checksumming when migrating from hash to hashE backend. Closes: #774494 * Fix build with process 1.2.1.0. * Android: Provide a version built with -fPIE -pie to support Android 5.0. * sync: Fix an edge case where syncing in a bare repository would try to merge and so fail. * Check git version at runtime, rather than assuming it will be the same as the git version used at build time when running git-checkattr and git-branch remove. * Switch to using relative paths to the git repository. - This allows the git repository to be moved while git-annex is running in it, with fewer problems. - On Windows, this avoids some of the problems with the absurdly small MAX_PATH of 260 bytes. In particular, git-annex repositories should work in deeper/longer directory structures than before. * Generate shorter keys for WORM and URL, avoiding keys that are longer than used for SHA256, so as to not break on systems like Windows that have very small maximum path length limits. * Bugfix: A file named HEAD in the work tree could confuse some git commands run by git-annex. -- Joey Hess <id@joeyh.name> Tue, 13 Jan 2015 12:10:08 -0400 git-annex (5.20141231) unstable; urgency=medium * vicfg: Avoid crashing on badly encoded config data. * Work around statfs() overflow on some XFS systems. * sync: Now supports remote groups, the same way git remote update does. * setpresentkey: A new plumbing-level command. * Run shutdown cleanup actions even if there were failures processing the command. Among other fixes, this means that addurl will stage added files even if adding one of the urls fails. * bittorrent: Fix locking problem when using addurl file:// * Windows: Fix local rsync filepath munging (fixes 26 test suite failures). * Windows: Got the rsync special remote working. * Windows: Fix handling of views of filenames containing '%' * OSX: Switched away from deprecated statfs64 interface. -- Joey Hess <id@joeyh.name> Wed, 31 Dec 2014 15:15:46 -0400 git-annex (5.20141219) unstable; urgency=medium * Webapp: When adding a new box.com remote, use the new style chunking. Thanks, Jon Ander Peñalba. * External special remote protocol now includes commands for setting and getting the urls associated with a key. * Urls can now be claimed by remotes. This will allow creating, for example, a external special remote that handles magnet: and *.torrent urls. * Use wget -q --show-progress for less verbose wget output, when built with wget 1.16. * Added bittorrent special remote. * addurl behavior change: When downloading an url ending in .torrent, it will download files from bittorrent, instead of the old behavior of adding the torrent file to the repository. * Added Recommends on aria2. * When possible, build with the haskell torrent library for parsing torrent files. As a fallback, can instead use btshowmetainfo from bittornado | bittorrent. * Fix build with -f-S3. -- Joey Hess <id@joeyh.name> Fri, 19 Dec 2014 16:53:26 -0400 git-annex (5.20141203) unstable; urgency=medium * proxy: New command for direct mode repositories, allows bypassing the direct mode guard in a safe way to do all sorts of things including git revert, git mv, git checkout ... * undo: New command to undo the most recent change to a file or to the contents of a directory. * Add undo action to nautilus and konqueror integration. * diffdriver: New git-annex command, to make git external diff drivers work with annexed files. * pre-commit: Block partial commit of unlocked annexed file, since that left a typechange staged in index due to some infelicity of git's handling of partial commits. * Work around behavior change in lsof 4.88's -F output format. * S3: Switched to using the haskell aws library. * S3: No longer buffers entire files in memory when uploading without chunking. * S3: When built with a new enough version of the haskell aws library, supports doing multipart uploads, in order to store extremely large files in S3 when not using chunking. * Don't show "(gpg)" when decrypting the remote encryption cipher, since this could be taken to read that's the only time git-annex runs gpg, which is not the case. * Debian package is now maintained by Gergely Nagy. * Windows: Remove Alt+A keyboard shortcut, which turns out to have scope outside the menus. * Windows: Install ssh and other bundled programs to Git/cmd, instead of Git/bin, since the latter is not in the default msysgit PATH. -- Joey Hess <id@joeyh.name> Wed, 03 Dec 2014 15:16:52 -0400 git-annex (5.20141125) unstable; urgency=medium * Remove fixup code for bad bare repositories created by versions 5.20131118 through 5.20131127. That fixup code would accidentally fire when --git-dir was incorrectly pointed at the working tree of a git-annex repository, possibly resulting in data loss. Closes: #768093 * Windows: Fix crash when user.name is not set in git config. -- Joey Hess <joeyh@debian.org> Wed, 05 Nov 2014 11:41:51 -0400 git-annex (5.20141024) unstable; urgency=medium * vicfg: Deleting configurations now resets to the default, where before it has no effect. * Remove hurd stuff from cabal file, since hackage currently rejects it, and the test suite fails on hurd. * initremote: Don't allow creating a special remote that has the same name as an existing git remote. * Windows: Use haskell setenv library to clean up several ugly workarounds for inability to manipulate the environment on windows. This includes making git-annex not re-exec itself on start on windows, and making the test suite on Windows run tests without forking. * glacier: Fix pipe setup when calling glacier-cli to retrieve an object. * info: When run on a single annexed file, displays some info about the file, including its key and size. * info: When passed the name or uuid of a remote, displays info about that remote. Remotes that support encryption, chunking, or embedded creds will include that in their info. * enableremote: When the remote has creds, update the local creds cache file. Before, the old version of the creds could be left there, and would continue to be used. -- Joey Hess <joeyh@debian.org> Fri, 24 Oct 2014 13:03:29 -0400 git-annex (5.20141013) unstable; urgency=medium * Adjust cabal file to support building w/o assistant on the hurd. * Support building with yesod 1.4. * S3: Fix embedcreds=yes handling for the Internet Archive. * map: Handle .git prefixed remote repos. Closes: #614759 * repair: Prevent auto gc from happening when fetching from a remote. -- Joey Hess <joeyh@debian.org> Mon, 13 Oct 2014 10:13:06 -0400 git-annex (5.20140927) unstable; urgency=medium * Really depend (not just build-depend) on new enough git for --no-gpg-sign to work. Closes: #763057 * Add temporary workaround for bug #763078 which broke building on armel and armhf. -- Joey Hess <joeyh@debian.org> Sat, 27 Sep 2014 14:25:09 -0400 git-annex (5.20140926) unstable; urgency=high * Depend on new enough git for --no-gpg-sign to work. Closes: #762446 * Work around failure to build on mips by using cabal, not Setup, to build in debian/rules. -- Joey Hess <joeyh@debian.org> Fri, 26 Sep 2014 15:09:02 -0400 git-annex (5.20140919) unstable; urgency=high * Security fix for S3 and glacier when using embedcreds=yes with encryption=pubkey or encryption=hybrid. CVE-2014-6274 The creds embedded in the git repo were *not* encrypted. git-annex enableremote will warn when used on a remote that has this problem. For details, see: https://git-annex.branchable.com/upgrades/insecure_embedded_creds/ * assistant: Detect when repository has been deleted or moved, and automatically shut down the assistant. Closes: #761261 * Windows: Avoid crashing trying to list gpg secret keys, for gcrypt which is not yet supported on Windows. * WebDav: Fix enableremote crash when the remote already exists. (Bug introduced in version 5.20140817.) * add: In direct mode, adding an annex symlink will check it into git, as was already done in indirect mode. -- Joey Hess <joeyh@debian.org> Fri, 19 Sep 2014 12:53:42 -0400 git-annex (5.20140915) unstable; urgency=medium * New annex.hardlink setting. Closes: #758593 * init: Automatically detect when a repository was cloned with --shared, and set annex.hardlink=true, as well as marking the repository as untrusted. * Fix parsing of ipv6 address in git remote address when it was not formatted as an url. * The annex-rsync-transport configuration is now also used when checking if a key is present on a rsync remote, and when dropping a key from the remote. * Promote file not found warning message to an error. * Fix transfer lock file FD leak that could occur when two separate git-annex processes were both working to perform the same set of transfers. * sync: Ensure that pending changes to git-annex branch are committed before push when in direct mode. (Fixing a very minor reversion.) * WORM backend: Switched to include the relative path to the file inside the repository, rather than just the file's base name. Note that if you're relying on such things to keep files separate with WORM, you should really be using a better backend. * Rather than crashing when there's a problem with the requested bloomfilter capacity/accuracy, fall back to a reasonable default bloom filter size. * Fix build with optparse-applicative 0.10. Closes: #761484 * webapp: Fixed visual glitch in xmpp pairing that was reported live by a user who tracked me down in front of a coffee cart in Portland. (New bug reporting method of choice?) -- Joey Hess <joeyh@debian.org> Mon, 15 Sep 2014 10:45:00 -0400 git-annex (5.20140831) unstable; urgency=medium * Make --help work when not in a git repository. Closes: #758592 * Ensure that all lock fds are close-on-exec, fixing various problems with them being inherited by child processes such as git commands. * When accessing a local remote, shut down git-cat-file processes afterwards, to ensure that remotes on removable media can be unmounted. Closes: #758630 * Fix handing of autocorrection when running outside a git repository. * Fix stub git-annex test support when built without tasty. * Do not preserve permissions and acls when copying files from one local git repository to another. Timestamps are still preserved as long as cp --preserve=timestamps is supported. Closes: #729757 -- Joey Hess <joeyh@debian.org> Sun, 31 Aug 2014 12:30:08 -0700 git-annex (5.20140817) unstable; urgency=medium * New chunk= option to chunk files stored in special remotes. Supported by: directory, S3, webdav, gcrypt, rsync, and all external and hook special remotes. * Partially transferred files are automatically resumed when using chunked remotes! * The old chunksize= option is deprecated. Do not use for new remotes. * Legacy code for directory remotes using the old chunksize= option will keep them working, but more slowly than before. * webapp: Automatically install Konqueror integration scripts to get and drop files. * repair: Removing bad objects could leave fsck finding no more unreachable objects, but some branches no longer accessible. Fix this, including support for fixing up repositories that were incompletely repaired before. * Fix cost calculation for non-encrypted remotes. * Display exception message when a transfer fails due to an exception. * WebDAV: Sped up by avoiding making multiple http connections when storing a file. * WebDAV: Avoid buffering whole file in memory when uploading and downloading. * WebDAV: Dropped support for DAV before 1.0. * testremote: New command to test uploads/downloads to a remote. * Dropping an object from a bup special remote now deletes the git branch for the object, although of course the object's content cannot be deleted due to the nature of bup. * unlock: Better error handling; continue past files that are not available or cannot be unlocked due to disk space, and try all specified files. * Windows: Now uses actual inode equivilants in new direct mode repositories, for safer detection of eg, renaming of files with the same size and mtime. * direct: Fix ugly warning messages. * WORM backend: When adding a file in a subdirectory, avoid including the subdirectory in the key name. * S3, Glacier, WebDAV: Fix bug that prevented accessing the creds when the repository was configured with encryption=shared embedcreds=yes. * direct: Avoid leaving file content in misctemp if interrupted. * git-annex-shell sendkey: Don't fail if a remote asks for a key to be sent that already has a transfer lock file indicating it's being sent to that remote. The remote may have moved between networks, or reconnected. * Switched from the old haskell HTTP library to http-conduit. -- Joey Hess <joeyh@debian.org> Sun, 17 Aug 2014 10:30:58 -0400 git-annex (5.20140717) unstable; urgency=high * Fix minor FD leak in journal code. Closes: #754608 * direct: Fix handling of case where a work tree subdirectory cannot be written to due to permissions. * migrate: Avoid re-checksumming when migrating from hashE to hash backend. * uninit: Avoid failing final removal in some direct mode repositories due to file modes. * S3: Deal with AWS ACL configurations that do not allow creating or checking the location of a bucket, but only reading and writing content to it. * resolvemerge: New plumbing command that runs the automatic merge conflict resolver. * Deal with change in git 2.0 that made indirect mode merge conflict resolution leave behind old files. * sync: Fix git sync with local git remotes even when they don't have an annex.uuid set. (The assistant already did so.) * Set gcrypt-publish-participants when setting up a gcrypt repository, to avoid unnecessary passphrase prompts. This is a security/usability tradeoff. To avoid exposing the gpg key ids who can decrypt the repository, users can unset gcrypt-publish-participants. * Install nautilus hooks even when ~/.local/share/nautilus/ does not yet exist, since it is not automatically created for Gnome 3 users. * Windows: Move .vbs files out of git\bin, to avoid that being in the PATH, which caused some weird breakage. (Thanks, divB) * Windows: Fix locking issue that prevented the webapp starting (since 5.20140707). -- Joey Hess <joeyh@debian.org> Thu, 17 Jul 2014 11:27:25 -0400 git-annex (5.20140709) unstable; urgency=medium * Fix race in direct mode merge code that could cause all files in the repository to be removed. It should be able to recover repositories experiencing this bug without data loss. See: http://git-annex.branchable.com/bugs/bad_merge_commit_deleting_all_files/ * Fix git version that supported --no-gpg-sign. * Fix bug in automatic merge conflict resolution, when one side is an annexed symlink, and the other side is a non-annexed symlink. * Really fix bug that caused the assistant to make many unnecessary empty merge commits. -- Joey Hess <joeyh@debian.org> Wed, 09 Jul 2014 15:28:03 -0400 git-annex (5.20140707) unstable; urgency=medium * assistant: Fix bug, introduced in last release, that caused the assistant to make many unnecessary empty merge commits. * assistant: Fix one-way assistant->assistant sync in direct mode. * Fix bug in annex.queuesize calculation that caused much more queue flushing than necessary. * importfeed: When annex.genmetadata is set, metadata from the feed is added to files that are imported from it. * Support users who have set commit.gpgsign, by disabling gpg signatures for git-annex branch commits and commits made by the assistant. * Fix memory leak when committing millions of changes to the git-annex branch, eg after git-annex add has run on 2 million files in one go. * Support building with bloomfilter 2.0.0. * Run standalone install process when the assistant is started (was only being run when the webapp was opened). * Android: patch git to avoid fchmod, which fails on /sdcard. * Windows: Got rid of that pesky DOS box when starting the webapp. * Windows: Added Startup menu item so assistant starts automatically on login. * Windows: Fix opening file browser from webapp when repo is in a directory with spaces. * Windows: Assistant now logs to daemon.log. -- Joey Hess <joeyh@debian.org> Mon, 07 Jul 2014 12:24:13 -0400 git-annex (5.20140613) unstable; urgency=medium * Ignore setsid failures. * Avoid leaving behind .tmp files when failing in some cases, including importing files to a disk that is full. * Avoid bad commits after interrupted direct mode sync (or merge). * Fix build with wai 0.3.0. * Deal with FAT's low resolution timestamps, which in combination with Linux's caching of higher res timestamps while a FAT is mounted, caused direct mode repositories on FAT to seem to have modified files after they were unmounted and remounted. * Windows: Fix opening webapp when repository is in a directory with spaces in the path. * Detect when Windows has lost its mind in a timezone change, and automatically apply a delta to the timestamps it returns, to get back to sane values. -- Joey Hess <joeyh@debian.org> Fri, 13 Jun 2014 09:58:07 -0400 git-annex (5.20140606) unstable; urgency=medium * webapp: When adding a new local repository, fix bug that caused its group and preferred content to be set in the current repository, even when not combining. * webapp: Avoid stomping on existing description, group and preferred content settings when enabling or combining with an already existing remote. * assistant: Make sanity checker tmp dir cleanup code more robust. * unused: Avoid checking view branches for unused files. * webapp: Include ssh port in mangled hostname. * Windows: Fix bug introduced in last release that caused files in the git-annex branch to have lines teminated with \r. * Windows: Fix retrieving of files from local bare git repositories. -- Joey Hess <joeyh@debian.org> Fri, 06 Jun 2014 12:54:06 -0400 git-annex (5.20140529) unstable; urgency=medium * Fix encoding of data written to git-annex branch. Avoid truncating unicode characters to 8 bits. Allow any encoding to be used, as with filenames (but utf8 is the sane choice). Affects metadata and repository descriptions, and preferred content expressions. * assistant: When there are multiple remotes giving different ways to access the same repository, honor remote cost settings and use the cheapest available. * webapp: More robust startup when annex directory is not a git repo. * initremote/enableremote: Basic support for using with regular git remotes; initremote stores the location of an already existing git remote, and enableremote setups up a remote using its stored location. * webapp: Support for enabling known git repositories on ssh servers. The repository must have been added using initremote. * webapp: When setting up a ssh remote, record it using initremote, so that it can be easily enabled elsewhere. * webapp: When setting up a ssh remote, if the user inputs ~/foo, normalize that to foo, since it's in the home directory by default. * Use exceptions in place of deprecated MonadCatchIO-transformers Thanks, Ben Gamari. * android: Run busybox install with -s, since some versions of Android prohibit making hard links. * Android webapp: Fix EvilSplicer bugs that mangled the css files, preventing icons from displaying, and also slightly broke the js files. -- Joey Hess <joeyh@debian.org> Thu, 29 May 2014 14:41:56 -0400 git-annex (5.20140517) unstable; urgency=medium * webapp: Switched to bootstrap 3. Thanks, Sören Brunk. * Standalone builds now check gpg signatures before upgrading. * Simplified repository description line format. The remote name, if any, is always in square brackets after the description. * assistant: Clean up stale tmp files on startup. * webapp: Better ssh password prompting. * Depend on git-remote-gcrypt 0.20130908-6. Older versions fail when the assistant is run with no controlling tty. * Added ddar special remote. Thanks, Robie Basak. * webapp: Fixed drag and drop to reorder the list of remotes. * group: When no groups are specified to set, lists the current groups of a repository. * Add remote.$name.annex-shell configuration. Thanks, Fraser Tweedale * Support symlinking git-annex and git-annex-shell from the Linux standalone bundle into PATH. Thanks, jlebar. -- Joey Hess <joeyh@debian.org> Sat, 17 May 2014 13:30:39 -0400 git-annex (5.20140421) unstable; urgency=medium * assistant: Now detects immediately when other repositories push changes to a ssh remote, and pulls. ** XMPP is no longer needed in this configuration! ** This requires the remote server have git-annex-shell with notifychanges support (>= 5.20140405) * webapp: Show a network signal icon next to ssh and xmpp remotes that it's currently connected with. * webapp: Rework xmpp nudge to prompt for either xmpp or a ssh remote to be set up. * sync, assistant, remotedaemon: Use ssh connection caching for git pushes and pulls. * remotedaemon: When network connection is lost, close all cached ssh connections. * Improve handling of monthly/yearly scheduling. * Avoid depending on shakespeare except for when building the webapp. * uninit: Avoid making unnecessary copies of files. * info: Allow use in a repository where annex.uuid is not set. * reinit: New command that can initialize a new repository using the configuration of a previously known repository. Useful if a repository got deleted and you want to clone it back the way it was. * drop --from: When local repository is untrusted, its copy of a file does not count. * Bring back rsync -p, but only when git-annex is running on a non-crippled file system. This is a better approach to fix #700282 while not unncessarily losing file permissions on non-crippled systems. * webapp: Start even if the current directory is listed in ~/.config/git-annex/autostart but no longer has a git repository in it. * findref: New command, like find but shows files in a specified git ref. * webapp: Fix UI for removing XMPP connection. * When init detects that git is not configured to commit, and sets user.email to work around the problem, also make it set user.name. * webapp: Support using git-annex on a remote server, which was installed from the standalone tarball or OSX app, and so does not have git-annex in PATH (and may also not have git or rsync in PATH). * standalone tarball, OSX app: Install a ~/.ssh/git-annex-wrapper, which can be used to run git-annex, git, rsync, etc. -- Joey Hess <joeyh@debian.org> Sun, 20 Apr 2014 19:43:14 -0400 git-annex (5.20140412) unstable; urgency=high * Last release didn't quite fix the high cpu issue in all cases, this should. -- Joey Hess <joeyh@debian.org> Fri, 11 Apr 2014 17:14:38 -0400 git-annex (5.20140411) unstable; urgency=high * importfeed: Filename template can now contain an itempubdate variable. Needs feed 0.3.9.2. * Fix rsync progress parsing in locales that use comma in number display. Closes: #744148 * assistant: Fix high CPU usage triggered when a monthly fsck is scheduled, and the last time the job ran was a day of the month > 12. This caused a runaway loop. Thanks to Anarcat for his assistance, and to Maximiliano Curia for identifying the cause of this bug. * Remove wget from OSX dmg, due to issues with cert paths that broke git-annex automatic upgrading. Instead, curl is used, unless the OSX system has wget installed, which will then be used. -- Joey Hess <joeyh@debian.org> Fri, 11 Apr 2014 14:59:49 -0400 git-annex (5.20140405) unstable; urgency=medium * git-annex-shell: Added notifychanges command. * Improve display of dbus notifications. Thanks, Johan Kiviniemi. * Fix nautilus script installation to not crash when the nautilus script dir does not exist. Instead, only install scripts when the directory already exists. -- Joey Hess <joeyh@debian.org> Sat, 05 Apr 2014 16:54:33 -0400 git-annex (5.20140402) unstable; urgency=medium * unannex, uninit: Avoid committing after every file is unannexed, for massive speedup. * --notify-finish switch will cause desktop notifications after each file upload/download/drop completes (using the dbus Desktop Notifications Specification) * --notify-start switch will show desktop notifications when each file upload/download starts. * webapp: Automatically install Nautilus integration scripts to get and drop files. * tahoe: Pass -d parameter before subcommand; putting it after the subcommand no longer works with tahoe-lafs version 1.10. (Thanks, Alberto Berti) * forget --drop-dead: Avoid removing the dead remote from the trust.log, so that if git remotes for it still exist anywhere, git annex info will still know it's dead and not show it. * git-annex-shell: Make configlist automatically initialize a remote git repository, as long as a git-annex branch has been pushed to it, to simplify setup of remote git repositories, including via gitolite. * add --include-dotfiles: New option, perhaps useful for backups. * Version 5.20140227 broke creation of glacier repositories, not including the datacenter and vault in their configuration. This bug is fixed, but glacier repositories set up with the broken version of git-annex need to have the datacenter and vault set in order to be usable. This can be done using git annex enableremote to add the missing settings. For details, see http://git-annex.branchable.com/bugs/problems_with_glacier/ * Added required content configuration. * assistant: Improve ssh authorized keys line generated in local pairing or for a remote ssh server to set environment variables in an alternative way that works with the non-POSIX fish shell, as well as POSIX shells. -- Joey Hess <joeyh@debian.org> Wed, 02 Apr 2014 16:42:53 -0400 git-annex (5.20140320) unstable; urgency=medium * Fix zombie leak and general inneficiency when copying files to a local git repo. * Fix ssh connection caching stop method to work with openssh 6.5p1, which broke the old method. * webapp: Added a "Sync now" item to each repository's menu. * webapp: Use securemem for constant time auth token comparisons. * copy --fast --to remote: Avoid printing anything for files that are already believed to be present on the remote. * Commands that allow specifying which repository to act on using the repository's description will now fail when multiple repositories match, rather than picking a repository at random. (So will --in=) * Better workaround for problem umasks when eg, setting up ssh keys. * "standard" can now be used as a first-class keyword in preferred content expressions. For example "standard or (include=otherdir/*)" * groupwanted can be used in preferred content expressions. * vicfg: Allows editing preferred content expressions for groups. * Improve behavior when unable to parse a preferred content expression (thanks, ion). * metadata: Add --get * metadata: Support --key option (and some other ones like --all) * For each metadata field, there's now an automatically maintained "$field-lastchanged" that gives the date of the last change to that field. Also the "lastchanged" field for the date of the last change to any of a file's metadata. * unused: In direct mode, files that are deleted from the work tree and so have no content present are no longer incorrectly detected as unused. * Avoid encoding errors when using the unused log file. * map: Fix crash when one of the remotes of a repo is a local directory that does not exist, or is not a git repo. * repair: Improve memory usage when git fsck finds a great many broken objects. * Windows: Fix some filename encoding bugs. * rsync special remote: Fix slashes when used on Windows. -- Joey Hess <joeyh@debian.org> Thu, 20 Mar 2014 13:21:12 -0400 git-annex (5.20140306) unstable; urgency=high * sync: Fix bug in direct mode that caused a file that was not checked into git to be deleted when there was a conflicting merge with a remote. * webapp: Now supports HTTPS. * webapp: No longer supports a port specified after --listen, since it was buggy, and that use case is better supported by setting up HTTPS. * annex.listen can be configured, instead of using --listen * annex.startupscan can be set to false to disable the assistant's startup scan. * Probe for quvi version at run time. * webapp: Filter out from Switch Repository list any repositories listed in autostart file that don't have a git directory anymore. (Or are bare) * webapp: Refuse to start in a bare git repository. * assistant --autostart: Refuse to start in a bare git repository. * webapp: Don't list the public repository group when editing a git repository; it only makes sense for special remotes. * view, vfilter: Add support for filtering tags and values out of a view, using !tag and field!=value. * vadd: Allow listing multiple desired values for a field. * view: Refuse to enter a view when no branch is currently checked out. * metadata: To only set a field when it's not already got a value, use -s field?=value * Run .git/hooks/pre-commit-annex whenever a commit is made. * sync: Automatically resolve merge conflict between and annexed file and a regular git file. * glacier: Pass --region to glacier checkpresent. * webdav: When built with a new enough haskell DAV (0.6), disable the http response timeout, which was only 5 seconds. * webapp: Include no-pty in ssh authorized_keys lines. * assistant: Smarter log file rotation, which takes free disk space into account. -- Joey Hess <joeyh@debian.org> Thu, 06 Mar 2014 12:28:04 -0400 git-annex (5.20140227) unstable; urgency=medium * metadata: Field names limited to alphanumerics and a few whitelisted punctuation characters to avoid issues with views, etc. * metadata: Field names are now case insensative. * When constructing views, metadata is available about the location of the file in the view's reference branch. Allows incorporating parts of the directory hierarchy in a view. For example `git annex view tag=* podcasts/=*` makes a view in the form tag/showname. * --metadata field=value can now use globs to match, and matches case insensatively, the same as git annex view field=value does. * annex.genmetadata can be set to make git-annex automatically set metadata (year and month) when adding files. * Make annex.web-options be used in several places that call curl. * Fix handling of rsync remote urls containing a username, including rsync.net. * Preserve metadata when staging a new version of an annexed file. * metadata: Support --json * webapp: Fix creation of box.com and Amazon S3 and Glacier repositories, broken in 5.20140221. * webdav: When built with DAV 0.6.0, use the new DAV monad to avoid locking files, which is not needed by git-annex's use of webdav, and does not work on Box.com. * webdav: Fix path separator bug when used on Windows. * repair: Optimise unpacking of pack files, and avoid repeated error messages about corrupt pack files. * Add build dep on regex-compat to fix build on mipsel, which lacks regex-tdfa. * Disable test suite on sparc, which is missing optparse-applicative. * Put non-object tmp files in .git/annex/misctmp, leaving .git/annex/tmp for only partially transferred objects. -- Joey Hess <joeyh@debian.org> Thu, 27 Feb 2014 11:34:19 -0400 git-annex (5.20140221) unstable; urgency=medium * metadata: New command that can attach metadata to files. * --metadata can be used to limit commands to acting on files that have particular metadata. * Preferred content expressions can use metadata=field=value to limit them to acting on files that have particular metadata. * view: New command that creates and checks out a branch that provides a structured view of selected metadata. * vfilter, vadd, vpop, vcycle: New commands for operating within views. * pre-commit: Update metadata when committing changes to locations of annexed files within a view. * Add progress display for transfers to/from external special remotes. * unused: Fix to actually detect unused keys when in direct mode. * fsck: When run with --all or --unused, while .gitattributes annex.numcopies cannot be honored since it's operating on keys instead of files, make it honor the global numcopies setting, and the annex.numcopies git config setting. * trust, untrust, semitrust, dead: Warn when the trust level is overridden in .git/config. * glacier: Do not try to run glacier value create when an existing glacier remote is enabled. * fsck: Refuse to do anything if more than one of --incremental, --more, and --incremental-schedule are given, since it's not clear which option should win. * Windows webapp: Can set up box.com, Amazon S3, and rsync.net remotes * Windows webapp: Can create repos on removable drives. * Windows: Ensure HOME is set, as needed by bundled cygwin utilities. -- Joey Hess <joeyh@debian.org> Fri, 21 Feb 2014 11:23:59 -0400 git-annex (5.20140210) unstable; urgency=medium * --in can now refer to files that were located in a repository at some past date. For example, --in="here@{yesterday}" * Fixed direct mode annexed content locking code, which is used to guard against recursive file drops. * This is the first beta-level release of the Windows port with important fixes (see below). (The webapp and assistant are still alpha-level on Windows.) * sync --content: Honor annex-ignore configuration. * sync: Don't try to sync with xmpp remotes, which are only currently supported when using the assistant. * sync --content: Re-pull from remotes after downloading content, since that can take a while and other changes may be pushed in the meantime. * sync --content: Reuse smart copy code from copy command, including handling and repairing out of date location tracking info. Closes: #737480 * sync --content: Drop files from remotes that don't want them after getting them. * sync: Fix bug in automatic merge conflict resolution code when used on a filesystem not supporting symlinks, which resulted in it losing track of the symlink bit of annexed files. * Added ways to configure rsync options to be used only when uploading or downloading from a remote. Useful to eg limit upload bandwidth. * Fix initremote with encryption=pubkey to work with S3, glacier, webdav, and external special remotes. * Avoid building with DAV 0.6 which is badly broken (see #737902). * Fix dropping of unused keys with spaces in their name. * Fix build on platforms not supporting the webapp. * Document in man page that sshcaching uses ssh ControlMaster. Closes: #737476 * Windows: It's now safe to run multiple git-annex processes concurrently on Windows; the lock files have been sorted out. * Windows: Avoid using unix-compat's rename, which refuses to rename directories. * Windows: Fix deletion of repositories by test suite and webapp. * Windows: Test suite 100% passes again. * Windows: Fix bug in symlink calculation code. * Windows: Fix handling of absolute unix-style git repository paths. * Android: Avoid crashing when unable to set file mode for ssh config file due to Android filesystem horribleness. -- Joey Hess <joeyh@debian.org> Mon, 10 Feb 2014 12:54:57 -0400 git-annex (5.20140127) unstable; urgency=medium * sync --content: New option that makes the content of annexed files be transferred. Similar to the assistant, this honors any configured preferred content expressions. * Remove --json option from commands not supporting it. * status: Support --json. * list: Fix specifying of files to list. * Allow --all to be mixed with matching options like --copies and --in (but not --include and --exclude). * numcopies: New command, sets global numcopies value that is seen by all clones of a repository. * The annex.numcopies git config setting is deprecated. Once the numcopies command is used to set the global number of copies, any annex.numcopies git configs will be ignored. * assistant: Make the prefs page set the global numcopies. * Add lackingcopies, approxlackingcopies, and unused to preferred content expressions. * Client, transfer, incremental backup, and archive repositories now want to get content that does not yet have enough copies. * Client, transfer, and source repositories now do not want to retain unused file contents. * assistant: Checks daily for unused file contents, and when possible moves them to a repository (such as a backup repository) that wants to retain them. * assistant: annex.expireunused can be configured to cause unused file contents to be deleted after some period of time. * webapp: Nudge user to see if they want to expire old unused file contents when a lot of them seem to be piling up in the repository. * repair: Check git version at run time. * assistant: Run the periodic git gc in batch mode. * added annex.secure-erase-command config option. * test suite: Use tasty-rerun, and expose tasty command-line options. * Optimise non-bare http remotes; no longer does a 404 to the wrong url every time before trying the right url. Needs annex-bare to be set to false, which is done when initially probing the uuid of a http remote. * webapp: After upgrading a git repository to git-annex, fix bug that made it temporarily not be synced with. * whereis: Support --all. * All commands that support --all also support a --key option, which limits them to acting on a single key. -- Joey Hess <joeyh@debian.org> Mon, 27 Jan 2014 13:43:28 -0400 git-annex (5.20140117) unstable; urgency=medium * Really fix FTBFS on mipsel and sparc due to test suite not being available on those architectures. -- Joey Hess <joeyh@debian.org> Fri, 17 Jan 2014 14:46:27 -0400 git-annex (5.20140116) unstable; urgency=medium * Added tahoe special remote. * external special remote protocol: Added GETGITDIR, and GETAVAILABILITY. * Refuse to build with git older than 1.7.1.1, which is needed for git checkout -B * map: Fix display of v5 direct mode repos. * repair: Support old git versions from before git fsck --no-dangling was implemented. * Fix a long-standing bug that could cause the wrong index file to be used when committing to the git-annex branch, if GIT_INDEX_FILE is set in the environment. This typically resulted in git-annex branch log files being committed to the master branch and later showing up in the work tree. (These log files can be safely removed.) * assistant: Detect if .git/annex/index is corrupt at startup, and recover. * repair: Fix bug in packed refs file exploding code that caused a .gitrefs directory to be created instead of .git/refs * Fix FTBFS on mipsel and sparc due to test suite not being available on those architectures. * Android: Avoid passing --clobber to busybox wget. -- Joey Hess <joeyh@debian.org> Thu, 16 Jan 2014 11:34:54 -0400 git-annex (5.20140107) unstable; urgency=medium * mirror: Support --all (and --unused). * external special remote protocol: Added GETUUID, GETWANTED, SETWANTED, SETSTATE, GETSTATE, DEBUG. * Windows: Fix bug in direct mode merge code that could cause files in subdirectories to go missing. * Windows: Avoid eating stdin when running ssh to add a authorized key, since this is used for password prompting. * Avoid looping if long-running git cat-file or git hash-object crashes and keeps crashing when restarted. * Assistant: Remove stale MERGE_HEAD files in lockfile cleanup. * Remotes can now be made read-only, by setting remote.<name>.annex-readonly * wanted, schedule: Avoid printing "ok" after requested value. * assistant: Ensure that .ssh/config and .ssh/authorized_keys are not group or world writable when writing to those files, as that can make ssh refuse to use them, if it allows another user to write to them. * addurl, importfeed: Honor annex.diskreserve as long as the size of the url can be checked. * add: Fix rollback when disk is completely full. * assistant: Fixed several minor memory leaks that manifested when adding a large number of files. * assistant: Start a new git-annex transferkeys process after a network connection change, so that remotes that use a persistent network connection are restarted. * Adjust Debian build deps to match current state of sparc, mipsel. -- Joey Hess <joeyh@debian.org> Tue, 07 Jan 2014 12:22:18 -0400 git-annex (5.20131230) unstable; urgency=medium * Added new external special remote interface. * importfeed: Support youtube playlists. * Add tasty to build-depends, so that test suite builds again. (tasty was stuck in incoming.) * Fix typo in test suite. * Fix bug in Linux standalone build's shimming that broke git-annex-shell. * Include git-receive-pack, git-upload-pack, git, and git-shell wrappers in the Linux standalone build, and OSX app, so they will be available when it's added to PATH. * addurl, importfeed: Sanitize | and some other symbols and special characters. * Auto-upgrade v3 indirect repos to v5 with no changes. This also fixes a problem when a direct mode repo was somehow set to v3 rather than v4, and so the automatic direct mode upgrade to v5 was not done. * Android: Avoid trying to use Android's own ionice, which does not allow specifying a command to run. Fixes transferring files to/from android and probably a few other things. -- Joey Hess <joeyh@debian.org> Mon, 30 Dec 2013 14:13:40 -0400 git-annex (5.20131221) unstable; urgency=low * assistant: Fix OSX-specific bug that caused the startup scan to try to follow symlinks to other directories, and add their contents to the annex. * assistant: Set StrictHostKeyChecking yes when creating ssh remotes, and add it to the configuration for any ssh remotes previously created by the assistant. This avoids repeated prompts by ssh if the host key changes, instead syncing with such a remote will fail. Closes: #732602 * Fix test suite to cover lock --force change. * Add plumbing-level lookupkey and examinekey commands. * find --format: Added hashdirlower, hashdirmixed, keyname, and mtime format variables. * assistant: Always batch changes found in startup scan. * An armel Linux standalone build is now available, which includes the webapp. * Programs from Linux and OSX standalone builds can now be symlinked into a directory in PATH as an alternative installation method, and will use readlink to find where the build was unpacked. * Include man pages in Linux and OSX standalone builds. * Linux standalone build now includes its own glibc and forces the linker to use it, to remove dependence on the host glibc. -- Joey Hess <joeyh@debian.org> Sat, 21 Dec 2013 12:00:17 -0400 git-annex (5.20131213) unstable; urgency=low * Avoid using git commit in direct mode, since in some situations it will read the full contents of files in the tree. * assistant: Batch jobs are now run with ionice and nocache, when those commands are available. * assistant: Run transferkeys as batch jobs. * Automatically fix up bad bare repositories created by versions 5.20131118 through 5.20131127. * rsync special remote: Fix fallback mode for rsync remotes that use hashDirMixed. Closes: #731142 * copy --from, get --from: When --force is used, ignore the location log and always try to get the file from the remote. * Deal with box.com changing the url of their webdav endpoint. * Android: Fix SRV record lookups for XMPP to use android getprop command to find DNS server, since there is no resolv.conf. * import: Add --skip-duplicates option. * lock: Require --force. Closes: #731606 * import: better handling of overwriting an existing file/directory/broken link when importing * Windows: assistant and webapp work! (very experimental) * Windows: Support annex.diskreserve. * Fix bad behavior in Firefox, which was caused by an earlier fix to bad behavior in Chromium. * repair: Improve repair of git-annex index file. * repair: Remove damaged git-annex sync branches. * status: Ignore new files that are gitignored. * Fix direct mode's handling when modifications to non-annexed files are pulled from a remote. A bug prevented the files from being updated in the work tree, and this caused the modification to be reverted. * OSX: Remove ssh and ssh-keygen from dmg as they're included in OSX by default. -- Joey Hess <joeyh@debian.org> Fri, 13 Dec 2013 14:20:32 -0400 git-annex (5.20131130) unstable; urgency=low * init: Fix a bug that caused git annex init, when run in a bare repository, to set core.bare=false. -- Joey Hess <joeyh@debian.org> Sat, 30 Nov 2013 16:32:35 -0400 git-annex (5.20131127.1) unstable; urgency=low * Rebuild that does not try to use quvi 0.9 from experimental. -- Joey Hess <joeyh@debian.org> Thu, 28 Nov 2013 07:57:36 -0400 git-annex (5.20131127) unstable; urgency=low * webapp: Detect when upgrades are available, and upgrade if the user desires. (Only when git-annex is installed using the prebuilt binaries from git-annex upstream, not from eg Debian.) * assistant: Detect when the git-annex binary is modified or replaced, and either prompt the user to restart the program, or automatically restart it. * annex.autoupgrade configures both the above upgrade behaviors. * Added support for quvi 0.9. Slightly suboptimal due to limitations in its interface compared with the old version. * Bug fix: annex.version did not get set on automatic upgrade to v5 direct mode repo, so the upgrade was performed repeatedly, slowing commands down. * webapp: Fix bug that broke switching between local repositories that use the new guarded direct mode. * Android: Fix stripping of the git-annex binary. * Android: Make terminal app show git-annex version number. * Android: Re-enable XMPP support. * reinject: Allow to be used in direct mode. * Futher improvements to git repo repair. Has now been tested in tens of thousands of intentionally damaged repos, and successfully repaired them all. * Allow use of --unused in bare repository. -- Joey Hess <joeyh@debian.org> Wed, 27 Nov 2013 18:41:44 -0400 git-annex (5.20131120) unstable; urgency=low * Fix Debian package to not try to run test suite, since haskell-tasty is not out of new or in Build-Depends yet. * dropunused, addunused: Allow "all" instead of a range to act on all unused data. * Ensure execute bit is set on directories when core.sharedrepository is set. * Ensure that core.sharedrepository is honored when creating the .git/annex directory. * Improve repair code in the case where the index file is corrupt, and this hides other problems from git fsck. -- Joey Hess <joeyh@debian.org> Wed, 20 Nov 2013 12:54:18 -0400 git-annex (5.20131118) unstable; urgency=low * Direct mode repositories now have core.bare=true set, to prevent accidentally running git commands that try to operate on the work tree, and so do the wrong thing in direct mode. * annex.version is now set to 5 for direct mode repositories. This upgrade is handled fully automatically, no need to run git annex upgrade * The "status" command has been renamed to "info", to allow "git annex status" to be used in direct mode repositories, now that "git status" won't work in them. * The -c option now not only modifies the git configuration seen by git-annex, but it is passed along to every git command git-annex runs. * watcher: Avoid loop when adding a file owned by someone else fails in indirect mode because its permissions cannot be modified. * webapp: Avoid encoding problems when displaying the daemon log file. * webapp: Improve UI around remote that have no annex.uuid set, either because setup of them is incomplete, or because the remote git repository is not a git-annex repository. * Include ssh-keygen in standalone bundle. * Allow optionally configuring git-annex with -fEKG to enable awesome remote monitoring interfaceat http://localhost:4242/ * Fix bug that caused bad information to be written to the git-annex branch when running describe or other commands with a remote that has no uuid. * Work around Android linker problem that had prevented git-annex from running on Android 4.3 and 4.4. * repair: Handle case where index file is corrupt, but all objects are ok. * assistant: Notice on startup when the index file is corrupt, and auto-repair. * Fix direct mode merge bug when a direct mode file was deleted and replaced with a directory. An ordering problem caused the directory to not get created in this case. Thanks to Tim for the test case. * Direct mode .git/annex/objects directories are no longer left writable, because that allowed writing to symlinks of files that are not present, which followed the link and put bad content in an object location. Thanks to Tim for the test case. * fsck: Fix up .git/annex/object directory permissions. * Switched to the tasty test framework. * Android: Adjust default .gitignore to ignore .thumbnails at any location in the tree, not just at its top. * webapp: Check annex.version. -- Joey Hess <joeyh@debian.org> Mon, 18 Nov 2013 10:45:43 -0400 git-annex (4.20131106) unstable; urgency=low * Improve local pairing behavior when two computers both try to start the pairing process separately. * sync: Work even when the local git repository is new and empty, with no master branch. * gcrypt, bup: Fix bug that prevented using these special remotes with encryption=pubkey. * Fix enabling of gcrypt repository accessed over ssh; git-annex-shell gcryptsetup had a bug that caused it to fail with permission denied. * Fix zombie process that occurred when switching between repository views in the webapp. * map: Work when there are gcrypt remotes. * Fix build w/o webapp. * Fix exception handling bug that could cause .git/annex/index to be used for git commits outside the git-annex branch. Known to affect git-annex when used with the git shipped with Ubuntu 13.10. -- Joey Hess <joeyh@debian.org> Wed, 06 Nov 2013 11:17:47 -0400 git-annex (4.20131101) unstable; urgency=low * The "git annex content" command is renamed to "git annex wanted". * New --want-get and --want-drop options which can be used to test preferred content settings. For example, "git annex find --in . --want-drop" * assistant: When autostarted, wait 5 seconds before running the startup scan, to avoid contending with the user's desktop login process. * webapp: When setting up a bare shared repository, enable non-fast-forward pushes. * sync: Show a hint about receive.denyNonFastForwards when a push fails. * directory, webdav: Fix bug introduced in version 4.20131002 that caused the chunkcount file to not be written. Work around repositories without such a file, so files can still be retreived from them. * assistant: Automatically repair damanged git repository, if it can be done without losing data. * assistant: Support repairing git remotes that are locally accessible (eg, on removable drives). * add: Fix reversion in 4.20130827 when adding unlocked files that have not yet been committed. * unannex: New, much slower, but more safe behavior: Copies files out of the annex. This avoids an unannex of one file breaking other files that link to the same content. Also, it means that the content remains in the annex using up space until cleaned up with "git annex unused". (The behavior of unannex --fast has not changed; it still hard links to content in the annex. --fast was not made the default because it is potentially unsafe; editing such a hard linked file can unexpectedly change content stored in the annex.) -- Joey Hess <joeyh@debian.org> Fri, 01 Nov 2013 11:34:27 -0400 git-annex (4.20131024) unstable; urgency=low * webapp: Fix bug when adding a remote and git-remote-gcrypt is not installed. * The assitant can now run scheduled incremental fsck jobs on the local repository and remotes. These can be configured using vicfg or with the webapp. * repair: New command, which can repair damaged git repositories (even ones not using git-annex). * webapp: When git repository damange is detected, repairs can be done using the webapp UI. * Automatically and safely detect and recover from dangling .git/annex/index.lock files, which would prevent git from committing to the git-annex branch, eg after a crash. * assistant: Detect stale git lock files at startup time, and remove them. * addurl: Better sanitization of generated filenames. * Better sanitization of problem characters when generating URL and WORM keys. * The control socket path passed to ssh needs to be 17 characters shorter than the maximum unix domain socket length, because ssh appends stuff to it to make a temporary filename. Closes: #725512 * status: Fix space leak in local mode, introduced in version 4.20130920. * import: Skip .git directories. * Remove bogus runshell loop check. * addurl: Improve message when adding url with wrong size to existing file. * Fixed handling of URL keys that have no recorded size. * status: Fix a crash if a temp file went away while its size was being checked for status. * Deal with git check-attr -z output format change in git 1.8.5. * Work around sed output difference that led to version containing a newline on OSX. * sync: Fix automatic resolution of merge conflicts where one side is an annexed file, and the other side is a non-annexed file, or a directory. * S3: Try to ensure bucket name is valid for archive.org. * assistant: Bug fix: When run in a subdirectory, files from incoming merges were wrongly added to that subdirectory, and removed from their original locations. * Windows: Deal with strange msysgit 1.8.4 behavior of not understanding DOS formatted paths for --git-dir and --work-tree. * Removed workaround for bug in git 1.8.4r0. * Added git-recover-repository command to git-annex source (not built by default; this needs to move to someplace else). * webapp: Move sidebar to the right hand side of the screen. -- Joey Hess <joeyh@debian.org> Thu, 24 Oct 2013 12:59:55 -0400 git-annex (4.20131002) unstable; urgency=low * Note that the layout of gcrypt repositories has changed, and if you created one you must manually upgrade it. See http://git-annex.branchable.com/upgrades/gcrypt/ * webapp: Support setting up and using encrypted git repositories on any ssh server, as well as on rsync.net. * git-annex-shell: Added support for operating inside gcrypt repositories. * Disable receive.denyNonFastForwards when setting up a gcrypt special remote, since gcrypt needs to be able to fast-forward the master branch. * import: Preserve top-level directory structure. * Use cryptohash rather than SHA for hashing when no external hash program is available. This is a significant speedup for SHA256 on OSX, for example. * Added SKEIN256 and SKEIN512 backends. * Android build redone from scratch, many dependencies updated, and entire build can now be done using provided scripts. * assistant: Clear the list of failed transfers when doing a full transfer scan. This prevents repeated retries to download files that are not available, or are not referenced by the current git tree. * indirect, direct: Better behavior when a file is not owned by the user running the conversion. * add, import, assistant: Better preserve the mtime of symlinks, when when adding content that gets deduplicated. * Send a git-annex user-agent when downloading urls. Overridable with --user-agent option. (Not yet done for S3 or WebDAV due to limitations of libraries used.) * webapp: Fixed a bug where when a new remote is added, one file may fail to sync to or from it due to the transferrer process not yet knowing about the new remote. * OSX: Bundled gpg upgraded, now compatible with config files written by MacGPG. * assistant: More robust inotify handling; avoid crashing if a directory cannot be read. * Moved list of backends and remote types from status to version command. -- Joey Hess <joeyh@debian.org> Wed, 02 Oct 2013 16:00:39 -0400 git-annex (4.20130920) unstable; urgency=low * webapp: Initial support for setting up encrypted removable drives. * Recommend using my patched gcrypt, which fixes some bugs: https://github.com/joeyh/git-remote-gcrypt * Support hot-swapping of removable drives containing gcrypt repositories. * list: New command, displays a compact table of remotes that contain files. (Thanks, anarcat for display code and mastensg for inspiration.) * fsck: Fix detection and fixing of present direct mode files that are wrongly represented as standin symlinks on crippled filesystems. * sync: Fix bug that caused direct mode mappings to not be updated when merging files into the tree on Windows. * sync: Don't fail if the directory it is run in gets removed by the sync. * addurl: Fix quvi audodetection, broken in last release. * status: In local mode, displays information about variance from configured numcopies levels. (--fast avoids calculating these) * gcrypt: Ensure that signing key is set to one of the participants keys. * webapp: Show encryption information when editing a remote. * Avoid unnecessarily catting non-symlink files from git, which can be so large it runs out of memory. -- Joey Hess <joeyh@debian.org> Fri, 20 Sep 2013 10:34:51 -0400 git-annex (4.20130911) unstable; urgency=low * Fix problem with test suite in non-unicode locale. -- Joey Hess <joeyh@debian.org> Wed, 11 Sep 2013 12:14:16 -0400 git-annex (4.20130909) unstable; urgency=low * initremote: Syntax change when setting up an encrypted special remote. Now use keyid=$KEYID rather than the old encryption=$KEYID * forget: New command, causes git-annex branch history to be forgotten in a way that will spread to other clones of the repository. (As long as they're running this version or newer of git-annex.) * forget --drop-dead: Completely removes mentions of repositories that have been marked as dead from the git-annex branch. * sync, assistant: Force push of the git-annex branch. Necessary to ensure it gets pushed to remotes after being rewritten by forget. * Added gcrypt support. This combines a fully encrypted git repository (using git-remote-gcrypt) with an encrypted git-annex special remote. * sync: Support syncing with gcrypt remotes. * importfeed: Also ignore transient problems with downloading content from feeds. * Honor core.sharedrepository when receiving and adding files in direct mode. * enableremote: gpg keys can be removed from those a remote encrypts to by passing "keyid-=$KEYID". keyid+= is also provided. (Thanks, guilhem for the patch.) * Added encryption=pubkey scheme, which encrypts to public keys directly rather than the hybrid approach. See documentation for advantages and disadvantages, but encryption=hybrid is the recommended scheme still. (Thanks, guilhem for the patch.) * Fix Feeds display in build flags. * Remind user when annex-ignore is set for some remotes, if unable to get or drop a file, possibly because it's on an ignored remote. * gpg: Force --no-textmode in case the user has it turned on in config. * webapp: Improve javascript's handling of longpolling connection failures, by reloading the current page in this case. Works around chromium behavior where ajax connections to urls that were already accessed are denied after navigating back to a previous page. * Allow building without quvi support. -- Joey Hess <joeyh@debian.org> Mon, 09 Sep 2013 09:47:02 -0400 git-annex (4.20130827) unstable; urgency=low * Youtube support! (And 53 other video hosts). When quvi is installed, git-annex addurl automatically uses it to detect when an page is a video, and downloads the video file. * web special remote: Also support using quvi, for getting files, or checking if files exist in the web. * unused: Is now a minimum of 30 times faster, and typically many more times than that (when a repository has several branches). (Thanks, guilhem for the patch.) * unused: Fix bugs in two edge cases involving manually staged changes. (Thanks, guilhem for the patch.) * Android: Fix bug in terminal app that caused it to spin using much CPU and battery. This problem was introduced in version 4.20130601. * sync, merge: Bug fix: Don't try to merge into master when in a bare repo. * import: Add options to control handling of duplicate files: --duplicate, --deduplicate, and --clean-duplicates * mirror: New command, makes two repositories contain the same set of files. * Set --clobber when running wget to ensure resuming works properly. * Unescape characters in 'file://...' URIs. (Thanks, guilhem for the patch.) * Better error message when trying to use a git remote that has annex.ignore set. * Fix bug that caused typechanged symlinks to be assumed to be unlocked files, so they were added to the annex by the pre-commit hook. * Debian: Run the builtin test suite as an autopkgtest. * Debian: Recommend ssh-askpass, which ssh will use when the assistant is run w/o a tty. Closes: #719832 -- Joey Hess <joeyh@debian.org> Tue, 27 Aug 2013 11:03:00 -0400 git-annex (4.20130815) unstable; urgency=low * assistant, watcher: .gitignore files and other git ignores are now honored, when git 1.8.4 or newer is installed. (Thanks, Adam Spiers, for getting the necessary support into git for this.) * importfeed: Ignores transient problems with feeds. Only exits nonzero when a feed has repeatedly had a problems for at least 1 day. * importfeed: Fix handling of dots in extensions. * Windows: Added support for encrypted special remotes. * Windows: Fixed permissions problem that prevented removing files from directory special remote. Directory special remotes now fully usable. -- Joey Hess <joeyh@debian.org> Thu, 15 Aug 2013 10:14:33 +0200 git-annex (4.20130802) unstable; urgency=low * dropunused behavior change: Now refuses to drop the last copy of a file, unless you use the --force. This was the last place in git-annex that could remove data referred to by the git history, without being forced. Like drop, dropunused checks remotes, and honors the global annex.numcopies setting. (However, .gitattributes settings cannot apply to unused files.) * Fix inverted logic in last release's fix for data loss bug, that caused git-annex sync on FAT or other crippled filesystems to add symlink standin files to the annex. * importfeed can be used to import files from podcast feeds. * webapp: When setting up a dedicated ssh key to access the annex on a host, set IdentitiesOnly to prevent the ssh-agent from forcing use of a different ssh key. That could result in unnecessary password prompts, or prevent git-annex-shell from being run on the remote host. * webapp: Improve handling of remotes whose setup has stalled. * Add status message to XMPP presence tag, to identify to others that the client is a git-annex client. Closes: #717652 * webapp: When creating a repository on a removable drive, set core.fsyncobjectfiles, to help prevent data loss when the drive is yanked. * Always build with -threaded, to avoid a deadlock when communicating with gpg. * unused: No longer shows as unused tmp files that are actively being transferred. * assistant: Fix NetWatcher to not sync with remotes that have remote.<name>.annex-sync set to false. * assistant: Fix deadlock that could occur when adding a lot of files at once in indirect mode. * assistant: Fix bug that caused it to stall when adding a very large number of files at once (around 5 thousand). * OSX: Make git-annex-webapp run in the background, so that the app icon can be clicked on the open a new webapp when the assistant is already running. * Improve test suite on Windows; now tests git annex sync. * Fix a few bugs involving filenames that are at or near the filesystem's maximum filename length limit. * find: Avoid polluting stdout with progress messages. Closes: #718186 * Escape ':' in file/directory names to avoid it being treated as a pathspec by some git commands. Closes: #718185 * Slow and ugly work around for bug #718517 in git 1.8.4~rc0, which broke git-cat-file --batch for filenames containing spaces. (Will be reverted after next git pre-release fixes the problem.) -- Joey Hess <joeyh@debian.org> Fri, 02 Aug 2013 11:35:16 -0400 git-annex (4.20130723) unstable; urgency=low * Fix data loss bug when adding an (uncompressed) tarball of a git-annex repository, or other file that begins with something that can be mistaken for a git-annex link. Closes: #717456 * New improved version of the git-annex logo, contributed by John Lawrence. * Rsync.net have committed to support git-annex and offer a special discounted rate for git-annex users. Updated the webapp to reflect this. http://www.rsync.net/products/git-annex-pricing.html * Install XDG desktop icon files. * Support unannex and uninit in direct mode. * Support import in direct mode. * webapp: Better display of added files. * fix: Preserve the original mtime of fixed symlinks. * uninit: Preserve .git/annex/objects at the end, if it still has content, so that old versions of files and deleted files are not deleted. Print a message with some suggested actions. * When a transfer is already being run by another process, proceed on to the next file, rather than dying. * Fix checking when content is present in a non-bare repository accessed via http. * Display byte sizes with more precision. * watcher: Fixed a crash that could occur when a directory was renamed or deleted before it could be scanned. * watcher: Partially worked around a bug in hinotify, no longer crashes if hinotify cannot process a directory (but can't detect changes in it) * directory special remote: Fix checking that there is enough disk space to hold an object, was broken when using encryption. * webapp: Differentiate between creating a new S3/Glacier/WebDav remote, and initializing an existing remote. When creating a new remote, avoid conflicts with other existing (or deleted) remotes with the same name. * When an XMPP server has SRV records, try them, but don't then fall back to the regular host if they all fail. * For long hostnames, use a hash of the hostname to generate the socket file for ssh connection caching. -- Joey Hess <joeyh@debian.org> Tue, 23 Jul 2013 10:46:05 -0400 git-annex (4.20130709) unstable; urgency=low * --all: New switch that makes git-annex operate on all data stored in the git annex, including old versions of files. Supported by fsck, get, move, copy. * --unused: New switch that makes git-annex operate on all data found by the last run of git annex unused. Supported by fsck, move, copy. * get, move, copy: Can now be run in a bare repository, like fsck already could. --all is enabled automatically in this case. * merge: Now also merges synced/master or similar branches, which makes it useful to put in a post-receive hook to make a repository automatically update its working copy when git annex sync or the assistant sync with it. * webapp: Fix ssh setup with nonstandard port, broken in last release. * init: Detect systems on which git commit fails due to not being able to determine the FQDN, and put in a workaround so committing to the git-annex branch works. * addurl --pathdepth: Fix failure when the pathdepth specified is deeper than the urls's path. * Windows: Look for .exe extension when searching for a command in path. * Pass -f to curl when downloading a file with it, so it propigates failure. * Windows: Fix url to object when using a http remote. * webapp: Fix authorized_keys line added when setting up a rsync remote on a server that also supports git-annex, to not force running git-annex-shell. * OSX Mountain Lion: Fixed gpg bundled in dmg to not fail due to a missing gpg-agent. * Android: gpg is built without --enable-minimal, so it interoperates better with other gpg builds that may default to using other algorithms for encryption. * dropunused, addunused: Complain when asked to operate on a number that does not correspond to any unused key. * fsck: Don't claim to fix direct mode when run on a symlink whose content is not present. * Make --numcopies override annex.numcopies set in .gitattributes. -- Joey Hess <joeyh@debian.org> Tue, 09 Jul 2013 13:55:39 -0400 git-annex (4.20130627) unstable; urgency=low * assistant --autostart: Automatically ionices the daemons it starts. * assistant: Daily sanity check thread is run niced. * bup: Handle /~/ in bup remote paths. Thanks, Oliver Matthews * fsck: Ensures that direct mode is used for files when it's enabled. * webapp: Fix bug when setting up a remote ssh repo repeatedly on the same server. * webapp: Ensure that ssh keys generated for different directories on a server are always different. * webapp: Fix bug setting up ssh repo if the user enters "~/" at the start of the path. * assistant: Fix bug that prevented adding files written by gnucash, and more generally support adding hard links to files. However, other operations on hard links are still unsupported. * webapp: Fix bug that caused the webapp to hang when built with yesod 1.2. -- Joey Hess <joeyh@debian.org> Thu, 27 Jun 2013 14:21:55 -0400 git-annex (4.20130621) unstable; urgency=low * Supports indirect mode on encfs in paranoia mode, and other filesystems that do not support hard links, but do support symlinks and other POSIX filesystem features. * Android: Add .thumbnails to .gitignore when setting up a camera repository. * Android: Make the "Open webapp" menu item open the just created repository when a new repo is made. * webapp: When the user switches to display a different repository, that repository becomes the default repository to be displayed next time the webapp gets started. * glacier: Better handling of the glacier inventory, which avoids duplicate uploads to the same glacier repository by `git annex copy`. * Direct mode: No longer temporarily remove write permission bit of files when adding them. * sync: Better support for bare git remotes. Now pushes directly to the master branch on such a remote, instead of to synced/master. This makes it easier to clone from a bare git remote that has been populated with git annex sync or by the assistant. * Android: Fix use of cp command to not try to use features present only on build system. * Windows: Fix hang when adding several files at once. * assistant: In direct mode, objects are now only dropped when all associated files are unwanted. This avoids a repreated drop/get loop of a file that has a copy in an archive directory, and a copy not in an archive directory. (Indirect mode still has some buggy behavior in this area, since it does not keep track of associated files.) Closes: #712060 * status: No longer shows dead repositories. * annex.debug can now be set to enable debug logging by default. The webapp's debugging check box does this. * fsck: Avoid getting confused by Windows path separators * Windows: Multiple bug fixes, including fixing the data written to the git-annex branch. * Windows: The test suite now passes on Windows (a few broken parts are disabled). * assistant: On Linux, the expensive transfer scan is run niced. * Enable assistant and WebDAV support on powerpc and sparc architectures, which now have the necessary dependencies built. -- Joey Hess <joeyh@debian.org> Fri, 21 Jun 2013 10:18:41 -0400 git-annex (4.20130601) unstable; urgency=medium * XMPP: Git push over xmpp made much more robust. * XMPP: Avoid redundant and unnecessary pushes. Note that this breaks compatibility with previous versions of git-annex, which will refuse to accept any XMPP pushes from this version. * XMPP: Send pings and use them to detect when contact with the server is lost. * hook special remote: Added combined hook program support. * Android app: Avoid using hard links to app's lib directory, which is sometimes on a different filesystem than the data directory. * Fix bug in parsing of parens in some preferred content expressions. This fixes the behavior of the manual mode group. * assistant: Work around git-cat-file's not reloading the index after files are staged. * Improve error handling when getting uuid of http remotes to auto-ignore, like with ssh remotes. * content: New command line way to view and configure a repository's preferred content settings. * sync: Fix double merge conflict resolution handling. * XMPP: Fix a file descriptor leak. * Android: Added an "Open WebApp" item to the terminal's menu. * Android: Work around Android devices where the `am` command doesn't work. * Can now restart certain long-running git processes if they crash, and continue working. -- Joey Hess <joeyh@debian.org> Sat, 01 Jun 2013 19:16:04 -0400 git-annex (4.20130521) unstable; urgency=low * Sanitize debian changelog version before putting it into cabal file. Closes: #708619 * Switch to MonadCatchIO-transformers for better handling of state while catching exceptions. * Fix a zombie that could result when running a process like gpg to read and write to it. * Allow building with gpg2. * Disable building with the haskell threaded runtime when the webapp is not built. This may fix builds on mips, s390x and sparc, which are failing to link -lHSrts_thr * Temporarily build without webapp on kfreebsd-i386, until yesod is installable there again. * Direct mode bug fix: After a conflicted merge was automatically resolved, the content of a file that was already present could incorrectly be replaced with a symlink. * Fix a bug in the git-annex branch handling code that could cause info from a remote to not be merged and take effect immediately. * Direct mode is now fully tested by the test suite. * Detect bad content in ~/.config/git-annex/program and look in PATH instead. * OSX: Fixed gpg included in dmg. * Linux standalone: Back to being built with glibc 2.13 for maximum portability. -- Joey Hess <joeyh@debian.org> Tue, 21 May 2013 13:10:26 -0400 git-annex (4.20130516) unstable; urgency=low * Android: The webapp is ported and working. * Windows: There is a very rough Windows port. Do not trust it with important data. * git-annex-shell: Ensure that received files can be read. Files transferred from some Android devices may have very broken permissions as received. * direct mode: Direct mode commands now work on files staged in the index, they do not need to be committed to git. * Temporarily add an upper bound to the version of yesod that can be built with, since yesod 1.2 has a great many changes that will require extensive work on the webapp. * Disable building with the haskell threaded runtime when the assistant is not built. This may fix builds on s390x and sparc, which are failing to link -lHSrts_thr * Avoid depending on regex-tdfa on mips, mipsel, and s390, where it fails to build. * direct: Fix a bug that could cause some files to be left in indirect mode. * When initializing a directory special remote with a relative path, the path is made absolute. * SHA: Add a runtime sanity check that sha commands output something that appears to be a real sha. * configure: Better checking that sha commands output in the desired format. * rsync special remotes: When sending from a crippled filesystem, use the destination's default file permissions, as the local ones can be arbitrarily broken. (Ie, ----rwxr-x for files on Android) * migrate: Detect if a file gets corrupted while it's being migrated. * Debian: Add a menu file. -- Joey Hess <joeyh@debian.org> Thu, 16 May 2013 11:03:35 -0400 git-annex (4.20130501) unstable; urgency=low * sync, assistant: Behavior changes: Sync with remotes that have annex-ignore set, so that git remotes on servers without git-annex installed can be used to keep clients' git repos in sync. * assistant: Work around misfeature in git 1.8.2 that makes `git commit --alow-empty -m ""` run an editor. * sync: Bug fix, avoid adding to the annex the dummy symlinks used on crippled filesystems. * Add public repository group. (And inpreferreddir to preferred content expressions.) * webapp: Can now set up Internet Archive repositories. * S3: Dropping content from the Internet Archive doesn't work, but their API indicates it does. Always refuse to drop from there. * Automatically register public urls for files uploaded to the Internet Archive. * To enable an existing special remote, the new enableremote command must be used. The initremote command now is used only to create new special remotes. * initremote: If two existing remotes have the same name, prefer the one with a higher trust level. * assistant: Improved XMPP protocol to better support multiple repositories using the same XMPP account. Fixes bad behavior when sharing with a friend when you or the friend have multiple reposotories on an XMPP account. Note that XMPP pairing with your own devices still pairs with all repositories using your XMPP account. * assistant: Fix bug that could cause incoming pushes to not get merged into the local tree. Particularly affected XMPP pushes. * webapp: Display some additional information about a repository on its edit page. * webapp: Install FDO desktop menu file when started in standalone mode. * webapp: Don't default to making repository in cwd when started from within a directory containing a git-annex file (eg, standalone tarball directory). * Detect systems that have no user name set in GECOS, and also don't have user.name set in git config, and put in a workaround so that commits to the git-annex branch (and the assistant) will still succeed despite git not liking the system configuration. * webapp: When told to add a git repository on a remote server, and the repository already exists as a non-bare repository, use it, rather than initializing a bare repository in the same directory. * direct, indirect: Refuse to do anything when the assistant or git-annex watch daemon is running. * assistant: When built with git before 1.8.0, use `git remote rm` to delete a remote. Newer git uses `git remote remove`. * rmurl: New command, removes one of the recorded urls for a file. * Detect when the remote is broken like bitbucket is, and exits 0 when it fails to run git-annex-shell. * assistant: Several improvements to performance and behavior when performing bulk adds of a large number of files (tens to hundreds of thousands). * assistant: Sanitize XMPP presence information logged for debugging. * webapp: Now automatically fills in any creds used by an existing remote when creating a new remote of the same type. Done for Internet Archive, S3, Glacier, and Box.com remotes. * Store an annex-uuid file in the bucket when setting up a new S3 remote. * Support building with DAV 0.4. -- Joey Hess <joeyh@debian.org> Wed, 01 May 2013 01:42:46 -0400 git-annex (4.20130417) unstable; urgency=low * initremote: Generates encryption keys with high quality entropy. This can be disabled using --fast to get the old behavior. The assistant still uses low-quality entropy when creating encrypted remotes, to avoid delays. (Thanks, guilhem for the patch.) * Bugfix: Direct mode no longer repeatedly checksums duplicated files. * assistant: Work around horrible, terrible, very bad behavior of gnome-keyring, by not storing special-purpose ssh keys in ~/.ssh/*.pub. Apparently gnome-keyring apparently will load and indiscriminately use such keys in some cases, even if they are not using any of the standard ssh key names. Instead store the keys in ~/.ssh/annex/, which gnome-keyring will not check. * addurl: Bugfix: Did not properly add file in direct mode. * assistant: Bug fix to avoid annexing the files that git uses to stand in for symlinks on FAT and other filesystem not supporting symlinks. * Adjust preferred content expressions so that content in archive directories is preferred until it has reached an archive or smallarchive repository. * webapp: New --listen= option allows running the webapp on one computer and connecting to it from another. (Note: Does not yet use HTTPS.) * Added annex.web-download-command setting. * Added per-remote annex-rsync-transport option. (guilhem again) * Ssh connection caching is now also used by rsync special remotes. (guilhem yet again) * The version number is now derived from git, unless built with VERSION_FROM_CHANGELOG. * assistant: Stop any transfers the assistant initiated on shutdown. * assistant: Added sequence numbers to XMPP git push packets. (Not yet used.) * addurl: Register transfer so the webapp can see it. * addurl: Automatically retry downloads that fail, as long as some additional content was downloaded. * webapp: Much improved progress bar display for downloads from encrypted remotes. * Avoid using runghc, as that needs ghci. * webapp: When a repository's group is changed, rescan for transfers. * webapp: Added animations. * webapp: Include the repository directory in the mangled hostname and ssh key name, so that a locked down ssh key for one repository is not re-used when setting up additional repositories on the same server. * Fall back to internal url downloader when built without curl. * fsck: Check content of direct mode files (only when the inode cache thinks they are unmodified). -- Joey Hess <joeyh@debian.org> Wed, 17 Apr 2013 09:07:38 -0400 git-annex (4.20130405) unstable; urgency=low * Group subcommands into sections in usage. Closes: #703797 * Per-command usage messages. * webapp: Fix a race that sometimes caused alerts or other notifications to be missed if they occurred while a page was loading. * webapp: Progess bar fixes for many types of special remotes. * Build debian package without using cabal, which writes to HOME. Closes: #704205 * webapp: Run ssh server probes in a way that will work when the login shell is a monstrosity that should have died 25 years ago, such as csh. * New annex.largefiles setting, which configures which files `git annex add` and the assistant add to the annex. * assistant: Check small files into git directly. * Remotes can be configured to use other MAC algorithms than HMACSHA1 to encrypt filenames. Thanks, guilhem for the patch. * git-annex-shell: Passes rsync --bwlimit options on rsync. Thanks, guilhem for the patch. * webapp: Added UI to delete repositories. Closes: #689847 * Adjust built-in preferred content expressions to make most types of repositories want content that is only located on untrusted, dead, and unwanted repositories. * drop --auto: Fix bug that prevented dropping files from untrusted repositories. * assistant: Fix bug that could cause direct mode files to be unstaged from git. * Update working tree files fully atomically. * webapp: Improved transfer queue management. * init: Probe whether the filesystem supports fifos, and if not, disable ssh connection caching. * Use lower case hash directories for storing files on crippled filesystems, same as is already done for bare repositories. -- Joey Hess <joeyh@debian.org> Fri, 05 Apr 2013 10:42:18 -0400 git-annex (4.20130323) unstable; urgency=low * webapp: Repository list is now included in the dashboard, and other UI tweaks. * webapp: Improved UI for pairing your own devices together using XMPP. * webapp: Display an alert when there are XMPP remotes, and a cloud transfer repository needs to be configured. * Add incrementalbackup repository group. * webapp: Encourage user to install git-annex on a server when adding a ssh server, rather than just funneling them through to rsync. * xmpp: --debug now enables a sanitized dump of the XMPP protocol * xmpp: Try harder to detect presence of clients when there's a git push to send. * xmpp: Re-enable XA flag, since disabling it did not turn out to help with the problems Google Talk has with not always sending presence messages to clients. * map: Combine duplicate repositories, for a nicer looking map. * Fix several bugs caused by a bad Ord instance for Remote. * webapp: Switch all forms to POST. * assistant: Avoid syncing with annex-ignored remotes when reconnecting to the network, or connecting a drive. * assistant: Fix OSX bug that prevented committing changed files to a repository when in indirect mode. * webapp: Improved alerts displayed when syncing with remotes, and when syncing with a remote fails. * webapp: Force wrap long filenames in transfer display. * assistant: The ConfigMonitor left one zombie behind each time it checked for changes, now fixed. * get, copy, move: Display an error message when an identical transfer is already in progress, rather than failing with no indication why. * assistant: Several optimisations to file transfers. * OSX app and standalone Linux tarball now both support being added to PATH; no need to use runshell to start git-annex. * webapp: When adding a removable drive, you can now specify the directory inside it to use. * webapp: Confirm whether user wants to combine repositories when adding a removable drive that already has a repository on it. -- Joey Hess <joeyh@debian.org> Fri, 22 Mar 2013 18:54:05 -0400 git-annex (4.20130314) unstable; urgency=low * Bugfix: git annex add, when ran without any file or directory specified, should add files in the current directory, but not act on unlocked files elsewhere in the tree. * Bugfix: drop --from an unavailable remote no longer updates the location log, incorrectly, to say the remote does not have the key. * Bugfix: If the UUID of a remote is not known, prevent --from, --to, and other ways of specifying remotes by name from selecting it, since it is not possible to sanely use it. * Bugfix: Fix bug in inode cache sentinal check, which broke copying to local repos if the repo being copied from had moved to a different filesystem or otherwise changed all its inodes * Switch from using regex-compat to regex-tdfa, as the C regex library is rather buggy. * status: Can now be run with a directory path to show only the status of that directory, rather than the whole annex. * Added remote.<name>.annex-gnupg-options setting. Thanks, guilhem for the patch. * addurl: Add --relaxed option. * addurl: Escape invalid characters in urls, rather than failing to use an invalid url. * addurl: Properly handle url-escaped characters in file:// urls. * assistant: Fix dropping content when a file is moved to an archive directory, and getting contennt when a file is moved back out. * assistant: Fix bug in direct mode that could occur when a symlink is moved out of an archive directory, and resulted in the file not being set to direct mode when it was transferred. * assistant: Generate better commits for renames. * assistant: Logs are rotated to avoid them using too much disk space. * assistant: Avoid noise in logs from git commit about typechanged files in direct mode repositories. * assistant: Set gc.auto=0 when creating repositories to prevent automatic commits from causing git-gc runs. * assistant: If gc.auto=0, run git-gc once a day, packing loose objects very non-aggressively. * assistant: XMPP git pull and push requests are cached and sent when presence of a new client is detected. * assistant: Sync with all git remotes on startup. * assistant: Get back in sync with XMPP remotes after network reconnection, and on startup. * assistant: Fix syncing after XMPP pairing. * assistant: Optimised handling of renamed files in direct mode, avoiding re-checksumming. * assistant: Detects most renames, including directory renames, and combines all their changes into a single commit. * assistant: Fix ~/.ssh/git-annex-shell wrapper to work when the ssh key does not force a command. * assistant: Be smarter about avoiding unnecessary transfers. * webapp: Work around bug in Warp's slowloris attack prevention code, that caused regular browsers to stall when they reuse a connection after leaving it idle for 30 seconds. (See https://github.com/yesodweb/wai/issues/146) * webapp: New preferences page allows enabling/disabling debug logging at runtime, as well as configuring numcopies and diskreserve. * webapp: Repository costs can be configured by dragging repositories around in the repository list. * webapp: Proceed automatically on from "Configure jabber account" to pairing. * webapp: Only show up to 10 queued transfers. * webapp: DTRT when told to create a git repo that already exists. * webapp: Set locally paired repositories to a lower cost than other network remotes. * Run ssh with -T to avoid tty allocation and any login scripts that may do undesired things with it. * Several improvements to Makefile and cabal file. Thanks, Peter Simmons * Stop depending on testpack. * Android: Enable test suite. -- Joey Hess <joeyh@debian.org> Thu, 14 Mar 2013 15:29:20 -0400 git-annex (4.20130227) unstable; urgency=low * annex.version is now set to 4 for direct mode repositories. * Should now fully support git repositories with core.symlinks=false; always using git's pseudosymlink files in such repositories. * webapp: Allow creating repositories on filesystems that lack support for symlinks. * webapp: Can now add a new local repository, and make it sync with the main local repository. * Android: Bundle now includes openssh. * Android: Support ssh connection caching. * Android: Assistant is fully working. (But no webapp yet.) * Direct mode: Support filesystems like FAT which can change their inodes each time they are mounted. * Direct mode: Fix support for adding a modified file. * Avoid passing -p to rsync, to interoperate with crippled filesystems. Closes: #700282 * Additional GIT_DIR support bugfixes. May actually work now. * webapp: Display any error message from git init if it fails to create a repository. * Fix a reversion in matching globs introduced in the last release, where "*" did not match files inside subdirectories. No longer uses the Glob library. * copy: Update location log when no copy was performed, if the location log was out of date. * Makefile now builds using cabal, taking advantage of cabal's automatic detection of appropriate build flags. * test: The test suite is now built into the git-annex binary, and can be run at any time. -- Joey Hess <joeyh@debian.org> Wed, 27 Feb 2013 14:07:24 -0400 git-annex (3.20130216) unstable; urgency=low * Now uses the Haskell uuid library, rather than needing a uuid program. * Now uses the Haskell Glob library, rather than pcre-light, avoiding the need to install libpcre. Currently done only for Cabal or when the Makefile is made to use -DWITH_GLOB * Android port now available (command-line only). * New annex.crippledfilesystem setting, allows use of git-annex repositories on FAT and even worse filesystems; avoiding use of hard links and locked down permissions settings. (Support is incomplete.) * init: Detect when the repository is on a filesystem that does not support hard links, or symlinks, or unix permissions, and set annex.crippledfilesystem, as well as annex.direct. * add: Improved detection of files that are modified while being added. * Fix a bug in direct mode, introduced in the previous release, where if a file was dropped and then got back, it would be stored in indirect mode. -- Joey Hess <joeyh@debian.org> Sat, 16 Feb 2013 10:03:26 -0400 git-annex (3.20130207) unstable; urgency=low * webapp: Now allows restarting any threads that crash. * Adjust debian package to only build-depend on DAV on architectures where it is available. * addurl --fast: Use curl, rather than haskell HTTP library, to support https. * annex.autocommit: New setting, can be used to disable autocommit of changed files by the assistant, while it still does data syncing and other tasks. * assistant: Ignore .DS_Store on OSX. * assistant: Fix location log when adding new file in direct mode. * Deal with stale mappings for deleted file in direct mode. * pre-commit: Update direct mode mappings. * uninit, unannex --fast: If hard link creation fails, fall back to slow mode. * Clean up direct mode cache and mapping info when dropping keys. * dropunused: Clean up stale direct mode cache and mapping info not removed before. -- Joey Hess <joeyh@debian.org> Thu, 07 Feb 2013 12:45:25 -0400 git-annex (3.20130124) unstable; urgency=low * Added source repository group, that only retains files until they've been transferred to another repository. Useful for things like repositories on cameras. * Added manual repository group. Use to prevent the assistant from downloading any file contents to keep things in sync. Instead `git annex get`, `git annex drop` etc can be used manually as desired. * webapp: More adjustments to longpoll code to deal with changes in variable quoting in different versions of shakespeare-js. * webapp: Avoid an error if a transfer is stopped just as it finishes. Closes: #698184 * webapp: Now always logs to .git/annex/daemon.log * webapp: Has a page to view the log, accessed from the control menu. * webapp: Fix crash adding removable drive that has an annex directory in it that is not a git repository. * Deal with incompatibility in gpg2, which caused prompts for encryption passphrases rather than using the supplied --passphrase-fd. * bugfix: Union merges involving two or more repositories could sometimes result in data from one repository getting lost. This could result in the location log data becoming wrong, and fsck being needed to fix it. * sync: Automatic merge conflict resolution now stages deleted files. * Depend on git 1.7.7.6 for --no-edit. Closes: #698399 * Fix direct mode mapping code to always store direct mode filenames relative to the top of the repository, even when operating inside a subdirectory. * fsck: Detect and fix consistency errors in direct mode mapping files. * Avoid filename encoding errors when writing direct mode mappings. -- Joey Hess <joeyh@debian.org> Tue, 22 Jan 2013 07:11:59 +1100 git-annex (3.20130114) unstable; urgency=low * Now handles the case where a file that's being transferred to a remote is modified in place, which direct mode allows. When this happens, the transfer now fails, rather than allow possibly corrupt data into the remote. * fsck: Better checking of file content in direct mode. * drop: Suggest using git annex move when numcopies prevents dropping a file. * webapp: Repo switcher filters out repos that do not exist any more (or are on a drive that's not mounted). * webapp: Use IP address, rather than localhost, since some systems may have configuration problems or other issues that prevent web browsers from connecting to the right localhost IP for the webapp. * webapp: Adjust longpoll code to work with recent versions of shakespeare-js. * assistant: Support new gvfs dbus names used in Gnome 3.6. * In direct mode, files with the same key are no longer hardlinked, as that would cause a surprising behavior if modifying one, where the other would also change. * webapp: Avoid illegal characters in hostname when creating S3 or Glacier remote. * assistant: Avoid committer crashing if a file is deleted at the wrong instant. -- Joey Hess <joeyh@debian.org> Mon, 14 Jan 2013 15:25:18 -0400 git-annex (3.20130107) unstable; urgency=low * webapp: Add UI to stop and restart assistant. * committer: Fix a file handle leak. * assistant: Make expensive transfer scan work fully in direct mode. * More commands work in direct mode repositories: find, whereis, move, copy, drop, log, fsck, add, addurl. * sync: No longer automatically adds files in direct mode. * assistant: Detect when system is not configured with a user name, and set environment to prevent git from failing. * direct: Avoid hardlinking symlinks that point to the same content when the content is not present. * Fix transferring files to special remotes in direct mode. -- Joey Hess <joeyh@debian.org> Mon, 07 Jan 2013 01:01:41 -0400 git-annex (3.20130102) unstable; urgency=low * direct, indirect: New commands, that switch a repository to and from direct mode. In direct mode, files are accessed directly, rather than via symlinks. Note that direct mode is currently experimental. Many git-annex commands do not work in direct mode. Some git commands can cause data loss when used in direct mode repositories. * assistant: Now uses direct mode by default when setting up a new local repository. * OSX assistant: Uses the FSEvents API to detect file changes. This avoids issues with running out of file descriptors on large trees, as well as allowing detection of modification of files in direct mode. Other BSD systems still use kqueue. * kqueue: Fix bug that made broken symlinks not be noticed. * vicfg: Quote filename. Closes: #696193 * Bugfix: Fixed bug parsing transfer info files, where the newline after the filename was included in it. This was generally benign, but in the assistant, it caused unexpected dropping of preferred content. * Bugfix: Remove leading \ from checksums output by sha*sum commands, when the filename contains \ or a newline. Closes: #696384 * fsck: Still accept checksums with a leading \ as valid, now that above bug is fixed. * SHA*E backends: Exclude non-alphanumeric characters from extensions. * migrate: Remove leading \ in SHA* checksums, and non-alphanumerics from extensions of SHA*E keys. -- Joey Hess <joeyh@debian.org> Wed, 02 Jan 2013 13:21:34 -0400 git-annex (3.20121211) unstable; urgency=low * webapp: Defaults to sharing box.com account info with friends, allowing one-click enabling of the repository. * Fix broken .config/git-annex/program installed by standalone tarball. * assistant: Retrival from glacier now handled. * Include ssh in standalone tarball and OSX app. * watch: Avoid leaving hard links to files behind in .git/annex/tmp if a file is deleted or moved while it's being quarantined in preparation to being added to the annex. * Allow `git annex drop --from web`; of course this does not remove any file from the web, but it does make git-annex remove all urls associated with a file. * webapp: S3 and Glacier forms now have a select list of all currently-supported AWS regions. * webdav: Avoid trying to set props, avoiding incompatibility with livedrive.com. Needs DAV version 0.3. * webapp: Prettify error display. * webapp: Fix bad interaction between required fields and modals. * webapp: Added help buttons and links next to fields that require explanations. * webapp: Encryption can be disabled when setting up remotes. * assistant: Avoid trying to drop content from remotes that don't have it. * assistant: Allow periods in ssh key comments. * get/copy --auto: Transfer data even if it would exceed numcopies, when preferred content settings want it. * drop --auto: Fix dropping content when there are no preferred content settings. * webapp: Allow user to specify the port when setting up a ssh or rsync remote. * assistant: Fix syncing to just created ssh remotes. * Enable WebDAV support in Debian package. Closes: #695532 -- Joey Hess <joeyh@debian.org> Tue, 11 Dec 2012 11:25:03 -0400 git-annex (3.20121127) unstable; urgency=low * Fix dirContentsRecursive, which had missed some files in deeply nested subdirectories. Could affect various parts of git-annex. * rsync: Fix bug introduced in last release that broke encrypted rsync special remotes. * The standalone builds now unset their special path and library path variables before running the system web browser. -- Joey Hess <joeyh@debian.org> Tue, 27 Nov 2012 17:07:32 -0400 git-annex (3.20121126) unstable; urgency=low * New webdav and Amazon glacier special remotes. * Display a warning when a non-existing file or directory is specified. * webapp: Added configurator for Box.com. * webapp: Show error messages to user when testing XMPP creds. * Fix build of assistant without yesod. * webapp: The list of repositiories refreshes when new repositories are added, including when new repository configurations are pushed in from remotes. * OSX: Fix RunAtLoad value in plist file. * Getting a file from chunked directory special remotes no longer buffers it all in memory. * S3: Added progress display for uploading and downloading. * directory special remote: Made more efficient and robust. * Bugfix: directory special remote could loop forever storing a key when a too small chunksize was configured. * Allow controlling whether login credentials for S3 and webdav are committed to the repository, by setting embedcreds=yes|no when running initremote. * Added smallarchive repository group, that only archives files that are in archive directories. Used by default for glacier when set up in the webapp. * assistant: Fixed handling of toplevel archive directory and client repository group. * assistant: Apply preferred content settings when a new symlink is created, or a symlink gets renamed. Made archive directories work. -- Joey Hess <joeyh@debian.org> Mon, 26 Nov 2012 11:37:49 -0400 git-annex (3.20121112) unstable; urgency=low * assistant: Can use XMPP to notify other nodes about pushes made to other repositories, as well as pushing to them directly over XMPP. * wepapp: Added an XMPP configuration interface. * webapp: Supports pairing over XMPP, with both friends, and other repos using the same account. * assistant: Drops non-preferred content when possible. * assistant: Notices, and applies config changes as they are made to the git-annex branch, including config changes pushed in from remotes. * git-annex-shell: GIT_ANNEX_SHELL_DIRECTORY can be set to limit it to operating on a specified directory. * webapp: When setting up authorized_keys, use GIT_ANNEX_SHELL_DIRECTORY. * Preferred content path matching bugfix. * Preferred content expressions cannot use "in=". * Preferred content expressions can use "present". * Fix handling of GIT_DIR when it refers to a git submodule. * Depend on and use the Haskell SafeSemaphore library, which provides exception-safe versions of SampleVar and QSemN. Thanks, Ben Gamari for an excellent patch set. * file:/// URLs can now be used with the web special remote. * webapp: Allow dashes in ssh key comments when pairing. * uninit: Check and abort if there are symlinks to annexed content that are not checked into git. * webapp: Switched to using the same multicast IP address that avahi uses. * bup: Don't pass - to bup-split to make it read stdin; bup 0.25 does not accept that. * bugfix: Don't fail transferring content from read-only repos. Closes: #691341 * configure: Check that checksum programs produce correct checksums. * Re-enable dbus, using a new version of the library that fixes the memory leak. * NetWatcher: When dbus connection is lost, try to reconnect. * Use USER and HOME environment when set, and only fall back to getpwent, which doesn't work with LDAP or NIS. * rsync special remote: Include annex-rsync-options when running rsync to test a key's presence. * The standalone tarball's runshell now takes care of installing a ~/.ssh/git-annex-shell wrapper the first time it's run. * webapp: Make an initial, empty commit so there is a master branch * assistant: Fix syncing local drives. * webapp: Fix creation of rsync.net repositories. * webapp: Fix renaming of special remotes. * webapp: Generate better git remote names. * webapp: Ensure that rsync special remotes are enabled using the same name they were originally created using. * Bugfix: Fix hang in webapp when setting up a ssh remote with an absolute path. -- Joey Hess <joeyh@debian.org> Mon, 12 Nov 2012 10:39:47 -0400 git-annex (3.20121017) unstable; urgency=low * Fix zombie cleanup reversion introduced in 3.20121009. * Additional fix to support git submodules. -- Joey Hess <joeyh@debian.org> Tue, 16 Oct 2012 21:10:14 -0400 git-annex (3.20121016) unstable; urgency=low * vicfg: New file format, avoids ambiguity with repos that have the same description, or no description. * Bug fix: A recent change caused git-annex-shell to crash. * Better preferred content expression for transfer repos. * webapp: Repository edit form can now edit the name of a repository. * webapp: Make bare repositories on removable drives, as there is nothing to ensure non-bare repos get updated when syncing. * webapp: Better behavior when pausing syncing to a remote when a transfer scan is running and queueing new transfers for that remote. * The standalone binaries are now built to not use ssh connection caching, in order to work with old versions of ssh. * A relative core.worktree is relative to the gitdir. Now that this is handled correctly, git-annex can be used in git submodules. * Temporarily disable use of dbus, as the haskell dbus library blows up when losing connection, which will need to be fixed upstream. -- Joey Hess <joeyh@debian.org> Tue, 16 Oct 2012 15:25:22 -0400 git-annex (3.20121010) unstable; urgency=low * Renamed --ingroup to --inallgroup. * Standard groups changed to client, transfer, archive, and backup. Each of these has its own standard preferred content setting. * dead: Remove dead repository from all groups. * Avoid unsetting HOME when running certian git commands. Closes: #690193 * test: Fix threaded runtime hang. * Makefile: Avoid building with -threaded if the ghc threaded runtime does not exist. * webapp: Improve wording of intro display. Closes: #689848 * webapp: Repositories can now be configured, to change their description, their group, or even to disable syncing to them. * git config remote.name.annex-sync can be used to control whether a remote gets synced. * Fix a crash when merging files in the git-annex branch that contain invalid utf8. * Automatically detect when a ssh remote does not have git-annex-shell installed, and set annex-ignore. -- Joey Hess <joeyh@debian.org> Fri, 12 Oct 2012 13:45:21 -0400 git-annex (3.20121009) unstable; urgency=low * watch, assistant: It's now safe to git annex unlock files while the watcher is running, as well as modify files checked into git as normal files. Additionally, .gitignore settings are now honored. Closes: #689979 * group, ungroup: New commands to indicate groups of repositories. * webapp: Adds newly created repositories to one of these groups: clients, drives, servers * vicfg: New command, allows editing (or simply viewing) most of the repository configuration settings stored in the git-annex branch. * Added preferred content expressions, configurable using vicfg. * get --auto: If the local repository has preferred content configured, only get that content. * drop --auto: If the repository the content is dropped from has preferred content configured, drop only content that is not preferred. * copy --auto: Only transfer content that the destination repository prefers. * assistant: Now honors preferred content settings when deciding what to transfer. * --copies=group:number can now be used to match files that are present in a specified number of repositories in a group. * Added --smallerthan, --largerthan, and --inall limits. * Only build-depend on libghc-clientsession-dev on arches that will have the webapp. * uninit: Unset annex.version. Closes: #689852 -- Joey Hess <joeyh@debian.org> Tue, 09 Oct 2012 15:13:23 -0400 git-annex (3.20121001) unstable; urgency=low * fsck: Now has an incremental mode. Start a new incremental fsck pass with git annex fsck --incremental. Now the fsck can be interrupted as desired, and resumed with git annex fsck --more. Thanks, Justin Azoff * New --time-limit option, makes long git-annex commands stop after a specified amount of time. * fsck: New --incremental-schedule option which is nice for scheduling eg, monthly incremental fsck runs in cron jobs. * Fix fallback to ~/Desktop when xdg-user-dir is not available. Closes: #688833 * S3: When using a shared cipher, S3 credentials are not stored encrypted in the git repository, as that would allow anyone with access to the repository access to the S3 account. Instead, they're stored in a 600 mode file in the local git repo. * webapp: Avoid crashing when ssh-keygen -F chokes on an invalid known_hosts file. * Always do a system wide installation when DESTDIR is set. Closes: #689052 * The Makefile now builds with the new yesod by default. Systems like Debian that have the old yesod 1.0.1 should set GIT_ANNEX_LOCAL_FEATURES=-DWITH_OLD_YESOD * copy: Avoid updating the location log when no copy is performed. * configure: Test that uuid -m works, falling back to plain uuid if not. * Avoid building the webapp on Debian architectures that do not yet have template haskell and thus yesod. (Should be available for arm soonish I hope). -- Joey Hess <joeyh@debian.org> Mon, 01 Oct 2012 13:56:55 -0400 git-annex (3.20120924) unstable; urgency=low * assistant: New command, a daemon which does everything watch does, as well as automatically syncing file contents between repositories. * webapp: An interface for managing and configuring the assistant. * The default backend used when adding files to the annex is changed from SHA256 to SHA256E, to simplify interoperability with OSX, media players, and various programs that needlessly look at symlink targets. To get old behavior, add a .gitattributes containing: * annex.backend=SHA256 * init: If no description is provided for a new repository, one will automatically be generated, like "joey@gnu:~/foo" * test: Set a lot of git environment variables so testing works in strange environments that normally need git config to set names, etc. Closes: #682351 Thanks, gregor herrmann * Disable ssh connection caching if the path to the control socket would be too long (and use relative path to minimise path to the control socket). * migrate: Check content before generating the new key, to avoid generating a key for corrupt data. * Support repositories created with --separate-git-dir. Closes: #684405 * reinject: When the provided file doesn't match, leave it where it is, rather than moving to .git/annex/bad/ * Avoid crashing on encoding errors in filenames when writing transfer info files and reading from checksum commands. * sync: Pushes the git-annex branch to remote/synced/git-annex, rather than directly to remote/git-annex. * Now supports matching files that are present on a number of remotes with a specified trust level. Example: --copies=trusted:2 Thanks, Nicolas Pouillard -- Joey Hess <joeyh@debian.org> Mon, 24 Sep 2012 13:47:48 -0400 git-annex (3.20120825) unstable; urgency=low * S3: Add fileprefix setting. * Pass --use-agent to gpg when in no tty mode. Thanks, Eskild Hustvedt. * Bugfix: Fix fsck in SHA*E backends, when the key contains composite extensions, as added in 3.20120721. -- Joey Hess <joeyh@debian.org> Sat, 25 Aug 2012 10:00:10 -0400 git-annex (3.20120807) unstable; urgency=low * initremote: Avoid recording remote's description before checking that its config is valid. * unused, status: Avoid crashing when ran in bare repo. * Avoid crashing when "git annex get" fails to download from one location, and falls back to downloading from a second location. -- Joey Hess <joeyh@debian.org> Tue, 07 Aug 2012 13:35:07 -0400 git-annex (3.20120721) unstable; urgency=low * get, move, copy: Now refuse to do anything when the requested file transfer is already in progress by another process. * status: Lists transfers that are currently in progress. * Fix passing --uuid to git-annex-shell. * When shaNsum commands cannot be found, use the Haskell SHA library (already a dependency) to do the checksumming. This may be slower, but avoids portability problems. * Use SHA library for files less than 50 kb in size, at which point it's faster than forking the more optimised external program. * SHAnE backends are now smarter about composite extensions, such as .tar.gz Closes: #680450 * map: Write map.dot to .git/annex, which avoids watch trying to annex it. -- Joey Hess <joeyh@debian.org> Sat, 21 Jul 2012 16:52:48 -0400 git-annex (3.20120629) unstable; urgency=low * cabal: Only try to use inotify on Linux. * Version build dependency on STM, and allow building without it, which disables the watch command. * Avoid ugly failure mode when moving content from a local repository that is not available. * Got rid of the last place that did utf8 decoding. * Accept arbitrarily encoded repository filepaths etc when reading git config output. This fixes support for remotes with unusual characters in their names. * sync: Automatically resolves merge conflicts. -- Joey Hess <joeyh@debian.org> Fri, 29 Jun 2012 10:17:49 -0400 git-annex (3.20120624) unstable; urgency=low * watch: New subcommand, a daemon which notices changes to files and automatically annexes new files, etc, so you don't need to manually run git commands when manipulating files. Available on Linux, BSDs, and OSX! * Enable diskfree on kfreebsd, using kqueue. * unused: Fix crash when key names contain invalid utf8. * sync: Avoid recent git's interactive merge. -- Joey Hess <joeyh@debian.org> Sun, 24 Jun 2012 12:36:50 -0400 git-annex (3.20120614) unstable; urgency=medium * addurl: Was broken by a typo introduced 2 released ago, now fixed. Closes: #677576 * Install man page when run by cabal, in a location where man will find it, even when installing under $HOME. Thanks, Nathan Collins -- Joey Hess <joeyh@debian.org> Thu, 14 Jun 2012 20:21:29 -0400 git-annex (3.20120611) unstable; urgency=medium * add: Prevent (most) modifications from being made to a file while it is being added to the annex. * initremote: Automatically describe a remote when creating it. * uninit: Refuse to run in a subdirectory. Closes: #677076 -- Joey Hess <joeyh@debian.org> Mon, 11 Jun 2012 10:32:01 -0400 git-annex (3.20120605) unstable; urgency=low * sync: Show a nicer message if a user tries to sync to a special remote. * lock: Reset unlocked file to index, rather than to branch head. * import: New subcommand, pulls files from a directory outside the annex and adds them. * Fix display of warning message when encountering a file that uses an unsupported backend. * Require that the SHA256 backend can be used when building, since it's the default. * Preserve parent environment when running hooks of the hook special remote. -- Joey Hess <joeyh@debian.org> Tue, 05 Jun 2012 14:03:39 -0400 git-annex (3.20120522) unstable; urgency=low * Pass -a to cp even when it supports --reflink=auto, to preserve permissions. * Clean up handling of git directory and git worktree. * Add support for core.worktree, and fix support for GIT_WORK_TREE and GIT_DIR. -- Joey Hess <joeyh@debian.org> Tue, 22 May 2012 11:16:13 -0400 git-annex (3.20120511) unstable; urgency=low * Rsync special remotes can be configured with shellescape=no to avoid shell quoting that is normally done when using rsync over ssh. This is known to be needed for certian rsync hosting providers (specificially hidrive.strato.com) that use rsync over ssh but do not pass it through the shell. * dropunused: Allow specifying ranges to drop. * addunused: New command, the opposite of dropunused, it relinks unused content into the git repository. * Fix use of several config settings: annex.ssh-options, annex.rsync-options, annex.bup-split-options. (And adjust types to avoid the bugs that broke several config settings.) -- Joey Hess <joeyh@debian.org> Fri, 11 May 2012 12:29:30 -0400 git-annex (3.20120430) unstable; urgency=low * Fix use of annex.diskreserve config setting. * Directory special remotes now check annex.diskreserve. * Support git's core.sharedRepository configuration. * Add annex.http-headers and annex.http-headers-command config settings, to allow custom headers to be sent with all HTTP requests. (Requested by the Internet Archive) * uninit: Clear annex.uuid from .git/config. Closes: #670639 * Added shared cipher mode to encryptable special remotes. This option avoids gpg key distribution, at the expense of flexibility, and with the requirement that all clones of the git repository be equally trusted. -- Joey Hess <joeyh@debian.org> Mon, 30 Apr 2012 13:16:10 -0400 git-annex (3.20120418) unstable; urgency=low * bugfix: Adding a dotfile also caused all non-dotfiles to be added. * bup: Properly handle key names with spaces or other things that are not legal git refs. * git-annex (but not git-annex-shell) supports the git help.autocorrect configuration setting, doing fuzzy matching using the restricted Damerau-Levenshtein edit distance, just as git does. This adds a build dependency on the haskell edit-distance library. * Renamed diskfree.c to avoid OSX case insensativity bug. * cabal now installs git-annex-shell as a symlink to git-annex. * cabal file now autodetects whether S3 support is available. -- Joey Hess <joeyh@debian.org> Wed, 18 Apr 2012 12:11:32 -0400 git-annex (3.20120406) unstable; urgency=low * Disable diskfree on kfreebsd, as I have a build failure on kfreebsd-i386 that is quite likely caused by it. -- Joey Hess <joeyh@debian.org> Sat, 07 Apr 2012 15:50:36 -0400 git-annex (3.20120405) unstable; urgency=low * Rewrote free disk space checking code, moving the portability handling into a small C library. * status: Display amount of free disk space. -- Joey Hess <joeyh@debian.org> Thu, 05 Apr 2012 16:19:10 -0400 git-annex (3.20120315) unstable; urgency=low * fsck: Fix up any broken links and misplaced content caused by the directory hash calculation bug fixed in the last release. * sync: Sync to lower cost remotes first. * status: Fixed to run in constant space. * status: More accurate display of sizes of tmp and bad keys. * unused: Now uses a bloom filter, and runs in constant space. Use of a bloom filter does mean it will not notice a small number of unused keys. For repos with up to half a million keys, it will miss one key in 1000. * Added annex.bloomcapacity and annex.bloomaccuracy, which can be adjusted as desired to tune the bloom filter. * status: Display amount of memory used by bloom filter, and detect when it's too small for the number of keys in a repository. * git-annex-shell: Runs hooks/annex-content after content is received or dropped. * Work around a bug in rsync (IMHO) introduced by openSUSE's SIP patch. * git-annex now behaves as git-annex-shell if symlinked to and run by that name. The Makefile sets this up, saving some 8 mb of installed size. * git-union-merge is a demo program, so it is no longer built by default. -- Joey Hess <joeyh@debian.org> Thu, 15 Mar 2012 11:05:28 -0400 git-annex (3.20120309) unstable; urgency=low * Fix key directory hash calculation code to behave as it did before version 3.20120227 when a key contains non-ascii characters (only WORM backend is likely to have been affected). -- Joey Hess <joeyh@debian.org> Fri, 09 Mar 2012 20:05:09 -0400 git-annex (3.20120230) unstable; urgency=low * "here" can be used to refer to the current repository, which can read better than the old "." (which still works too). * Directory special remotes now support chunking files written to them, avoiding writing files larger than a specified size. * Add progress bar display to the directory special remote. * Add configurable hooks that are run when git-annex starts and stops using a remote: remote.name.annex-start-command and remote.name.annex-stop-command * Fix a bug in symlink calculation code, that triggered in rare cases where an annexed file is in a subdirectory that nearly matched to the .git/annex/object/xx/yy subdirectories. -- Joey Hess <joeyh@debian.org> Mon, 05 Mar 2012 13:38:13 -0400 git-annex (3.20120229) unstable; urgency=low * Fix test suite to not require a unicode locale. * Fix cabal build failure. Thanks, Sergei Trofimovich -- Joey Hess <joeyh@debian.org> Wed, 29 Feb 2012 02:31:31 -0400 git-annex (3.20120227) unstable; urgency=low * Modifications to support ghc 7.4's handling of filenames. This version can only be built with ghc 7.4 or newer. See the ghc7.0 branch for older ghcs. * S3: Fix irrefutable pattern failure when accessing encrypted S3 credentials. * Use the haskell IfElse library. * Fix teardown of stale cached ssh connections. * Fixed to use the strict state monad, to avoid leaking all kinds of memory due to lazy state update thunks when adding/fixing many files. * Fixed some memory leaks that occurred when committing journal files. * Added a annex.queuesize setting, useful when adding hundreds of thousands of files on a system with plenty of memory. * whereis: Prints the urls of files that the web special remote knows about. * addurl --fast: Verifies that the url can be downloaded (only getting its head), and records the size in the key. * When checking that an url has a key, verify that the Content-Length, if available, matches the size of the key. * addurl: Added a --file option, which can be used to specify what file the url is added to. This can be used to override the default filename that is used when adding an url, which is based on the url. Or, when the file already exists, the url is recorded as another location of the file. * addurl: Normalize badly encoded urls. * addurl: Add --pathdepth option. * rekey: New plumbing level command, can be used to change the keys used for files en masse. * Store web special remote url info in a more efficient location. (Urls stored with this version will not be visible to older versions.) * Deal with NFS problem that caused a failure to remove a directory when removing content from the annex. * Make a single location log commit after a remote has received or dropped files. Uses a new "git-annex-shell commit" command when available. * To avoid commits of data to the git-annex branch after each command is run, set annex.alwayscommit=false. Its data will then be committed less frequently, when a merge or sync is done. * configure: Check if ssh connection caching is supported by the installed version of ssh and default annex.sshcaching accordingly. * move --from, copy --from: Now 10 times faster when scanning to find files in a remote on a local disk; rather than go through the location log to see which files are present on the remote, it simply looks at the disk contents directly. -- Joey Hess <joeyh@debian.org> Mon, 27 Feb 2012 12:58:21 -0400 git-annex (3.20120123) unstable; urgency=low * fsck --from: Fscking a remote is now supported. It's done by retrieving the contents of the specified files from the remote, and checking them, so can be an expensive operation. Still, if the remote is a special remote, or a git repository that you cannot run fsck in locally, it's nice to have the ability to fsck it. * If you have any directory special remotes, now would be a good time to fsck them, in case you were hit by the data loss bug fixed in the previous release! * fsck --from remote --fast: Avoids expensive file transfers, at the expense of not checking file size and/or contents. * Ssh connection caching is now enabled automatically by git-annex. Only one ssh connection is made to each host per git-annex run, which can speed some things up a lot, as well as avoiding repeated password prompts. Concurrent git-annex processes also share ssh connections. Cached ssh connections are shut down when git-annex exits. * To disable the ssh caching (if for example you have your own broader ssh caching configuration), set annex.sshcaching=false. -- Joey Hess <joeyh@debian.org> Mon, 23 Jan 2012 13:48:48 -0400 git-annex (3.20120116) unstable; urgency=medium * Fix data loss bug in directory special remote, when moving a file to the remote failed, and partially transferred content was left behind in the directory, re-running the same move would think it succeeded and delete the local copy. -- Joey Hess <joeyh@debian.org> Mon, 16 Jan 2012 16:43:45 -0400 git-annex (3.20120115) unstable; urgency=low * Add a sanity check for bad StatFS results. On architectures where StatFS does not currently work (s390, mips, powerpc, sparc), this disables the diskreserve checking code, and attempting to configure an annex.diskreserve will result in an error. * Fix QuickCheck dependency in cabal file. * Minor optimisations. -- Joey Hess <joeyh@debian.org> Sun, 15 Jan 2012 13:54:20 -0400 git-annex (3.20120113) unstable; urgency=low * log: Add --gource mode, which generates output usable by gource. * map: Fix display of remote repos * Add annex-trustlevel configuration settings, which can be used to override the trust level of a remote. * git-annex, git-union-merge: Support GIT_DIR and GIT_WORK_TREE. * Add libghc-testpack-dev to build depends on all arches. -- Joey Hess <joeyh@debian.org> Fri, 13 Jan 2012 15:35:17 -0400 git-annex (3.20120106) unstable; urgency=low * Support unescaped repository urls, like git does. * log: New command that displays the location log for files, showing each repository they were added to and removed from. * Fix overbroad gpg --no-tty fix from last release. -- Joey Hess <joeyh@debian.org> Sat, 07 Jan 2012 13:16:23 -0400 git-annex (3.20120105) unstable; urgency=low * Added annex-web-options configuration settings, which can be used to provide parameters to whichever of wget or curl git-annex uses (depends on which is available, but most of their important options suitable for use here are the same). * Dotfiles, and files inside dotdirs are not added by "git annex add" unless the dotfile or directory is explicitly listed. So "git annex add ." will add all untracked files in the current directory except for those in dotdirs. * Added quickcheck to build dependencies, and fail if test suite cannot be built. * fsck: Do backend-specific check before checking numcopies is satisfied. * Run gpg with --no-tty. Closes: #654721 -- Joey Hess <joeyh@debian.org> Thu, 05 Jan 2012 13:44:12 -0400 git-annex (3.20111231) unstable; urgency=low * sync: Improved to work well without a central bare repository. Thanks to Joachim Breitner. * Rather than manually committing, pushing, pulling, merging, and git annex merging, we encourage you to give "git annex sync" a try. * sync --fast: Selects some of the remotes with the lowest annex.cost and syncs those, in addition to any specified at the command line. * Union merge now finds the least expensive way to represent the merge. * reinject: Add a sanity check for using an annexed file as the source file. * Properly handle multiline git config values. * Fix the hook special remote, which bitrotted a while ago. * map: --fast disables use of dot to display map * Test suite improvements. Current top-level test coverage: 75% * Improve deletion of files from rsync special remotes. Closes: #652849 * Add --include, which is the same as --not --exclude. * Format strings can be specified using the new --format option, to control what is output by git annex find. * Support git annex find --json * Fixed behavior when multiple insteadOf configs are provided for the same url base. * Can now be built with older git versions (before 1.7.7); the resulting binary should only be used with old git. * Updated to build with monad-control 0.3. -- Joey Hess <joeyh@debian.org> Sat, 31 Dec 2011 14:55:29 -0400 git-annex (3.20111211) unstable; urgency=medium * Fix bug in last version in getting contents from bare repositories. * Ensure that git-annex branch changes are merged into git-annex's index, which fixes a bug that could cause changes that were pushed to the git-annex branch to get reverted. As a side effect, it's now safe for users to check out and commit changes directly to the git-annex branch. * map: Fix a failure to detect a loop when both repositories are local and refer to each other with relative paths. * Prevent key names from containing newlines. * add: If interrupted, add can leave files converted to symlinks but not yet added to git. Running the add again will now clean up this situtation. * Fix caching of decrypted ciphers, which failed when drop had to check multiple different encrypted special remotes. * unannex: Can be run on files that have been added to the annex, but not yet committed. * sync: New command that synchronises the local repository and default remote, by running git commit, pull, and push for you. * Version monad-control dependency in cabal file. -- Joey Hess <joeyh@debian.org> Sun, 11 Dec 2011 21:24:39 -0400 git-annex (3.20111203) unstable; urgency=low * The VFAT filesystem on recent versions of Linux, when mounted with shortname=mixed, does not get along well with git-annex's mixed case .git/annex/objects hash directories. To avoid this problem, new content is now stored in all-lowercase hash directories. Except for non-bare repositories which would be a pain to transition and cannot be put on FAT. (Old mixed-case hash directories are still tried for backwards compatibility.) * Flush json output, avoiding a buffering problem that could result in doubled output. * Avoid needing haskell98 and other fixes for new ghc. Thanks, Mark Wright. * Bugfix: dropunused did not drop keys with two spaces in their name. * Support for storing .git/annex on a different device than the rest of the git repository. * --inbackend can be used to make git-annex only operate on files whose content is stored using a specified key-value backend. * dead: A command which says that a repository is gone for good and you don't want git-annex to mention it again. -- Joey Hess <joeyh@debian.org> Sat, 03 Dec 2011 21:01:45 -0400 git-annex (3.20111122) unstable; urgency=low * merge: Improve commit messages to mention what was merged. * Avoid doing auto-merging in commands that don't need fully current information from the git-annex branch. In particular, git annex add no longer needs to auto-merge. * init: When run in an already initalized repository, and without a description specified, don't delete the old description. * Optimised union merging; now only runs git cat-file once, and runs in constant space. * status: Now displays trusted, untrusted, and semitrusted repositories separately. * status: Include all special remotes in the list of repositories. * status: Fix --json mode. * status: --fast is back * Fix support for insteadOf url remapping. Closes: #644278 * When not run in a git repository, git-annex can still display a usage message, and "git annex version" even works. * migrate: Don't fall over a stale temp file. * Avoid excessive escaping for rsync special remotes that are not accessed over ssh. * find: Support --print0 -- Joey Hess <joeyh@debian.org> Tue, 22 Nov 2011 14:31:45 -0400 git-annex (3.20111111) unstable; urgency=low * Handle a case where an annexed file is moved into a gitignored directory, by having fix --force add its change. * Avoid cyclic drop problems. * Optimized copy --from and get --from to avoid checking the location log for files that are already present. * Automatically fix up badly formatted uuid.log entries produced by 3.20111105, whenever the uuid.log is changed (ie, by init or describe). * map: Support remotes with /~/ and /~user/ -- Joey Hess <joeyh@debian.org> Fri, 11 Nov 2011 13:44:18 -0400 git-annex (3.20111107) unstable; urgency=low * merge: Use fast-forward merges when possible. Thanks Valentin Haenel for a test case showing how non-fast-forward merges could result in an ongoing pull/merge/push cycle. * Don't try to read config from repos with annex-ignore set. * Bugfix: In the past two releases, git-annex init has written the uuid.log in the wrong format, with the UUID and description flipped. -- Joey Hess <joeyh@debian.org> Mon, 07 Nov 2011 12:47:44 -0400 git-annex (3.20111105) unstable; urgency=low * The default backend used when adding files to the annex is changed from WORM to SHA256. To get old behavior, add a .gitattributes containing: * annex.backend=WORM * Sped up some operations on remotes that are on the same host. * copy --to: Fixed leak when copying many files to a remote on the same host. * uninit: Add guard against being run with the git-annex branch checked out. * Fail if --from or --to is passed to commands that do not support them. * drop --from is now supported to remove file content from a remote. * status: Now always shows the current repository, even when it does not appear in uuid.log. * fsck: Now works in bare repositories. Checks location log information, and file contents. Does not check that numcopies is satisfied, as .gitattributes information about numcopies is not available in a bare repository. * unused, dropunused: Now work in bare repositories. * Removed the setkey command, and added a reinject command with a more useful interface. * The fromkey command now takes the key as its first parameter. The --key option is no longer used. * Built without any filename containing .git being excluded. Closes: #647215 * Record uuid when auto-initializing a remote so it shows in status. * Bugfix: Fixed git-annex init crash in a bare repository when there was already an existing git-annex branch. * Pass -t to rsync to preserve timestamps. -- Joey Hess <joeyh@debian.org> Sat, 05 Nov 2011 15:47:52 -0400 git-annex (3.20111025) unstable; urgency=low * A remote can have a annexUrl configured, that is used by git-annex instead of its usual url. (Similar to pushUrl.) * migrate: Copy url logs for keys when migrating. * git-annex-shell: GIT_ANNEX_SHELL_READONLY and GIT_ANNEX_SHELL_LIMITED environment variables can be set to limit what commands can be run. This is used by gitolite's new git-annex support! -- Joey Hess <joeyh@debian.org> Tue, 25 Oct 2011 13:03:08 -0700 git-annex (3.20111011) unstable; urgency=low * This version of git-annex only works with git 1.7.7 and newer. The breakage with old versions is subtle, and affects the annex.numcopies settings in .gitattributes, so be sure to upgrade git to 1.7.7. (Debian package now depends on that version.) * Don't pass absolute paths to git show-attr, as it started following symlinks when that's done in 1.7.7. Instead, use relative paths, which show-attr only handles 100% correctly in 1.7.7. Closes: #645046 * Fix referring to remotes by uuid. * New or changed repository descriptions in uuid.log now have a timestamp, which is used to ensure the newest description is used when the uuid.log has been merged. * Note that older versions of git-annex will display the timestamp as part of the repository description, which is ugly but otherwise harmless. * Add timestamps to trust.log and remote.log too. * git-annex-shell: Added the --uuid option. * git-annex now asks git-annex-shell to verify that it's operating in the expected repository. * Note that this git-annex will not interoperate with remotes using older versions of git-annex-shell. * Now supports git's insteadOf configuration, to modify the url used to access a remote. Note that pushInsteadOf is not used; that and pushurl are reserved for actual git pushes. Closes: #644278 * status: List all known repositories. * When displaying a list of repositories, show git remote names in addition to their descriptions. * Add locking to avoid races when changing the git-annex branch. * Various speed improvements gained by using ByteStrings. * Contain the zombie hordes. -- Joey Hess <joeyh@debian.org> Tue, 11 Oct 2011 23:00:02 -0400 git-annex (3.20110928) unstable; urgency=low * --in can be used to make git-annex only operate on files believed to be present in a given repository. * Arbitrarily complex expressions can be built to limit the files git-annex operates on, by combining the options --not --and --or -( and -) Example: git annex get --exclude '*.mp3' --and --not -( --in usbdrive --or --in archive -) * --copies=N can be used to make git-annex only operate on files with the specified number of copies. (And --not --copies=N for the inverse.) * find: Rather than only showing files whose contents are present, when used with --exclude --copies or --in, displays all files that match the specified conditions. * Note that this is a behavior change for git-annex find! Old behavior can be gotten by using: git-annex find --in . * status: Massively sped up; remove --fast mode. * unused: File contents used by branches and tags are no longer considered unused, even when not used by the current branch. This is the final piece of the puzzle needed for git-annex to to play nicely with branches. -- Joey Hess <joeyh@debian.org> Wed, 28 Sep 2011 18:14:02 -0400 git-annex (3.20110915) unstable; urgency=low * whereis: Show untrusted locations separately and do not include in location count. * Fix build without S3. * addurl: Always use whole url as destination filename, rather than only its file component. * get, drop, copy: Added --auto option, which decides whether to get/drop content as needed to work toward the configured numcopies. * bugfix: drop and fsck did not honor --exclude -- Joey Hess <joeyh@debian.org> Thu, 15 Sep 2011 22:25:46 -0400 git-annex (3.20110906) unstable; urgency=low * Improve display of newlines around error and warning messages. * Fix Makefile to work with cabal again. -- Joey Hess <joeyh@debian.org> Tue, 06 Sep 2011 13:45:16 -0400 git-annex (3.20110902) unstable; urgency=low * Set EMAIL when running test suite so that git does not need to be configured first. Closes: #638998 * The wget command will now be used in preference to curl, if available. * init: Make description an optional parameter. * unused, status: Sped up by avoiding unnecessary stats of annexed files. * unused --remote: Reduced memory use to 1/4th what was used before. * Add --json switch, to produce machine-consumable output. -- Joey Hess <joeyh@debian.org> Fri, 02 Sep 2011 21:20:37 -0400 git-annex (3.20110819) unstable; urgency=low * Now "git annex init" only has to be run once, when a git repository is first being created. Clones will automatically notice that git-annex is in use and automatically perform a basic initalization. It's still recommended to run "git annex init" in any clones, to describe them. * Added annex-cost-command configuration, which can be used to vary the cost of a remote based on the output of a shell command. * Fix broken upgrade from V1 repository. Closes: #638584 -- Joey Hess <joeyh@debian.org> Fri, 19 Aug 2011 20:34:09 -0400 git-annex (3.20110817) unstable; urgency=low * Fix shell escaping in rsync special remote. * addurl: --fast can be used to avoid immediately downloading the url. * Added support for getting content from git remotes using http (and https). * Added curl to Debian package dependencies. -- Joey Hess <joeyh@debian.org> Wed, 17 Aug 2011 01:29:02 -0400 git-annex (3.20110719) unstable; urgency=low * add: Be even more robust to avoid ever leaving the file seemingly deleted. Closes: #634233 * Bugfix: Make add ../ work. * Support the standard git -c name=value * unannex: Clean up use of git commit -a. -- Joey Hess <joeyh@debian.org> Tue, 19 Jul 2011 23:39:53 -0400 git-annex (3.20110707) unstable; urgency=low * Fix sign bug in disk free space checking. * Bugfix: Forgot to de-escape keys when upgrading. Could result in bad location log data for keys that contain [&:%] in their names. (A workaround for this problem is to run git annex fsck.) * add: Avoid a failure mode that resulted in the file seemingly being deleted (content put in the annex but no symlink present). -- Joey Hess <joeyh@debian.org> Thu, 07 Jul 2011 19:29:39 -0400 git-annex (3.20110705) unstable; urgency=low * uninit: Delete the git-annex branch and .git/annex/ * unannex: In --fast mode, file content is left in the annex, and a hard link made to it. * uninit: Use unannex in --fast mode, to support unannexing multiple files that link to the same content. * Drop the dependency on the haskell curl bindings, use regular haskell HTTP. * Fix a pipeline stall when upgrading (caused by #624389). -- Joey Hess <joeyh@debian.org> Tue, 05 Jul 2011 14:37:39 -0400 git-annex (3.20110702) unstable; urgency=low * Now the web can be used as a special remote. This feature replaces the old URL backend. * addurl: New command to download an url and store it in the annex. * Sped back up fsck, copy --from, and other commands that often have to read a lot of information from the git-annex branch. Such commands are now faster than they were before introduction of the git-annex branch. * Always ensure git-annex branch exists. * Modify location log parser to allow future expansion. * --force will cause add, etc, to operate on ignored files. * Avoid mangling encoding when storing the description of repository and other content. * cabal can now be used to build git-annex. This is substantially slower than using make, does not build or install documentation, does not run the test suite, and is not particularly recommended, but could be useful to some. -- Joey Hess <joeyh@debian.org> Sat, 02 Jul 2011 15:00:18 -0400 git-annex (3.20110624) experimental; urgency=low * New repository format, annex.version=3. Use `git annex upgrade` to migrate. * git-annex now stores its logs in a git-annex branch. * merge: New subcommand. Auto-merges the new git-annex branch. * Improved handling of bare git repos with annexes. Many more commands will work in them. * git-annex is now more robust; it will never leave state files uncommitted when some other git process comes along and locks the index at an inconvenient time. * rsync is now used when copying files from repos on other filesystems. cp is still used when copying file from repos on the same filesystem, since --reflink=auto can make it significantly faster on filesystems such as btrfs. * Allow --trust etc to specify a repository by name, for temporarily trusting repositories that are not configured remotes. * unlock: Made atomic. * git-union-merge: New git subcommand, that does a generic union merge operation, and operates efficiently without touching the working tree. -- Joey Hess <joeyh@debian.org> Fri, 24 Jun 2011 14:32:18 -0400 git-annex (0.20110610) unstable; urgency=low * Add --numcopies option. * Add --trust, --untrust, and --semitrust options. * get --from is the same as copy --from * Bugfix: Fix fsck to not think all SHAnE keys are bad. -- Joey Hess <joeyh@debian.org> Fri, 10 Jun 2011 11:48:40 -0400 git-annex (0.20110601) unstable; urgency=low * Minor bugfixes and error message improvements. * Massively sped up `git annex lock` by avoiding use of the uber-slow `git reset`, and only running `git checkout` once, even when many files are being locked. * Fix locking of files with staged changes. * Somewhat sped up `git commit` of modifications to unlocked files. * Build fix for older ghc. -- Joey Hess <joeyh@debian.org> Wed, 01 Jun 2011 11:50:47 -0400 git-annex (0.20110522) unstable; urgency=low * Closer emulation of git's behavior when told to use "foo/.git" as a git repository instead of just "foo". Closes: #627563 * Fix bug in --exclude introduced in 0.20110516. -- Joey Hess <joeyh@debian.org> Fri, 27 May 2011 20:20:41 -0400 git-annex (0.20110521) unstable; urgency=low * status: New subcommand to show info about an annex, including its size. * --backend now overrides any backend configured in .gitattributes files. * Add --debug option. Closes: #627499 -- Joey Hess <joeyh@debian.org> Sat, 21 May 2011 11:52:53 -0400 git-annex (0.20110516) unstable; urgency=low * Add a few tweaks to make it easy to use the Internet Archive's variant of S3. In particular, munge key filenames to comply with the IA's filename limits, disable encryption, support their nonstandard way of creating buckets, and allow x-archive-* headers to be specified in initremote to set item metadata. * Added filename extension preserving variant backends SHA1E, SHA256E, etc. * migrate: Use current filename when generating new key, for backends where the filename affects the key name. * Work around a bug in Network.URI's handling of bracketed ipv6 addresses. -- Joey Hess <joeyh@debian.org> Mon, 16 May 2011 14:16:52 -0400 git-annex (0.20110503) unstable; urgency=low * Fix hasKeyCheap setting for bup and rsync special remotes. * Add hook special remotes. * Avoid crashing when an existing key is readded to the annex. * unused: Now also lists files fsck places in .git/annex/bad/ * S3: When encryption is enabled, the Amazon S3 login credentials are stored, encrypted, in .git-annex/remotes.log, so environment variables need not be set after the remote is initialized. -- Joey Hess <joeyh@debian.org> Tue, 03 May 2011 20:56:01 -0400 git-annex (0.20110427) unstable; urgency=low * Switch back to haskell SHA library, so git-annex remains buildable on Debian stable. * Added rsync special remotes. This could be used, for example, to store annexed content on rsync.net (encrypted naturally). Or anywhere else. * Bugfix: Avoid pipeline stall when running git annex drop or fsck on a lot of files. Possibly only occured with ghc 7. -- Joey Hess <joeyh@debian.org> Wed, 27 Apr 2011 22:50:26 -0400 git-annex (0.20110425) unstable; urgency=low * Use haskell Crypto library instead of haskell SHA library. * Remove testpack from build depends for non x86 architectures where it is not available. The test suite will not be run if it cannot be compiled. * Avoid using absolute paths when staging location log, as that can confuse git when a remote's path contains a symlink. Closes: #621386 -- Joey Hess <joeyh@debian.org> Mon, 25 Apr 2011 15:47:00 -0400 git-annex (0.20110420) unstable; urgency=low * Update Debian build dependencies for ghc 7. * Debian package is now built with S3 support. Thanks Joachim Breitner for making this possible. * Somewhat improved memory usage of S3, still work to do. Thanks Greg Heartsfield for ongoing work to improve the hS3 library for git-annex. -- Joey Hess <joeyh@debian.org> Thu, 21 Apr 2011 15:00:48 -0400 git-annex (0.20110419) unstable; urgency=low * Don't run gpg in batch mode, so it can prompt for passphrase when there is no agent. * Add missing build dep on dataenc. * S3: Fix stalls when transferring encrypted data. * bup: Avoid memory leak when transferring encrypted data. -- Joey Hess <joeyh@debian.org> Tue, 19 Apr 2011 21:26:51 -0400 git-annex (0.20110417) unstable; urgency=low * bup is now supported as a special type of remote. * The data sent to special remotes (Amazon S3, bup, etc) can be encrypted using GPG for privacy. * Use lowercase hash directories for locationlog files, to avoid some issues with git on OSX with the mixed-case directories. No migration is needed; the old mixed case hash directories are still read; new information is written to the new directories. * Unused files on remotes, particulary special remotes, can now be identified and dropped, by using "--from remote" with git annex unused and git annex dropunused. * Clear up short option confusion between --from and --force (-f is now --from, and there is no short option for --force). * Add build depend on perlmagick so docs are consistently built. Closes: #621410 * Add doc-base file. Closes: #621408 * Periodically flush git command queue, to avoid boating memory usage too much. * Support "sha1" and "sha512" commands on FreeBSD, and allow building if any/all SHA commands are not available. Thanks, Fraser Tweedale -- Joey Hess <joeyh@debian.org> Sun, 17 Apr 2011 12:00:24 -0400 git-annex (0.20110401) experimental; urgency=low * Amazon S3 is now supported as a special type of remote. Warning: Encrypting data before sending it to S3 is not yet supported. * Note that Amazon S3 support is not built in by default on Debian yet, as hS3 is not packaged. * fsck: Ensure that files and directories in .git/annex/objects have proper permissions. * Added a special type of remote called a directory remote, which simply stores files in an arbitrary local directory. * Bugfix: copy --to --fast never really copied, fixed. -- Joey Hess <joeyh@debian.org> Fri, 01 Apr 2011 21:27:22 -0400 git-annex (0.20110328) experimental; urgency=low * annex.diskreserve can be given in arbitrary units (ie "0.5 gigabytes") * Generalized remotes handling, laying groundwork for remotes that are not regular git remotes. (Think Amazon S3.) * Provide a less expensive version of `git annex copy --to`, enabled via --fast. This assumes that location tracking information is correct, rather than contacting the remote for every file. * Bugfix: Keys could be received into v1 annexes from v2 annexes, via v1 git-annex-shell. This results in some oddly named keys in the v1 annex. Recognise and fix those keys when upgrading, instead of crashing. -- Joey Hess <joeyh@debian.org> Mon, 28 Mar 2011 10:47:29 -0400 git-annex (0.20110325) experimental; urgency=low * Free space checking is now done, for transfers of data for keys that have free space metadata. (Notably, not for SHA* keys generated with git-annex 0.2x or earlier.) The code is believed to work on Linux, FreeBSD, and OSX; check compile-time messages to see if it is not enabled for your OS. * Add annex.diskreserve config setting, to control how much free space to reserve for other purposes and avoid using (defaults to 1 mb). * Add --fast flag, that can enable less expensive, but also less thorough versions of some commands. * fsck: In fast mode, avoid checking checksums. * unused: In fast mode, just show all existing temp files as unused, and avoid expensive scan for other unused content. * migrate: Support migrating v1 SHA keys to v2 SHA keys with size information that can be used for free space checking. * Fix space leak in fsck and drop commands. * migrate: Bugfix for case when migrating a file results in a key that is already present in .git/annex/objects. * dropunused: Significantly sped up; only read unused log file once. -- Joey Hess <joeyh@debian.org> Fri, 25 Mar 2011 00:47:37 -0400 git-annex (0.20110320) experimental; urgency=low * Fix dropping of files using the URL backend. * Fix support for remotes with '.' in their names. * Add version command to show git-annex version as well as repository version information. * No longer auto-upgrade to repository format 2, to avoid accidental upgrades, etc. Use git-annex upgrade when you're ready to run this version. -- Joey Hess <joeyh@debian.org> Sun, 20 Mar 2011 16:36:33 -0400 git-annex (0.20110316) experimental; urgency=low * New repository format, annex.version=2. * The first time git-annex is run in an old format repository, it will automatically upgrade it to the new format, staging all necessary changes to git. Also added a "git annex upgrade" command. * Colons are now avoided in filenames, so bare clones of git repos can be put on USB thumb drives formatted with vFAT or similar filesystems. * Added two levels of hashing to object directory and .git-annex logs, to improve scalability with enormous numbers of annexed objects. (With one hundred million annexed objects, each directory would contain fewer than 1024 files.) * The setkey, fromkey, and dropkey subcommands have changed how the key is specified. --backend is no longer used with these. -- Joey Hess <joeyh@debian.org> Wed, 16 Mar 2011 16:20:23 -0400 git-annex (0.24) unstable; urgency=low Branched the 0.24 series, which will be maintained for a while to support v1 git-annex repos, while main development moves to the 0.2011 series, with v2 git-annex repos. * Add Suggests on graphviz. Closes: #618039 * When adding files to the annex, the symlinks pointing at the annexed content are made to have the same mtime as the original file. While git does not preserve that information, this allows a tool like metastore to be used with annexed files. (Currently this is only done on systems supporting POSIX 200809.) -- Joey Hess <joeyh@debian.org> Wed, 16 Mar 2011 18:35:13 -0400 git-annex (0.23) unstable; urgency=low * Support ssh remotes with a port specified. * whereis: New subcommand to show where a file's content has gotten to. * Rethink filename encoding handling for display. Since filename encoding may or may not match locale settings, any attempt to decode filenames will fail for some files. So instead, do all output in binary mode. -- Joey Hess <joeyh@debian.org> Sat, 12 Mar 2011 15:02:49 -0400 git-annex (0.22) unstable; urgency=low * Git annexes can now be attached to bare git repositories. (Both the local and remote host must have this version of git-annex installed for it to work.) * Support filenames that start with a dash; when such a file is passed to a utility it will be escaped to avoid it being interpreted as an option. (I went a little overboard and got the type checker involved in this, so such files are rather comprehensively supported now.) * New backends: SHA512 SHA384 SHA256 SHA224 (Supported on systems where corresponding shaNsum commands are available.) * describe: New subcommand that can set or change the description of a repository. * Fix test suite to reap zombies. (Zombies can be particularly annoying on OSX; thanks to Jimmy Tang for his help eliminating the infestation... for now.) * Make test suite not rely on a working cp -pr. (The Unix wars are still ON!) * Look for dir.git directories the same as git does. * Support remote urls specified as relative paths. * Support non-ssh remote paths that contain tilde expansions. * fsck: Check for and repair location log damage. * Bugfix: When fsck detected and moved away corrupt file content, it did not update the location log. -- Joey Hess <joeyh@debian.org> Fri, 04 Mar 2011 15:10:57 -0400 git-annex (0.21) unstable; urgency=low * test: Don't rely on chmod -R working. * unannex: Fix recently introduced bug when attempting to unannex more than one file at a time. * test: Set git user name and email in case git can't guess values. * Fix display of unicode filenames. -- Joey Hess <joeyh@debian.org> Fri, 11 Feb 2011 23:21:08 -0400 git-annex (0.20) unstable; urgency=low * Preserve specified file ordering when instructed to act on multiple files or directories. For example, "git annex get a b" will now always get "a" before "b". Previously it could operate in either order. * unannex: Commit staged changes at end, to avoid some confusing behavior with the pre-commit hook, which would see some types of commits after an unannex as checking in of an unlocked file. * map: New subcommand that uses graphviz to display a nice map of the git repository network. * Deal with the mtl/monads-fd conflict. * configure: Check for sha1sum. -- Joey Hess <joeyh@debian.org> Tue, 08 Feb 2011 18:57:24 -0400 git-annex (0.19) unstable; urgency=low * configure: Support using the uuidgen command if the uuid command is not available. * Allow --exclude to be specified more than once. * There are now three levels of repository trust. * untrust: Now marks the current repository as untrusted. * semitrust: Now restores the default trust level. (What untrust used to do.) * fsck, drop: Take untrusted repositories into account. * Bugfix: Files were copied from trusted remotes first even if their annex.cost was higher than other remotes. * Improved temp file handling. Transfers of content can now be resumed from temp files later; the resume does not have to be the immediate next git-annex run. * unused: Include partially transferred content in the list. * Bugfix: Running a second git-annex while a first has a transfer in progress no longer deletes the first processes's temp file. -- Joey Hess <joeyh@debian.org> Fri, 28 Jan 2011 14:31:37 -0400 git-annex (0.18) unstable; urgency=low * Bugfix: `copy --to` and `move --to` forgot to stage location log changes after transferring the file to the remote repository. (Did not affect ssh remotes.) * fsck: Fix bug in moving of corrupted files to .git/annex/bad/ * migrate: Fix support for --backend option. * unlock: Fix behavior when file content is not present. * Test suite improvements. Current top-level test coverage: 80% -- Joey Hess <joeyh@debian.org> Fri, 14 Jan 2011 14:17:44 -0400 git-annex (0.17) unstable; urgency=low * unannex: Now skips files whose content is not present, rather than it being an error. * New migrate subcommand can be used to switch files to using a different backend, safely and with no duplication of content. * bugfix: Fix crash caused by empty key name. (Thanks Henrik for reporting.) -- Joey Hess <joeyh@debian.org> Sun, 09 Jan 2011 10:04:11 -0400 git-annex (0.16) unstable; urgency=low * git-annex-shell: Avoid exposing any git repo config except for the annex.uuid when doing configlist. * bugfix: Running `move --to` with a remote whose UUID was not yet known could result in git-annex not recording on the local side where the file was moved to. This could not result in data loss, or even a significant problem, since the remote *did* record that it had the file. * Also, add a general guard to detect attempts to record information about repositories with missing UUIDs. * bugfix: Running `move --to` with a non-ssh remote failed. * bugfix: Running `copy --to` with a non-ssh remote actually did a move. * Many test suite improvements. Current top-level test coverage: 65% -- Joey Hess <joeyh@debian.org> Fri, 07 Jan 2011 14:33:13 -0400 git-annex (0.15) unstable; urgency=low * Support scp-style urls for remotes (host:path). * Support ssh urls containing "~". * Add trust and untrust subcommands, to allow configuring repositories that are trusted to retain files without explicit checking. * Fix bug in numcopies handling when multiple remotes pointed to the same repository. * Introduce the git-annex-shell command. It's now possible to make a user have it as a restricted login shell, similar to git-shell. * Note that git-annex will always use git-annex-shell when accessing a ssh remote, so all of your remotes need to be upgraded to this version of git-annex at the same time. * Now rsync is exclusively used for copying files to and from remotes. scp is not longer supported. -- Joey Hess <joeyh@debian.org> Fri, 31 Dec 2010 22:00:52 -0400 git-annex (0.14) unstable; urgency=low * Bugfix to git annex unused in a repository with nothing yet annexed. * Support upgrading from a v0 annex with nothing in it. * Avoid multiple calls to git ls-files when passed eg, "*". -- Joey Hess <joeyh@debian.org> Fri, 24 Dec 2010 17:38:48 -0400 git-annex (0.13) unstable; urgency=low * Makefile: Install man page and html (when built). * Makefile: Add GHCFLAGS variable. * Fix upgrade from 0.03. * Support remotes using git+ssh and ssh+git as protocol. Closes: #607056 -- Joey Hess <joeyh@debian.org> Tue, 14 Dec 2010 13:05:10 -0400 git-annex (0.12) unstable; urgency=low * Add --exclude option to exclude files from processing. * mwdn2man: Fix a bug in newline supression. Closes: #606578 * Bugfix to git annex add of an unlocked file in a subdir. Closes: #606579 * Makefile: Add PREFIX variable. -- Joey Hess <joeyh@debian.org> Sat, 11 Dec 2010 17:32:00 -0400 git-annex (0.11) unstable; urgency=low * If available, rsync will be used for file transfers from remote repositories. This allows resuming interrupted transfers. * Added remote.annex-rsync-options. * Avoid deleting temp files when rsync fails. * Improve detection of version 0 repos. * Add uninit subcommand. Closes: #605749 -- Joey Hess <joeyh@debian.org> Sat, 04 Dec 2010 17:27:42 -0400 git-annex (0.10) unstable; urgency=low * In .gitattributes, the annex.numcopies attribute can be used to control the number of copies to retain of different types of files. * Bugfix: Always correctly handle gitattributes when in a subdirectory of the repository. (Had worked ok for ones like "*.mp3", but failed for ones like "dir/*".) * fsck: Fix warning about not enough copies of a file, when locations are known, but are not available in currently configured remotes. * precommit: Optimise to avoid calling git-check-attr more than once. * The git-annex-backend attribute has been renamed to annex.backend. -- Joey Hess <joeyh@debian.org> Sun, 28 Nov 2010 19:28:05 -0400 git-annex (0.09) unstable; urgency=low * Add copy subcommand. * Fix bug in setkey subcommand triggered by move --to. -- Joey Hess <joeyh@debian.org> Sat, 27 Nov 2010 17:14:59 -0400 git-annex (0.08) unstable; urgency=low * Fix `git annex add ../foo` (when ran in a subdir of the repo). * Add configure step to build process. * Only use cp -a if it is supported, falling back to cp -p or plain cp as needed for portability. * cp --reflink=auto is used if supported, and will make git annex unlock much faster on filesystems like btrfs that support copy on write. -- Joey Hess <joeyh@debian.org> Sun, 21 Nov 2010 13:45:44 -0400 git-annex (0.07) unstable; urgency=low * find: New subcommand. * unused: New subcommand, finds unused data. (Split out from fsck.) * dropunused: New subcommand, provides for easy dropping of unused keys by number, as listed by the unused subcommand. * fsck: Print warnings to stderr; --quiet can now be used to only see problems. -- Joey Hess <joeyh@debian.org> Mon, 15 Nov 2010 18:41:50 -0400 git-annex (0.06) unstable; urgency=low * fsck: Check if annex.numcopies is satisfied. * fsck: Verify the sha1 of files when the SHA1 backend is used. * fsck: Verify the size of files when the WORM backend is used. * fsck: Allow specifying individual files if fscking everything is not desired. * fsck: Fix bug, introduced in 0.04, in detection of unused data. -- Joey Hess <joeyh@debian.org> Sat, 13 Nov 2010 16:24:29 -0400 git-annex (0.05) unstable; urgency=low * Optimize both pre-commit and lock subcommands to not call git diff on every file being committed/locked. (This actually also works around a bug in ghc, that caused git-annex 0.04 pre-commit to sometimes corrupt filename being read from git ls-files and fail. See <http://hackage.haskell.org/trac/ghc/ticket/4493> The excessive number of calls made by pre-commit exposed the ghc bug. Thanks Josh Triplett for the debugging.) * Build with -O2. -- Joey Hess <joeyh@debian.org> Thu, 11 Nov 2010 18:31:09 -0400 git-annex (0.04) unstable; urgency=low * Add unlock subcommand, which replaces the symlink with a copy of the file's content in preparation of changing it. The "edit" subcommand is an alias for unlock. * Add lock subcommand. * Unlocked files will now automatically be added back into the annex when committed (and the updated symlink committed), by some magic in the pre-commit hook. * The SHA1 backend is now fully usable. * Add annex.version, which will be used to automate upgrades between incompatible versions. * Reorganised the layout of .git/annex/ * The new layout will be automatically upgraded to the first time git-annex is used in a repository with the old layout. * Note that git-annex 0.04 cannot transfer content from old repositories that have not yet been upgraded. * Annexed file contents are now made unwritable and put in unwriteable directories, to avoid them accidentally being removed or modified. (Thanks Josh Triplett for the idea.) * Add build dep on libghc6-testpack-dev. Closes: #603016 * Avoid using runghc to run test suite as it is not available on all architectures. Closes: #603006 -- Joey Hess <joeyh@debian.org> Wed, 10 Nov 2010 14:23:23 -0400 git-annex (0.03) unstable; urgency=low * Fix support for file:// remotes. * Add --verbose * Fix SIGINT handling. * Fix handling of files with unusual characters in their name. * Fixed memory leak; git-annex no longer reads the whole file list from git before starting, and will be much faster with large repos. * Fix crash on unknown symlinks. * Added remote.annex-scp-options and remote.annex-ssh-options. * The backends to use when adding different sets of files can be configured via gitattributes. * In .gitattributes, the git-annex-backend attribute can be set to the names of backends to use when adding different types of files. * Add fsck subcommand. (For now it only finds unused key contents in the annex.) -- Joey Hess <joeyh@debian.org> Sun, 07 Nov 2010 18:26:04 -0400 git-annex (0.02) unstable; urgency=low * Can scp annexed files from remote hosts, and check remote hosts for file content when dropping files. * New move subcommand, that makes it easy to move file contents from or to a remote. * New fromkey subcommand, for registering urls, etc. * git-annex init will now set up a pre-commit hook that fixes up symlinks before they are committed, to ensure that moving symlinks around does not break them. * More intelligent and fast staging of modified files; git add coalescing. * Add remote.annex-ignore git config setting to allow completly disabling a given remote. * --from/--to can be used to control the remote repository that git-annex uses. * --quiet can be used to avoid verbose output * New plumbing-level dropkey and addkey subcommands. * Lots of bug fixes. -- Joey Hess <joeyh@debian.org> Wed, 27 Oct 2010 16:39:29 -0400 git-annex (0.01) unstable; urgency=low * First prerelease. -- Joey Hess <joeyh@debian.org> Wed, 20 Oct 2010 12:54:24 -0400