As a haskell package, git-annex can be installed using cabal. For example:

cabal update
PATH=$HOME/bin:$PATH
cabal install c2hs git-annex --bindir=$HOME/bin

The above downloads the latest release and installs it into a ~/bin/ directory, which you can put in your PATH.

But maybe you want something newer (or older). Then download the version you want, and use cabal as follows inside its source tree:

cabal update
PATH=$HOME/bin:$PATH
cabal install c2hs --bindir=$HOME/bin
cabal install --only-dependencies
cabal configure
cabal build
cabal install --bindir=$HOME/bin