findExeutable does not respect order of search path on Windows

[duncan]

On Windows findExeutable is behaving rather strangely.

From a command line window (ie not MSYS):

H:\>echo %PATH%
C:\Program Files\MiKTeX 2.7\miktex\bin;
C:\WINDOWS\system32;
C:\WINDOWS;
C:\WINDOWS\System32\Wbem;
C:\Program Files\QuickTime\QTSystem\;
c:\Program Files\Microsoft SQL Server\90\Tools\binn\;
C:\Program Files\MATLAB\R2009a\bin;
C:\Program Files\MATLAB\R2009a\bin\win32;
C:\Program Files\Haskell\bin;
D:\dcoutts\ghc-6.10.1\bin;
D:\dcoutts\Gtk2Hs\bin;
D:\dcoutts\bin

(formatted onto multiple lines for clarity)

Now it appears I've got three versions of haddock installed:

H:\>where haddock
C:\Program Files\Haskell\bin\haddock.exe
D:\dcoutts\ghc-6.10.1\bin\haddock.exe
D:\dcoutts\bin\haddock.exe

However in ghci:

H:\>ghci
Prelude> System.Directory.findExecutable "haddock"
Just "D:\\dcoutts\\GHC-61~1.1\\bin\\haddock.exe"

But this is bonkers! The version it has found is not in the first directory on the search path.

I don't think it's related to the global/user split in the Windows %PATH%. All of the dirs containing haddock.exe are in the user portion of my %PATH%.

On Windows System.Directory.findExecutable uses the SearchPathA Win32 function. The  MSDN docs mention that it searches system and the current dirs first and has some "safe" search mode but none of those things seem to be relevant here.

This problem confuses users of Cabal which uses findExecutable to find build tools. See [this thread  http://haskell.org/pipermail/cabal-devel/2009-April/005158.html] from the cabal mailing list.

Perhaps we should just use the pure Haskell version that we use on non-Windows systems. Still, it'd be nice to reflect properly the Windows search path semantics if it's not as simple as looking in the %PATH%.

Actually the semantics are pretty weird. Consider this...

Let's cut down to just two versions of haddock:

H:\>where haddock
C:\Program Files\Haskell\bin\haddock.exe
D:\dcoutts\ghc-6.10.1\bin\haddock.exe

Now lets see what versions we've got where:

H:\>"C:\Program Files\Haskell\bin\haddock.exe" --version
Haddock version 2.1.0, (c) Simon Marlow 2006
Ported to use the GHC API by David Waern 2006-2008

H:\>D:\dcoutts\ghc-6.10.1\bin\haddock.exe --version
Haddock version 2.3.0, (c) Simon Marlow 2006
Ported to use the GHC API by David Waern 2006-2008

As expected, just running haddock from the command line gives us the first one:

H:\>haddock --version
Haddock version 2.1.0, (c) Simon Marlow 2006
Ported to use the GHC API by David Waern 2006-2008

Now in ghci:

Prelude> System.Process.runCommand "haddock --version"
Haddock version 2.1.0, (c) Simon Marlow 2006
Ported to use the GHC API by David Waern 2006-2008

That's fine. It's the first one and consistent with running from the command line (internally runCommand uses the command processor).

But now try this:

Prelude> System.Process.runProcess "haddock" ["--version"] Nothing Nothing Nothing Nothing Nothing
Haddock version 2.3.0, (c) Simon Marlow 2006
Ported to use the GHC API by David Waern 2006-2008

WT!? Where did that come from? It's using a different search path from the command interpreter. This one however is consistent with the findExecutable function (as it is supposed to be).

No idea what is going on. Can anyone else reproduce or explain this?

[NeilMitchell]

FWIW, I considered the user vs system %PATH% split, but when I was having problems I checked and all my copies of haddock were on the user part of %PATH%, so that doesn't seem relevant.

[duncan]

Replying to NeilMitchell:

FWIW, I considered the user vs system %PATH% split, but when I was having problems I checked and all my copies of haddock were on the user part of %PATH%, so that doesn't seem relevant.

Yeah, same here.

I note that we do not pass the %PATH% to the SearchPath function. We pas NULL which lets it do some system-dependent thing. It might be interesting to see if passing the %PATH% makes any difference to the behaviour. A quick google search doesn't bring up anything about this oddity.

[duncan]

Ahh! It's looking in the directory where the .exe for the currently running process lives. So when we're in ghci (or runghc Setup) then it always finds the haddock.exe in the same dir as ghci/runghc.

Neil, I'm guessing that you're running cabal.exe from `C:\Program Files\Haskell\bin\ is that correct? If so, that explains why it's picking up haddock.exe` from the same directory.

So the question is what behaviour we want. I guess it is good if findExecutable and runProcess are consistent with each other. However it should be clearly documented along with findExecutable that the location of the .exe of the current process is relevant. Cabal should probably look for haddock first in the ghc bin dir and only secondarily try using findExecutable (sigh).

[NeilMitchell]

Ahh! That explains it! I am indeed running cabal from next to haddock.

I don't see why findExecutable is using the path you started the program from. It seems reasonable to use the current directory first, but other than that should jump straight to the %PATH%. For finding .dll's it makes sense for findExecutable to try the relative to .exe path first, which I suspect is why this behaviour exists.

Cabal should probably check GHC bin dir for haddock anyway, given the GHC/haddock dependency.

[duncan]

Replying to NeilMitchell:

Ahh! That explains it! I am indeed running cabal from next to haddock.

Ok good, I'm glad we figured it out.

You win the prize for finding this month's most peculiar bug :-)

I don't see why findExecutable is using the path you started the program from. It seems reasonable to use the current directory first, but other than that should jump straight to the %PATH%. For finding .dll's it makes sense for findExecutable to try the relative to .exe path first, which I suspect is why this behaviour exists.

Though the MSDN page says it should not be used for locating .dll files because LoadLibrary? uses a different order again.

Cabal should probably check GHC bin dir for haddock anyway, given the GHC/haddock dependency.

Yeah:  http://hackage.haskell.org/trac/hackage/ticket/546

So, to close this ticket I think we just want to document that findExecutable searches a whole bunch of dirs. We should pin it down. The MSDN docs do not explicitly say. It's probably the exe dir, current dir, windows and system dirs and then the %PATH% (and it's modified by "safe search" in Vista I think).

[simonmar]

Is this ok?

-- On Windows, 'findExecutable' calls the Win32 function 'SearchPath',
-- which may search other places before checking the directories in
-- @PATH@.  Where it actually searches depends on registry settings,
-- but notably includes the directory containing the current
-- executable. See <http://msdn.microsoft.com/en-us/library/aa365527.aspx>
-- for more details.

[duncan]

Replying to simonmar:

Is this ok?

Yes, it's suitable vague :-). Linking to the MS docs is a good idea.

In particular, I think the search order corresponds to the program you'd run if you call createProcess using the direct method rather than using a shell intermediary. That fact is probably worth mentioning, and that it can be different to what you get if you go via a shell because the dir of the current process is different.

[simonmar]

Fixed:

Fri May 29 13:34:49 BST 2009  Simon Marlow <marlowsd@gmail.com>
  * Fix #3189: add docs to findExecutable

d'oh, I got the bug number wrong in the patch. Oh well.

The docs for findExceutable now say:

-- | Given an executable file name, searches for such file in the
-- directories listed in system PATH. The returned value is the path
-- to the found executable or Nothing if an executable with the given
-- name was not found. For example (findExecutable \"ghc\") gives you
-- the path to GHC.
--
-- The path returned by 'findExecutable' corresponds to the
-- program that would be executed by 'System.Process.createProcess'
-- when passed the same string (as a RawCommand, not a ShellCommand).
--
-- On Windows, 'findExecutable' calls the Win32 function 'SearchPath',
-- which may search other places before checking the directories in
-- @PATH@.  Where it actually searches depends on registry settings,
-- but notably includes the directory containing the current
-- executable. See
-- <http://msdn.microsoft.com/en-us/library/aa365527.aspx> for more
-- details.  
--