Module → Package lookup in ghc-pkg?

[dmhouse@…]

Hi all,

I'm writing code for Emacs's haskell-mode, and it's necessary at one point to be able to find out the package a given module is defined in. The only way of doing this currently is to call ghc-pkg field <package> exposed-modules for every package, then building a lookup table of modules to packages. Could we have one of:

1. A way to batch call ghc-pkg once to fetch the exposed-modules (or any field, in fact, as we need to fetch haddock-html for each package as well) for all packages?
2. A way to lookup which package(s) a given module is defined in from ghc-pkg?

Cheers,

-David House

[neil]

If Hoogle was integrated with ghc-pkg/Cabal then Hoogle could provide this functionality.

[claus]

i, too, would like to be able to get more information about installed packages more easily: see the end of  http://www.haskell.org/pipermail/haskell-cafe/2007-March/023150.html

however, hoogle would only be one of many tools making use of this kind of index, and i think the functionality is better located in ghc-pkg/cabal.

in fact, i thought it should be one of the things cabal ought to be good at, but looking through the haddocks for Distribution.*, i seem to be running around in circles: is there any standard way to get at the installed package.conf files, or directly at the InstalledPackageInfos?? there's a getInstalledPackages, but that only gives me the PackageIdentifiers?, with no obvious way to go further?

i only started to make progress when i abandoned cabal and focussed on its data structures. here's a quick hack, i'd appreciate suggestions on how to overcome its obvious limitations:

import Distribution.Simple.Configure
import Distribution.Simple
import Distribution.InstalledPackageInfo
import System.FilePath

getInfo packageName = do
  Just ghc <- findProgram "ghc" Nothing
  let ghcDir = takeDirectory $ takeDirectory ghc -- drop bin/ghc[.exe]
  conf <- readFile (ghcDir </> "package.conf")   -- how to get this path?
  let infos = (read conf :: [InstalledPackageInfo])
  return [ (name, exposedModules info, haddockHTMLs info) 
         | info <- infos
         , let name = pkgName (package info)
         , name == packageName ]

example output:

*Main> getInfo "haskell-src"
[("haskell-src",["Language.Haskell.Lexer","Language.Haskell.Parser","Language.Ha
skell.ParseMonad","Language.Haskell.ParseUtils","Language.Haskell.Pretty","Langu
age.Haskell.Syntax"],["$topdir\\html\\libraries\\haskell-src"])]

[simonmar]

in fact, i thought it should be one of the things cabal ought to be good at, but looking through the haddocks for Distribution.*, i seem to be running around in circles: is there any standard way to get at the installed package.conf files, or directly at the InstalledPackageInfos??? there's a getInstalledPackages, but that only gives me the PackageIdentifiers??, with no obvious way to go further?

This is parly by design. Cabal interacts with the package database only using the ghc-pkg tool (hc-pkg in general), the idea being that the representation and location of the package database can be compiler-dependent. There's no requirement that the package database is stored in the way that GHC does it, or even that the package database contains InstalledPackageInfos - a compiler might want to add extra information about each package. In fact, GHC already has an alternative way to store the package database (files in package.conf.d).

To get an InstalledPackageInfo for a given package, you have to use ghc-pkg describe and parse the output (I believe it should work). Parsing package.conf directly is non-portable, and risks breakage in the future. We could add a way to get the Read/Show version of an InstalledPackageInfo, if that would make it easier.

[claus]

To get an InstalledPackageInfo? for a given package, you have to use ghc-pkg describe and parse the output (I believe it should work). Parsing package.conf directly is non-portable, and risks breakage in the future.

hmm. i was looking for a portable, cabal-based way to get the info. i wouldn't mind if cabal offered me a way to get that without telling me what the actual representation is, but InstalledPackageInfo? seems to be the only concrete format defined so far, so converting whatever the actual format is into that would seem to make sense.

if there is no cabal way (ghc-pkg certainly isn't), the next best thing would be to use the ghc api, provided that the ghc-pkg functionality is exposed there? alternatively, ghc-pkg itself could be extended, but i assume that different tools prefer the info in different formats, so i'd prefer a programmable solution, without reparsing unstable output formats. btw, 'ghc --simple-output field exposed-modules' should probably drop the field label?

just to explain the dilemma: currently, my haskell mode for vim uses vimscript to extract part of the info from haddock's index files, and another part of the info from calling 'ghc -e :browse' on all imported modules. it then builds dictionaries for fast access to that info, and in the case of standard libs, caches that dictionary in a file for speed. now, haddock is the only tool that knows which identifiers in the installed libs have haddock comments, and what their relative urls are, but i can't help feeling that there must be an easier way to get all that info (i'm not even trying to extract types from haddock index files). i assume David has something similar in mind for emacs?

[simonmar]

hmm. i was looking for a portable, cabal-based way to get the info. i wouldn't mind if cabal offered me a way to get that without telling me what the actual representation is, but InstalledPackageInfo seems to be the only concrete format defined so far, so converting whatever the actual format is into that would seem to make sense.

Cabal doesn't currently have what you're asking for, but there's no reason it couldn't be added. It could be implemented exactly as I described: invoke ghc-pkg describe and parse the output. You would have to tell Cabal what command to use for ghc-pkg, or re-use some of Cabal's configuration machinery. In theory this method is portable; in practice only GHC has an hc-pkg command at the moment.

[guest]

i've got a minor patch that extends ghci's :info to handle module names as well, so

Prelude> :i Prelude Data.Maybe Distribution.System Data.Array Data.Sequence
module Prelude is in package base
module Data.Maybe is in package base
module Distribution.System is in package Cabal-1.3
module Data.Array is in package array-0.1
module Data.Sequence is in package containers-0.1

that seemed to be the most obvious place to put it, because both i and haskell modes tend to use ghci a lot, and i often wondered why :info <module> didn't work. and with 'ghc -e', any ghci command is as good as a ghc option. it only gives information on modules in exposed packages, so

Prelude> :i GHC

Top level: Not in scope: type constructor or class `GHC'

Prelude> :set -package ghc
package flags have changed, ressetting and loading new packages...
Loading package old-locale-1.0 ... linking ... done.
Loading package old-time-1.0 ... linking ... done.
Loading package filepath-1.0 ... linking ... done.
Loading package directory-1.0 ... linking ... done.
Loading package array-0.1 ... linking ... done.
Loading package containers-0.1 ... linking ... done.
Loading package hpc-0.5 ... linking ... done.
Loading package bytestring-0.9 ... linking ... done.
Loading package pretty-1.0 ... linking ... done.
Loading package packedstring-0.1 ... linking ... done.
Loading package template-haskell ... linking ... done.
Loading package Win32-2.1 ... linking ... done.
Loading package process-1.0 ... linking ... done.
Loading package Cabal-1.3 ... linking ... done.
Loading package random-1.0 ... linking ... done.
Loading package haskell98 ... linking ... done.
Loading package ghc-6.9.20071019 ... linking ... done.
Prelude> :i GHC
module GHC is in package ghc-6.9.20071019

would this help with your request? claus

[guest]

it occurred to me that ghc-pkg find-module Data.Sequence is just a minor variation on ghc-pkg list containers, so i implemented it as such:

 http://www.haskell.org/pipermail/cvs-ghc/2007-November/039421.html

it would still be nice to be able to combine multiple ghc-pkg queries into one, but i guess that would be another ticket?-)

[guest]

Replying to guest:

i've got a minor patch that extends ghci's :info to handle module names as well, so {{{ Prelude> :i Prelude Data.Maybe Distribution.System Data.Array Data.Sequence

@Simons:

i'm having a minor problem with this one: getting the info is easy enough, but a name might be both module and something else, so i'd like :info to check both, reporting errors only if it can't find either. but the error output for the existing checks seems to be hardwired deep in the call chain, so i don't know how to disable that for names that have successful module information.

[guest]

Replying to guest:

it occurred to me that ghc-pkg find-module Data.Sequence is just a minor variation on ghc-pkg list containers, so i implemented it as such:  http://www.haskell.org/pipermail/cvs-ghc/2007-November/039421.html

this addresses point 2 of this ticket.

it would still be nice to be able to combine multiple ghc-pkg queries into one, but i guess that would be another ticket?-)

which is point 1 of this ticket, and also asked for in #1839, where you can find a patch in progress.

claus

[simonpj]

Replying to guest:

i'm having a minor problem with this one: getting the info is easy enough, but a name might be both module and something else, so i'd like :info to check both, reporting errors only if it can't find either. but the error output for the existing checks seems to be hardwired deep in the call chain, so i don't know how to disable that for names that have successful module information.

I think that GHC.parseName should return an empty list (with no error message) if it doesn't find any bindings for the specified string. That way the client of the API can complain if necessary. Once parseName has returned a list of Names I think they should all be lookable up (using GHC.getInfo) without errors.

parseName is defined in InteractiveEval. It looks to me that it can fail in two ways:

• hscParseIdentifier can fail, printing an error message, if the name doesn't look like an identifier (e.g. "foo/bar").
• Then TcRnDriver.tcRnLookupRdrName can fail (printing an error message) if the name isn't in scope. In both cases one could return an Either instead of a Maybe to contain the error messages.

In general, making the GHC API return error messages rather than print them is a good idea. (I think you can set the "how to print an error message" hook before calling the GHC API, which might serve the same purpose, but it's less functional.)

does that help?

Simon

[claus]

Replying to simonpj:

Replying to guest: I think that GHC.parseName should return an empty list (with no error message) if it doesn't find any bindings for the specified string. That way the client of the API can complain if necessary. Once parseName has returned a list of Names I think they should all be lookable up (using GHC.getInfo) without errors.

yes, if GHC.parseName (and ghc api functions in general) could be made to behave that way, it would help. i could then ignore that extra error output in case i've already found module-related info.

there seems to be a third case, btw, failing to load interface:

$ /cygdrive/c/fptools/ghc/compiler/stage2/ghc-inplace --interactive
GHCi, version 6.9.20071106: http://www.haskell.org/ghc/  :? for help
Loading package base ... linking ... done.
Prelude> :i foo/bar
<interactive>:1:3: parse error on input `/'
[]

Prelude> :i List
module List -- in package haskell98

Top level: Not in scope: type constructor or class `List'
[]

Prelude> :i Data.List
module Data.List -- in package base

Top level:
    Failed to load interface for `Data':
      Use -v to see a list of the files searched for.
[]

[simonmar]

Patch pushed.