Check for required C libraries during configure

[guest]

Cabal should check for required C libraries and other non-Haskell dependencies at the configure stage, rather than just waiting for the linker to fail with an obscure error message during the build stage.

Error messages should be helpful for the typical Cabal user, who should not be assumed to know anything about C libraries or linking.

That is not so simple. For example:

The haddock package current requires the C libraries for readline, due to its dependency on the GHC API. If that library is missing, the user should have some way of figuring out how do the right thing on the given platform - for most Linux distros, install the readline-dev package (as opposed to just readline), for Mac OS X download the GNUreadline framework from the GHC website and drag it into the appropriate Frameworks folder depending on whether this is Tiger and below or Leopard and above, etc.

[guest]

See also:

• #15 and #159 - implement dependency analysis
• #216 - specify more kinds of dependencies in .cabal files
• #177 - check for required directories

[duncan]

I don't think the dependency analysis tickets are relevant here. Just knowing the required C headers and C libs and the include and library search paths is enough information to do the checks. It's just the same check that autoconf does and we can do it in a similar way.

I'm not sure that being able to specify the foreign package dependencies would help here as we don't have a platform independent way of checking for them anyway. Those are more hints for package managers I'd say.

So we should look up exactly how autoconf does these checks. My understanding is:

• for headers it makes a small .c file that #includes the header and calls gcc/cpp to process the file. It passes the extra include search dirs with the -I flag.
• for libs it makes a small .o file and calls gcc/ld with the -l and -L flags.

If they fail then it means we cannot find the header/lib in question.

[duncan]

I've been thinking about this and I think that with packages that use confiure scripts there is actually no point at which we can do this check. The reason is that we need any extra settings from the .buildinfo file, however that file is generated by ./configure and the Setup.hs script never sees it.

In fact the first point we could see the full information is during preBuild, we could not do the check during configure at all.

We could try and hack it by reading the .buildinfo file directly during postConf but that's pretty hairy since the user can modify the hooks however they like and there is no requirement that they use any particular file.

So we may have to only do this check for build-type: Simple.

[alexeevg]

we should look up exactly how autoconf does these checks

I started looking into it and stumbled upon a little problem. Maybe it's a silly question, but I'm going to ask it anyway. Autoconf uses AC_CHECK_LIB macro to check if the library is present. AC_CHECK_LIB takes the library name and a function(symbol) name that is expected to be exported from the library, generates a C program depending on this symbol and tries compiling it. This is a bit of info we miss in BuildInfo? - we have extra-libs, but no function names.

Error messages should be helpful for the typical Cabal user, who should not be assumed to know > anything about C libraries or linking.

This is a bit unclear to me - we have no info on required packages, we have only includes/extra-libs, how do we make error messages user-friendly? Autoconf doesn't do it.

[alexeevg]

Answering my own question:

This is a bit of info we miss in BuildInfo?? - we have extra-libs, but no function names.

Hmm, it seems that Autoconf is a little paranoid here, ld will fail if library is not found regardless of whether or not the program depends on it.

[duncan]

Replying to alexeevg:

Autoconf uses AC_CHECK_LIB macro to check if the library is present. AC_CHECK_LIB takes the library name and a function(symbol) name that is expected to be exported from the library, generates a C program depending on this symbol and tries compiling it. This is a bit of info we miss in BuildInfo? - we have extra-libs, but no function names.

Right, we do not have this additional information. As you say, we will have to rely on ld being paranoid and failing when it cannot find a library, even if the object files do not depend on any function exported by that library.

Actually, I think ld pretty much has to have this behavior since it does not know what symbols the missing library might export that would shadow symbols exported by other libraries. So it cannot actually determine if the library is not needed, at least for ELF linkers.

Error messages should be helpful for the typical Cabal user, who should not be assumed to know > anything about C libraries or linking.

This is a bit unclear to me - we have no info on required packages, we have only includes/extra-libs, how do we make error messages user-friendly? Autoconf doesn't do it.

You're right that we cannot easily or accurately map missing C libs back to which native OS packages that provide them. I think we can still produce reasonable error messages. We can at least explain which C libraries are missing, we must make it clear that these are C libraries (not Haskell packages) that we expected to find on the system. Then there are two possible causes with corresponding solutions:

• The most likely is that the C package is just not installed, or that the development flavour of the package is not installed (binary Linux distros often split packages into runtime and dev variants). We can recommend users look for a native package with a similar name to the C lib, and if there is a "-dev" version then install that. We could provide more detail and say the native package should provide a file libfoo.so or libfoo.a or whatever.

• Another possibility is that the C lib is installed but in some non-standard location. In that case we should tell users about the --extra-include-dirs and/or --extra-lib-dirs flags.

Just for my own future reference:

•  AC_CHECK_LIB
•  AC_CHECK_HEADER

[duncan]

Now partially done!

[duncan]

Thanks! Works great.

For example:

$ setup configure -O0
Configuring foo-1.0...
Warning: Required C header not found: foo.h
Warning: Required C library not found: bar
Warning: Required C library not found: baz

And adding --extra-include-dirs=./test/ --extra-lib-dirs=./test/ makes it work fine (with the test headers and lib files in ./test/ obviously).

Mon Jan 26 18:58:32 GMT 2009  gleb.alexeev@gmail.com
  * Warn if C dependencies not found (kind of fixes #262)
  
  This is just a basic check - generate a sample program and check if it compiles and links with relevant flags. Error messages (warning messages, 
  actually) could use some improvement.

Fri Jan 30 12:02:28 GMT 2009  Gleb Alexeyev <gleb.alexeev@gmail.com>
  * #262 iterative tests for foreign dependencies
  Optimize for succesful case. First try all libs and includes in one command, 
  proceed with further tests only if the first test fails. The same goes for lib
s 
  and headers: look for an offending one only when overall test fails.

The last thing I need to fix before we can make these errors rather than just warnings is doing the check in the right place for packages that use configure scripts. We can also make some improvements to the error messages but it's already a lot more informative than it used to be when it'd just fail half way through building.

[duncan]

The check is now done after running ./configure so we can pick up any settings it provides. Those settings (cpp-options, cc-options and ld-options) are also now taken into account. This means the check should now be accurate so we can make it fatal rather than a warning. This is important to help people identify the cause of the problem.

Sat Jan 31 18:22:13 GMT 2009  Duncan Coutts <duncan@haskell.org>
  * Do the check for foreign libs after running configure
  This lets us pick up build info discovered by the ./configure script

Sat Jan 31 18:40:16 GMT 2009  Duncan Coutts <duncan@haskell.org>
  * Use the cc, cpp and ld options when checking foreign headers and libs
  In partiular this is needed for packages that use ./configure
  scripts to write .buildinfo files since they typically do not
  split the cpp/cc/ldoptions into the more specific fields.

Sat Jan 31 18:48:13 GMT 2009  Duncan Coutts <duncan@haskell.org>
  * Improve the error message for missing foreign libs and make it fatal
  The check should now be accurate enough that we can make it an
  error rather than just a warning.

The error message now includes a bit more advice:

Configuring foo-1.0...
setup: Missing dependencies on foreign libraries:
* Missing header file: foo.h
* Missing C libraries: foo, bar, baz
This problem can usually be solved by installing the system packages that
provide these libraries (you may need the "-dev" versions). If the libraries
are already installed but in a non-standard location then you can use the
flags --extra-include-dirs= and --extra-lib-dirs= to specify where they are.

Or a real example:

Configuring curl-1.3.4...
setup: Missing dependency on a foreign library:
* Missing header file: curl.h
* Missing C library: curl
This problem can usually be solved by installing the system package that
provides this library (you may need the "-dev" version). If the library is
already installed but in a non-standard location then you can use the flags
--extra-include-dirs= and --extra-lib-dirs= to specify where it is.

Thanks to Gleb Alexeyev for implementing most of this. It should result in many fewer confused users.

[duncan]

I've built nearly every package on hackage to look for regressions. Two things need fixing:

We do not look for .h files in the root directory of the package. This breaks HsPerl5-0.0.6, cedict-0.2.5 and halfs-0.2.

We do not add the include dirs for dependent packages. This breaks hscurses-1.3.0.2, plugins-1.4.0 and unix-compat-0.1.2.1. Of these plugins-1.4.0 is not a good test case because it is broken for other reasons (the .h file it #includes cannot be included on its own).

The solution to the first issue should be easy. Just add . into the list of include dirs. The second requires copying the code from the pre-processor module where we do the same thing for hsc2hs. Perhaps it's possible to share the code.

[duncan]

Fixed.

Wed Feb 18 11:05:20 GMT 2009  Duncan Coutts <duncan@haskell.org>   
  * Use more cc options when checking for header files and libs
  Use -I. to simulate the search path that gets used when we tell ghc
  to -#include something. Also use the include dirs and cc options of
  dependent packages. These two changes fix about 3 packages each.

[elga]

*  simyo

[edouard]

A l'image du RIB bancaire, un code RIO est un identifiant unique attribue a la ligne et a un contrat client permettant d'identifier son demande de portabilite vers un autre operateur.  rio bouygues