readline/rlstdc.h not found (and other errors) when building readline on Mac OS X with framework GNUreadline

[thorkilnaur]

When building readline on a Mac OS X with the GNUreadline framework in /Libraries/Frameworks, the build fails:

$ runhaskell Setup.hs build
Preprocessing library readline-1.0.1...
Building readline-1.0.1...
In file included from include/HsReadline.h:7,

                 from HsReadline_cbits.c:1:0:

/Library/Frameworks/GNUreadline.framework/Headers/readline/readline.h:36:31:
     error: readline/rlstdc.h: No such file or directory
...

This is in spite of:

$ runhaskell Setup.hs configure
Configuring readline-1.0.1...
...
checking for GNUreadline.framework... checking for readline... yes
checking for rl_readline_version... yes
checking for rl_begin_undo_group... yes
checking for rl_erase_empty_line... yes
checking for rl_free_undo_list... yes
checking for rl_completion_word_break_hook in -lreadline... yes
...
$

This problem affects the tnaur osx buildbot slaves after I have installed the GNUreadline framework in /Libraries/Frameworks. See  http://darcs.haskell.org/buildbot/all/tnaur%20PPC%20OSX%20stable/builds/29/step-stage1/0 for an example of this. Earlier, the tnaur osx buildbot slaves have have avoided this problem by simply not detecting readline and, therefore, never attempting to build the readline library.

The enclosed patch introduces a fix for this, changing the readline.cabal file. Please note that this is included for illustration only, it is not intended to be the solution to this problem.

Best regards Thorkil

[maeder]

I noticed this problem, too (#1395, comment 7). Adding a "GNUreadline/" prefix to some include lines was not sufficient and should be reverted! Setting include-dirs in readline.cabal is only a solution if the framework is (always) installed under /Library/Frameworks. (It would not work with my local framework.)

When I built ghc-6.8.1 on mac I've put the following into mk/build.mk:

SRC_HC_OPTS += -optl-F/home/maeder/Library/Frameworks -optl-framework -optlGNU
readline
CC_OPTS += -I/home/maeder/Library/Frameworks/GNUreadline.framework/Headers

I've also added the following last line to the target stamp/configure.library.build$(CONFIGURE_STAMP_EXTRAS).% in libraries/Makefile so that cabal knows how to include my header files:

                  --with-hsc2hs=../../utils/hsc2hs/hsc2hs-inplace \
                  --hsc2hs-options="-C $(CC_OPTS)" \

(A GNUreadline framework indicator is missing in my distro's package.conf, because configure did not recognize it, anyway.)

See also #1798

[judah]

I'm looking into this.

[judah]

The problem is with GNUreadline.framework itself. The files readline.h, history.h etc. all import <readline/rlstdc.h> instead of <GNUreadline/readline/rlstdc.h>. The latter is needed to correctly import from the framework when compiling the readline package..

If some other version of readline were installed at e.g. /usr/local/include/readline, then headers from that directory would be used instead of throwing an error. My machine was set up like that, which is why I didn't see this issue when I fixed #1395.

I've edited, tested and attached the script that generates the framework. My rebuild of the framework is at  http://www.math.ucla.edu/~jjacobson/ghc/GNUreadline-framework.zip. (The framework contains the script build-framework-R.sh inside itself.)

Thorkil, can you please test this on your buildbots and see if it fixes the problem? If so, we should probably start distributing this new version of the framework along with 6.8.2.

Christian, I took a look at your comments on #1395 and #1798; with this change you shouldn't need -I/Library/Frameworks/GNUreadline.framework/Headers anymore. Of course, for now you'll still need to use -F if you want to store frameworks in your home directory.

[judah]

A new version of the script that generates GNUreadline.framework.

[judah]

A diff against the old version of the script.

[maeder]

As I said in #1798, I think, the compiler should set the plain (and portable) include path directly rather than that we change all includes to conditional ones (and also have to make sure that the condition is properly set). User code with includes would have to got through this pain, too!

[maeder]

For instance, I see in the Shellac-readline package a source line:

foreign import ccall "readline/history.h clear_history" clear_history :: IO ()

Would that work with your solution?

[maeder]

readline/history.h will be found in /usr/include/. So the above goes through by chance.

[judah]

Christian, I agree that conditional compilation in Haskell libraries is undesirable, but the problem here is with header files in the framework itself, not with libraries that #import the framework.

My proposed fix was that if a header file is part of GNUreadline.framework, it should *always* (not conditionally) import other readline headers as <GNUreadline/*>. That's necessary given how GHC currently behaves; and it will still work once #1798 is implemented (although that proposal could make this change unnecessary).

Given that #1798 won't be done until 6.10 at the earliest, is there any problem with making this change for now?

[maeder]

Hi Judah, your change will not make a problem (I hope). Upload a new framework http://www.haskell.org/ghc/dist/mac_frameworks/GNUreadline-framework.zip

[maeder]

Maybe my above CC_OPTS will not longer work because the GNUreadline/ prefix cannot be resolved without a switch that points to my local framework.

[maeder]

As I said above, my distros do not contain the framework indicator in the package.conf file. This, sadly, requires that at least "-optl-framework -optlGNUreadline" needs to be passed to ghc explicitely for linking . However, ghci will fail with this flag (or a framework indicator in package.conf) if the framework is not found under /Library/Frameworks/, even if also the proper framework path is supplied, because ghci uses compiler/ghci/Linker.lhs.

So a distribution that contained the framework GNUreadline in the package.conf file would be ghci-unusable for me (who is not allowed to copy the GNUreadline.framework to /Library/Frameworks/)

[thorkilnaur]

Sorry about opening this ticket without investigating more thoroughly: Having looked at #1798 and #1395, this problem seems better handled under #1395. In any case I suggest that the basic problem of locating the readline include files should be solved under Mac OS X by using the gcc -F option:

$ man gcc
...
       -Fdir
           Add the framework directory dir to the head of the list of directo-
           ries to be searched for header files.  These directories are inter-
           leaved with those specified by -I options and are scanned in a
           left-to-right order.

           A framework directory is a directory with frameworks in it.  A
           framework is a directory with a "Headers" and/or "PrivateHeaders"
           directory contained directly in it that ends in ".framework".  The
           name of a framework is the name of this directory excluding the
           ".framework".  Headers associated with the framework are found in
           one of those two directories, with "Headers" being searched first.
           A subframework is a framework directory that is in a framework's
           "Frameworks" directory.  Includes of subframework headers can only
           appear in a header of a framework that contains the subframework,
           or in a sibling subframework header.  Two subframeworks are sib-
           lings if they occur in the same framework.  A subframework should
           not have the same name as a framework, a warning will be issued if
           this is violated.  Currently a subframework cannot have subframe-
           works, in the future, the mechanism may be extended to support
           this.  The standard frameworks can be found in "/Sys-
           tem/Library/Frameworks" and "/Library/Frameworks".  An example
           include looks like "#include <Framework/header.h>", where Framework
           denotes the name of the framework and header.h is found in the
           "PrivateHeaders" or "Headers" directory.
...

I have not looked at Cabal in detail, nor have I figured out how Cabal in detail is used by the readline package. But it would seem possible to set up the readline package to use Cabal to perform the necessary checks (it already checks for a GNUreadline package, this is harmless on non-Mac OS X'es, hence may be considered portable) and then, depending on the system, to pass the required -F option to gcc. I don't think ghc has such a direct option, but something like -optc-FGNUreadline would probably do the trick.

I will take the liberty of actually closing this ticket as a duplicate of #1395.

[maeder]

The -F option has the disadvantage that user code like that of my comment 5 (from Shellac-readline) above does not go through on Macs without change

[thorkilnaur]

The basic problem seems to be that Mac OS X'es have this additional and different way of setting up paths for include files and libraries. Essentially, this forces a program like GHC to more or less duplicate this mechanism. And to a certain extent this has happened: GHC has a -framework option to pass to the Mac linker and a -framework-path option to pass something to the Mac linker -F option.

But GHC doesn't have an option that matches the Mac gcc -F option: GHC doesn't do any Mac-like framework management similar to what the Mac gcc does. And that is really what you are missing, if I understand your comment correctly.

So in that situation, I still believe that it is preferable to let the Cabal-packaging of the readline library handle these differences. Likewise, various fixes could probably be concocted to solve the Shellac-readline problem (such as the use of symbolic links or installing the GNU readline library in one of the places that it is normally placed in).

Best regards Thorkil

[thorkilnaur]

I have now verified that the modified GNUreadline library by judah (thanks, also thanks for your kind remainder) allows the build of the Haskell readline library to proceed further. Compare  http://darcs.haskell.org/buildbot/all/tnaur%20PPC%20OSX%20head%202/builds/2/step-stage1/0 and  http://darcs.haskell.org/buildbot/all/tnaur%20PPC%20OSX%20head%202/builds/1/step-stage1/0 where you should note that these builds are haunted by other problems being investigated (#1843).

Best regards Thorkil