let ./configure check for a GNUreadline framework

[maeder@…]

on mac readline dependencies -- like those for GMP -- can be satisfied by lib-files or by frameworks. Build this into the configure step of libraries/readline.

[igloo]

We're using the 6.6.2 milestone for bugs that will only be done if we decide to do 6.6.2 after all. This sounds like 6.8 should have it too, so I'm remilestoning it; feel free to put it back if I'm wrong.

[igloo]

Fixed by:

Thu Sep  6 06:26:28 BST 2007  judah.jacobson@gmail.com
  * Link against GNUreadline.framework, if it's available.
  This can be overridden by setting --with-readline-includes or 
  --with-readline-libraries.

[guest]

It would be nice if also the frameworks under $HOME/Library/Frameworks are checked (and used) when building ghc. This requires an extra flag -F$HOME/Library/Frameworks.

[igloo]

Does this make a runtime dependency on frameworks in $HOME/Library/Frameworks? If so then this sounds like it could cause problems for a multi-user system.

[guest]

Replying to igloo:

Interesting question. I've created the mac binary dists with a local GNUreadline framework. I hope my home directory wasn't left as framework search path in the ghc-6.6.1 binary. "man ld" on Mac did not give me a clue about this. (There's nothing like a RUN_LIBRARY_PATH.)

       -search_paths_first
              By  default  when  the  -dynamic  flag is in effect, the -lx and
              -weak-lx  options  first  search  for  a  file   of   the   form
              `libx.dylib'  in each directory in the library search path, then
              a file of the form `libx.a'  is  searched  for  in  the  library
              search  paths.   This  option  changes  it  so that in each path
              `libx.dylib' is searched for then `libx.a' before the next  path
              in the library search path is searched.

       -framework name[,suffix]
              Specifies  a  framework  to link against. Frameworks are dynamic
              shared libraries, but they are stored  in  different  locations,
              and therefore must be searched for differently. When this option
              is specified, ld searches  for  framework  `name.framework/name'
              first  in  any directories specified with the -F option, then in
              the standard framework  directories  /Library/Frameworks,  /Net-
              work/Library/Frameworks,  and  /System/Library/Frameworks.   The
              placement of the -framework option is significant, as it  deter-
              mines  when  and how the framework is searched.  If the optional
              suffix is specified the framework is first searched for the name
              with the suffix and then without.

       -weak_framework name[,suffix]
              This  is the same as the -framework name[,suffix] but forces the
              framework and all references to it to be marked as weak imports.
              Care  must  be  taken when using this as the use of the non-weak
              symbol references in an object file may  cause  the  program  to
              crash  when  the symbol or framework is not present at execution
              time.

       -Fdir  Add dir to the list of directories in which to search for frame-
              works.   Directories  specified  with -F are searched before the
              standard framework directories.

[guest]

I didn't find any dependencies in my binaries created with local frameworks (using grep and otool). Christian

[guest]

In order to use my local GNUreadline framework I had to change libraries/readline/configure.ac (attached) and libraries/readline/include/HsReadline.h. The case distinction in the latter file did not work because recursively included files like readline/rlstdc.h did not have the GNUreadline/ prefix.

(I wonder if (or why) this worked for a global GNUreadline.framework in /Libary/Frameworks/.)

So I left HsReadline.h as:

#ifndef HSREADLINE_H
#define HSREADLINE_H

#include "HsReadlineConfig.h"
#include <stdio.h>
#include <readline/readline.h>
#include <readline/history.h>

extern void hs_rl_message (const char *s);

#endif

Regarding configure.ac I have no idea how to best check first for a global framework and then for a local one and if READLINE_CFLAGS and READLINE_LDFLAGS need to be extended or can be simply set.

I'm also not sure if the modified file HsReadline.h works now with a global framework or if READLINE_CFLAGS has to be set to -I/Library/Frameworks/GNUreadline.framework/Headers, too.

[guest]

While the above was sufficient to build the readline package, the stage2 ghc could not load it:

compiler/stage2/ghc-inplace --interactive -package readline
GHCi, version 6.8.0.20071017: http://www.haskell.org/ghc/  :? for help
Loading package base ... linking ... done.
Loading package old-locale-1.0 ... linking ... done.
Loading package old-time-1.0 ... linking ... done.
Loading package filepath-1.1 ... linking ... done.
Loading package directory-1.0 ... linking ... done.
Loading package unix-2.2 ... linking ... done.
Loading package process-1.0 ... linking ... done.
Loading package readline-1.0.1 ... ghc-6.8.0.20071017: can't load framework: GNUreadline (not found)

ghc-inplace does not depend on the GNUreadline.framework:

otool -L compiler/stage2/ghc-inplace
compiler/stage2/ghc-inplace:
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.3.9)
        GMP.framework/Versions/A/GMP (compatibility version 7.0.0, current version 7.0.0)

[guest]

The problem is the frameworks entry in driver/package.conf.inplace for readline. If I delete it everything is fine. Furthermore the ccOptions and ldOptions entries should not point to my home directory!

[guest]

I was not able build a correct binary dist with a local GNUreadline framework, because I could not pass cc- and ld-Flags to the build process that don't end up in driver/package.conf.inplace.

Furthermore in compiler/ghci/Linker.lhs the home directory of the current user should be added to defaultFrameworkPaths (line 1141) in order to avoid an unjustified failure message (without even trying to load the framework, which succeeds because it is already loaded by the ghc binary).

[guest]

Instead of adding and searching the user's home directory is suffices in compiler/ghci/Linker.lhs (line 1136) to simply ignore the unjustified failure by:

Nothing       -> return Nothing

[judah]

Replying to guest:

Interesting question. I've created the mac binary dists with a local GNUreadline framework. I hope my home directory wasn't left as framework search path in the ghc-6.6.1 binary. "man ld" on Mac did not give me a clue about this. (There's nothing like a RUN_LIBRARY_PATH.)

I did some research, and figured out what's going on. Loading at runtime is handled by dyld, not ld; in fact, the -F flag is irrelevant to how dyld works. Instead, it uses the environmental variables DYLD_FRAMEWORK_PATH and DYLD_FALLBACK_FRAMEWORK_PATH. The latter defaults to searching $HOME/Library/Frameworks as well as /Library/Frameworks et. al.

As a result, binaries can always load frameworks stored in either the home or root locations, regardless of the flags passed to the compiler at build time.

See my page OSXFrameworks for more info.

[thorkilnaur]

There is some additional discussion on #1931 which I closed as a duplicate of this.

Best regards Thorkil

[maeder]

With Judah's new framework I can install also the Shellac-readline package (probably because readline/history.h is found in /usr/include/ by chance)

As mentioned above (comment 11) compiler/ghci/Linker.lhs should not fail if it does not find a framework, because ghci itself has loaded the frameworks GMP and GNUreadline already (possibly from the home directory).

(Only) after this simple change GNUreadline should be mentioned as framework in the package.conf file (as it is already the case for the GMP framework, that happens to be under /Library/Frameworks for me, too.)

[maeder]

readline/configure.ac to look for frameworks under $HOME

[maeder]

check for frameworks under home directory

[maeder]

Could some check in my changes to compiler/ghci/Linker.lhs (attached) which I need for ghci -package readline? The diff is:

@@ -74,6 +74,7 @@
 import Foreign

 import System.IO
+import System.IO.Error as IO
 import System.Directory

 import Control.Exception
@@ -1129,7 +1130,12 @@
 -- name. They are searched for in different paths than normal libraries.
 #ifdef darwin_TARGET_OS
 loadFramework extraPaths rootname
-   = do        { mb_fwk <- findFile mk_fwk (extraPaths ++ defaultFrameworkPaths)
+   = do        { either_dir <- IO.try getHomeDirectory
+        ; let homeFrameworkPath = case either_dir of
+                Left _ -> []
+                Right dir -> [dir ++ "/Library/Frameworks"]
+        ; mb_fwk <- findFile mk_fwk
+            (extraPaths ++ homeFrameworkPath ++ defaultFrameworkPaths)
        ; case mb_fwk of
            Just fwk_path -> loadDLL fwk_path
            Nothing       -> return (Just "not found") }

(I just see that the file uses tabs). My addition allows to find a framework in a users home directory.

My attached configure.ac is also needed for building ghc with frameworks under $HOME. (I'm just trying if compiler/main/DriverPipeline.hs can be extended with -F$HOME/Library/Frameworks.)

[thorkilnaur]

To solve the problem of the "inner" .h files using #include <readline/...h>, why not call the framework "readline" instead of "GNUreadline" and do away with the intermediate "readline" directory? To verify this idea, I created the following structure, by hand:

Thorkil-Naurs-Computer:/Library/Frameworks thorkilnaur$ ls -lR /Library/Frameworks/readline.framework
total 1080
drwxr-xr-x   10 thorkiln  admin     340 Dec 10 20:14 Headers
-rwxr-xr-x    1 thorkiln  admin  551468 Dec 10 20:19 readline

/Library/Frameworks/readline.framework/Headers:
total 152
-rw-r--r--   1 thorkiln  admin   4684 Dec 10 20:14 chardefs.h
-rw-r--r--   1 thorkiln  admin  10207 Dec 10 20:14 history.h
-rw-r--r--   1 thorkiln  admin   3529 Dec 10 20:14 keymaps.h
-rw-r--r--   1 thorkiln  admin  34515 Dec 10 20:14 readline.h
-rw-r--r--   1 thorkiln  admin   2305 Dec 10 20:14 rlconf.h
-rw-r--r--   1 thorkiln  admin   1503 Dec 10 20:14 rlstdc.h
-rw-r--r--   1 thorkiln  admin   2977 Dec 10 20:14 rltypedefs.h
-rw-r--r--   1 thorkiln  admin   3167 Dec 10 20:14 tilde.h
Thorkil-Naurs-Computer:/Library/Frameworks thorkilnaur$ 

With this in place, I was able to build the Haskell readline library without any changes.

Am I overlooking something here?

Best regards Thorkil

[judah]

Replying to thorkilnaur:

To solve the problem of the "inner" .h files using #include <readline/...h>, why not call the framework "readline" instead of "GNUreadline" and do away with the intermediate "readline" directory?

I've looked into this in the past; if another version of readline.h is available (e.g., /usr/include/readline/readline.h, which is where libedit was stored by default on my iMac), that file will be used instead of readline.framework/Headers/readline.h. Worse, even if you specify -F, gcc will always use /usr/include instead of the framework.

Another thought: can Shellac-readline just use "HsReadline.h" instead of <readline/readline.h>?

[judah]

To clarify my previous comment: The framework needs to be called GNUreadline.framework so that we say #include <GNUreadline/*>.

If we called the framework readline.framework, then we'd need the statement #include <readline/*> which uses /usr/include/readline/readline.h if it exists.

[judah]

FYI, please note my proposal for Cabal to add -F automatically at:  http://hackage.haskell.org/trac/hackage/ticket/189.

[thorkilnaur]

It is true, as the following session shows, that gcc blocks direct attempts to place /Library/Frameworks before /usr/include in the list of directories to be searched for header files:

$ cat readlineT2.c 
#include <readline/readline.h>
int main() { return 0; }
$ gcc readlineT2.c -c -v -F /Library/Frameworks 
Using built-in specs.
Target: powerpc-apple-darwin8
Configured with: /private/var/tmp/gcc/gcc-5250.obj~12/src/configure --disable-checking -enable-werror --prefix=/usr --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-transform-name=/^[cg][^.-]*$/s/$/-4.0/ --with-gxx-include-dir=/include/c++/4.0.0 --build=powerpc-apple-darwin8 --host=powerpc-apple-darwin8 --target=powerpc-apple-darwin8
Thread model: posix
gcc version 4.0.1 (Apple Computer, Inc. build 5250)
 /usr/libexec/gcc/powerpc-apple-darwin8/4.0.1/cc1 -quiet -v -F /Library/Frameworks -D__DYNAMIC__ readlineT2.c -fPIC -quiet -dumpbase readlineT2.c -auxbase readlineT2 -version -o /var/tmp//ccgWIl6U.s
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/powerpc-apple-darwin8/4.0.1/../../../../powerpc-apple-darwin8/include"
ignoring duplicate directory "/Library/Frameworks"
  as it is a non-system directory that duplicates a system directory
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/powerpc-apple-darwin8/4.0.1/include
 /usr/include
 /System/Library/Frameworks
 /Library/Frameworks
End of search list.
...

However, we can do as follows and get the desired effect:

$ mkdir localFrameworks
$ ln -s /Library/Frameworks/readline.framework localFrameworks/
$ gcc readlineT2.c -c -v -F localFrameworks    
Using built-in specs.
Target: powerpc-apple-darwin8
Configured with: /private/var/tmp/gcc/gcc-5250.obj~12/src/configure --disable-checking -enable-werror --prefix=/usr --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-transform-name=/^[cg][^.-]*$/s/$/-4.0/ --with-gxx-include-dir=/include/c++/4.0.0 --build=powerpc-apple-darwin8 --host=powerpc-apple-darwin8 --target=powerpc-apple-darwin8
Thread model: posix
gcc version 4.0.1 (Apple Computer, Inc. build 5250)
 /usr/libexec/gcc/powerpc-apple-darwin8/4.0.1/cc1 -quiet -v -F localFrameworks -D__DYNAMIC__ readlineT2.c -fPIC -quiet -dumpbase readlineT2.c -auxbase readlineT2 -version -o /var/tmp//cc7uOQDF.s
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/powerpc-apple-darwin8/4.0.1/../../../../powerpc-apple-darwin8/include"
#include "..." search starts here:
#include <...> search starts here:
 localFrameworks
 /usr/lib/gcc/powerpc-apple-darwin8/4.0.1/include
 /usr/include
 /System/Library/Frameworks
 /Library/Frameworks
End of search list.
...

Of the several inconvenient ways that we have for dealing with this problem, this latter one is my favorite.

Best regards Thorkil

[maeder]

With the attached files compiler/main/DriverPipeline.hs and utils/hsc2hs/Main.hs I was able to build ghc-6.8.1.20071206 with a local GNUreadline framework. (No further change to Cabal was necessary, so far.)

[thorkilnaur]

I am a bit confused about what the subject of this trac ticket really is. To reduce my confusion, I suggest that we limit the issue of the present ticket to things that involve changing the Haskell readline library and/or it's build process. Other related changes (such as the proposed changes to the ghc linker) are then handled in separate trac tickets (such as #1798).

I hope that you consider this acceptable, but please feel free to shout in any case. I have also added a comment to #1798.

Best regards Thorkil

[maeder]

The two files compiler/main/DriverPipeline.hs and utils/hsc2hs/Main.hs are not ready for check in. Both files miss catching an error of getHomeDirectory and the relevant parts of utils/hsc2hs/Main.hs should be put between cpp directives. (I'm also not sure if the use of getHomeDirectory prevents ghc being bootstrapped with some earlier ghc versions.)

#ifdef darwin_TARGET_OS
    hdir <- IO.try getHomeDirectory
#endif

Only after these changes it makes sense to check in readline/configure.2.ac.

[maeder]

frameworks moved up for the linker

[maeder]

In the newly attached file, compiler/main/DriverPipeline.hs, I've removed tabs and trailing white space, used IO.try when calling getHomeDirectory, and moved frameworks for the linker before other package libraries (because I got a link error otherwise).

[maeder]

Link error with older DriverPipeline?.hs

[maeder]

My modification of utils/hsc2hs/Main.hs (probably) caused the problem reported in  http://article.gmane.org/gmane.comp.lang.haskell.cafe/34373

[guest]

Replying to maeder:

My modification of utils/hsc2hs/Main.hs (probably) caused the problem reported in  http://article.gmane.org/gmane.comp.lang.haskell.cafe/34373

Cabal already handles this type of problem by passing different flags based on whether hsc2hs is using gcc or ghc. This is done in Distribution.Simple.PreProcess.ppHsc2hs and Simple.Program.hsc2hsProgram.

Perhaps Cabal should add the -F$HOME/Library/Frameworks instead of hsc2hs itself? See  http://hackage.haskell.org/trac/hackage/ticket/189

[guest]

By the way; I wrote the previous comment while logged in as guest, but forgot to sign it.

-Judah

[maeder]

Instead of modifying hsc2hs it is possible to add

                   --hsc2hs-options="-C -F$(HOME)/Library/Frameworks" \

to libraries/Makefile (as line 246). I'll attach a patch that works on other platforms, too.

[maeder]

I've created #2021 for the DriverPipeline.hs issue. The subject of this ticket can be settled by committing libraries/readline/configure.ac and libraries/Makefile afterwards.

[maeder]

./configure checks for the GNUreadline framework so let's close this ticket

(changes for frameworks in home dirs can be discussed elsewhere)