Configurably use system-provided libffi

[nomeata]

Hi,

both Debian and Fedora patch ghc to use the system-wide installed libffi, as required by distribution policies:  http://anonscm.debian.org/darcs/pkg-haskell/experimental/ghc/patches/system-libffi  http://pkgs.fedoraproject.org/gitweb/?p=ghc.git;a=blob;f=ghc-use-system-libffi.patch

It would be helpful for us if we would not have to patch this, but rather trigger this by a ./configure flag.

Thanks, Joachim

[igloo]

It would be simpler if we can always do the same thing.

Perhaps we should look at whether OS X comes with libffi, and whether there's a bin tarball for mingw.

[slyfox]

Moreover, ghc does not embed RPATH to libffi for packages using it:

sf haskell-updater-1.2.0.5 # /usr/bin/ghc -package Cabal-1.14.0 --make /var/tmp/portage/app-admin/haskell-updater-1.2.0.5-r1/work/haskell-updater-1.2.0.5/Setup.lhs -dynamic -o setup
Linking setup ...
sf haskell-updater-1.2.0.5 # lddtree ./setup 
setup => ./setup (interpreter => /lib64/ld-linux-x86-64.so.2)
    libHSCabal-1.14.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/Cabal-1.14.0/libHSCabal-1.14.0-ghc7.4.1.so
    libHSprocess-1.1.0.1-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/process-1.1.0.1/libHSprocess-1.1.0.1-ghc7.4.1.so
    libHSpretty-1.1.1.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/pretty-1.1.1.0/libHSpretty-1.1.1.0-ghc7.4.1.so
    libHSdirectory-1.1.0.2-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/directory-1.1.0.2/libHSdirectory-1.1.0.2-ghc7.4.1.so
    libHSunix-2.5.1.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/unix-2.5.1.0/libHSunix-2.5.1.0-ghc7.4.1.so
    librt.so.1 => /lib64/librt.so.1
        libpthread.so.0 => /lib64/libpthread.so.0
    libutil.so.1 => /lib64/libutil.so.1
    libdl.so.2 => /lib64/libdl.so.2
    libHSbytestring-0.9.2.1-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/bytestring-0.9.2.1/libHSbytestring-0.9.2.1-ghc7.4.1.so
    libHSold-time-1.1.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/old-time-1.1.0.0/libHSold-time-1.1.0.0-ghc7.4.1.so
    libHSold-locale-1.0.0.4-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/old-locale-1.0.0.4/libHSold-locale-1.0.0.4-ghc7.4.1.so
    libHSfilepath-1.3.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/filepath-1.3.0.0/libHSfilepath-1.3.0.0-ghc7.4.1.so
    libHScontainers-0.4.2.1-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/containers-0.4.2.1/libHScontainers-0.4.2.1-ghc7.4.1.so
    libHSdeepseq-1.3.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/deepseq-1.3.0.0/libHSdeepseq-1.3.0.0-ghc7.4.1.so
    libHSarray-0.4.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/array-0.4.0.0/libHSarray-0.4.0.0-ghc7.4.1.so
    libHSbase-4.5.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/base-4.5.0.0/libHSbase-4.5.0.0-ghc7.4.1.so
    libHSinteger-gmp-0.4.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/integer-gmp-0.4.0.0/libHSinteger-gmp-0.4.0.0-ghc7.4.1.so
    libgmp.so.10 => /usr/lib64/libgmp.so.10
    libHSghc-prim-0.2.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/ghc-prim-0.2.0.0/libHSghc-prim-0.2.0.0-ghc7.4.1.so
    libHSrts-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/libHSrts-ghc7.4.1.so
        libffi.so.5 => not found
    libm.so.6 => /lib64/libm.so.6
    libc.so.6 => /lib64/libc.so.6

Notice libffi.so.5 => not found.

sf haskell-updater-1.2.0.5 # ls -l /usr/lib/ghc-7.4.1/libffi.so*
-rwxr-xr-x 1 root root 30832 апр.  14 21:20 /usr/lib/ghc-7.4.1/libffi.so
-rw-r--r-- 1 root root 30832 апр.  14 21:20 /usr/lib/ghc-7.4.1/libffi.so.5
-rw-r--r-- 1 root root 30832 апр.  14 21:19 /usr/lib/ghc-7.4.1/libffi.so.5.0.10
sf haskell-updater-1.2.0.5 # ls -l /usr/lib/libffi.so*
lrwxrwxrwx 1 root root    15 апр.  14 11:53 /usr/lib/libffi.so -> libffi.so.6.0.0
lrwxrwxrwx 1 root root    15 апр.  14 11:53 /usr/lib/libffi.so.6 -> libffi.so.6.0.0
-rwxr-xr-x 1 root root 30816 апр.  14 11:53 /usr/lib/libffi.so.6.0.0

[trommler]

I would also like to see this fixed for the ghc package I maintain at openSUSE. So I'd like to work on this.

[slyfox]

A note:

what is missing in fedora/debian patch is: - the real request for libffi's library and include path via pkg-config. - ./configure time convigurability

Gentoo, for example, allws multiple libffi headers in the following locations: /usr/lib64/libffi-3.0.11/include/ffi.h /usr/lib64/libffi-3.0.11/include/ffitarget.h

[trommler]

Replying to slyfox:

Gentoo, for example, allws multiple libffi headers in the following locations: /usr/lib64/libffi-3.0.11/include/ffi.h /usr/lib64/libffi-3.0.11/include/ffitarget.h

I will add a new option to ./configure called --with-system-libffi and have an optional parameter for it so you can specify which library/module you want to use for ffi.

Is that what you had in mind?

[trommler]

The patch validates on Linux with and without the new flag --with-system-libffi. Please validate on MacOS and Windows before pushing. I tried to validate on MacOS but failed to compile ghc-7.6 branch without the patch.

The patch adds three flags to configure:

--with-system-libffi use system provided libffi

--with-ffi-includes find includes for libffi in non-standard directory

--with-ffi-libraries find libraries for libffi in non-standard directory

[trommler]

Replying to slyfox:

Moreover, ghc does not embed RPATH to libffi for packages using it:

In fact ghc does not set any RPATH (or RUNPATH if you are using ELF's new dynamic tags) in the libraries that come with it. I relies Cabal setting the correct RPATH/RUNPATH in the binaries using a library.

As this is a more general issue with the way shared libraries are built (see for example trac #7062) and installed in ghc, I did not yet add a patch that just fixes the symptom.

[trommler]

Replying to trommler:

As this is a more general issue with the way shared libraries are built (see for example trac #7062) and installed in ghc, I did not yet add a patch that just fixes the symptom.

In HEAD linking of shared libraries now has been changed to embed RPATH/RUNPATH (see #3072) so I will prepare another patch to embed RPATH/RUNPATH for libffi.

[igloo]

See also #4496

[ian@…]

commit 3005e90936c47c1f71672bf6c84fff20cb14014b

Author: Ian Lynagh <ian@well-typed.com>
Date:   Thu Nov 29 22:22:39 2012 +0000

    Add configure option to use system provided libffi; fixes #5743
    
    Based on patch from Peter Trommler:
    
        From 293495d40f62e691520331a41c6d85d82e120169 Mon Sep 17 00:00:00 2001
        From: Peter Trommler <ptrommler@acm.org>
        Date: Sun, 21 Oct 2012 18:47:01 +0200
        Subject: [PATCH] Add configure option to use system provided libffi This
         fixes track # 5743 and #4496.

 configure.ac        |   55 +++++++++++++++++++++++++++++++++++++++++++++++++++
 ghc.mk              |   11 ++++++++-
 mk/config.mk.in     |    7 ++++++
 rts/ghc.mk          |   42 ++++++++++++++++++++++++++++++++++----
 rts/package.conf.in |    3 ++
 5 files changed, 111 insertions(+), 7 deletions(-)

[igloo]

In #4496 I said:

On the Mac I have /usr/lib/libffi.dylib and /usr/include/ffi/*.h.

However, I can't see any mingw tarballs that seem likely to include it.

I can't see any sign of libffi in mingw64 either.

I really don't like having this be conditional, but I can't see a better alternative, so I've applied the patch. Thanks.

I'm not sure if it's worth listing the new flag explicitly on  http://hackage.haskell.org/trac/ghc/wiki/Building/Using#Runtheconfigurescript as it'll only really be of interest to a handful of packagers, and that page only tries to list the most common flags, leaving the rest to ./configure --help.

[PHO]

Your patch mentions a variable LIBFFI_LIBS, which isn't defined anywhere. I think you need something like:

diff --git a/rts/ghc.mk b/rts/ghc.mk
index e3c9fa6..d3638ac 100644
--- a/rts/ghc.mk
+++ b/rts/ghc.mk
@@ -386,12 +386,14 @@ rts/dist/build/AutoApply_HC_OPTS += -fno-PIC -static
 endif
 endif
 
-# add CFLAGS for libffi
+# add CFLAGS and LIBS for libffi
 # ffi.h triggers prototype warnings, so disable them here:
 ifeq "$(UseSystemLibFFI)" "YES"
 LIBFFI_CFLAGS = $(addprefix -I,$(FFIIncludeDir))
+LIBFFI_LIBS   = $(addprefix -L,$(FFILibDir)) -lffi
 else
 LIBFFI_CFLAGS =
+LIBFFI_LIBS   = -Lrts/dist/build -lffi
 endif
 rts/Interpreter_CC_OPTS += -Wno-strict-prototypes $(LIBFFI_CFLAGS)
 rts/Adjustor_CC_OPTS    += -Wno-strict-prototypes $(LIBFFI_CFLAGS)

[trommler]

Replying to PHO:

Your patch mentions a variable LIBFFI_LIBS, which isn't defined anywhere. I think you need something like:

This was part of my original patch but did not make it into the above commit:

ifneq "$(UseSystemLibFFI)" "YES"
LIBFFI_LIBS = -Lrts/dist/build -lffi 
else
LIBFFI_LIBS =
endif

In the case of UseSystemLibFFI=="YES" the correct linker flags will be set as part of rts/libs.depend, hence the empty definition for LIBFFI_LIBS.

Tests driver dynHelloWorld, dynlibs T3807 and T5373 fail here with unresolved symbols from ffi when built with the in-tree libffi.

I will reopen, assign to me, and prepare a patch.

[trommler]

Fix in-tree libffi linking

[PHO]

Replying to trommler:

In the case of UseSystemLibFFI=="YES" the correct linker flags will be set as part of rts/libs.depend, hence the empty definition for LIBFFI_LIBS.

I get it. But then what about FFILibDir? If I understand this correctly, rts/libs.depend contains only -l flags so -L$(FFILibDir) won't be passed to the linker.

Apart from that, $(rts_LD_OPTS) seems to be ignored too. Or am I missing some point?

[ptrommler@…]

commit 7ee5bedc5e5039c9bb3ba06b50a4395e579a4b0e

Author: Peter Trommler <ptrommler@acm.org>
Date:   Fri Nov 30 13:40:10 2012 +0100

    Link in-tree libffi to rts. Fixes trac #5743.

 rts/ghc.mk |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

[igloo]

I've applied the patch, thanks. I think we only use rts_LD_OPTS when calling ld directly.

If you have any problems building, it's probably best to open a new ticket giving details, as this one is starting to get a little large and most of the history probably won't be relevant.