Complete shared library support

[simonmar]

This ticket is a placeholder for the shared libraries work, which is mostly working but needs to be completed (binary distributions, issues with -rpath etc.).

[simonmar]

I've made some progress on this, but it doesn't look like I'll have time to finish it for 6.10.1. Since it doesn't involve API changes, we could possibly get it into a later 6.10 release, though.

[clemens]

I fixed the following for Linux/x86_64 (probably also other archs) in today's HEAD "./configure --enable-shared; make" should compile error free. It should give you all libraries as shared objects, including a stage2 GHC package. The GHC package should be linkable into a working ghc binary (including a wrapper) that also works in interactive mode. Running the testsuits gives a few isolated regressions in ghci and template haskell I still have to dig into.

I won't go into details about the Windows status. At the moment, linking the RTS dies with unresolved RTS imp* symbols. Linking the RTS with its own import lib (libHSrts*.dll.a) should fix the problem, but I haven't got there yet.

However, for libgmp on Windows, we need to apply the same trick as with libffi, namely change it into a Haskell package. Most of the hacking was already done for libffi, so the changes from libffi/Makefile could be worked into gmp/Makefile without much trouble, I think.

In any case, I'd like to argue that we should give priority to Linux dynamic linking. With this I mean that we ship dynamic lib support even if it's just for this single architecture. This will help to build up a small user base that will hopefully report regressions as soon as they pop up. Also Linux is the most hassle free OS when it comes to dynamic library linking, so it is easier to establish that as base camp for further climbing.

Except for libffi change, most of the changes are quite minimal. We could probably target 6.10.2 as milestone for merging this into stable.

On Linux x86-64 dynamic at the moment:

Unexpected failures:
   2014(normal)
   T2386(normal)
   T2597a(normal)
   T2597b(normal)
   TH_genEx(normal)
   TH_spliceD1(normal)
   TH_spliceDecl3(normal)
   TH_spliceDecl4(normal)
   TH_spliceE5(normal)
   prog001(ghci)
   prog002(ghci)
   prog003(ghci)

[simonmar]

Some issues to resolve before we can deploy shared libraries. Some of these are probable just questions to be answered, but hint at documentation that we need to write.

• Do GHC distributions contain both shared and static libraries?
• Will Cabal build both shared and static libraries by default?
• Can a module built with -dynamic link against static libraries, if the dynamic versions are not available?
• If modules are compiled with -fPIC and/or -dynamic, can they then be linked statically? On which platforms? If not, what goes wrong if you try this?
• On x86_64 we should have -dynamic turned on by default, for loading modules into GHCi, otherwise data references to shared libraries will only have 32-bit relocations and thus will break.
• Using -static at compile-time on x86_64 should therefore come with a warning that the modules cannot be loaded into GHCi. Can we detect this and report an error in GHCi?
• What is the performance overhead of turning on -dynamic by default on x86_64?
• What is the performance overhead (and code-size overhead) of shared libraries in general?

[juhpetersen]

Is there anything one can do to help with testing this?

[simonmar]

Grab the GHC sources, configure with --enable-shared, build as normal and report problems that you encounter.

[clemens]

We need a buildbot for regression testing. I asked a few times on #ghc without much response. Probably I wasn't persistant enough to chase after those responsible.

I haven't look at most recent HEAD but frankly, I would be surprised if --enable-shared still works out of the box, where I left it a few months ago. Also I would be surpised if the number of failing test cases given above would have remained constant. Honestly, there is more exciting stuff around than hacking on shared libs.

If someone wants to take the job, this one should feel free to reassign the bug to himself.

[igloo]

We'll make it possible to build it in parallel with the non-shared GHC in the new build system, and then we won't need a dedicated buildbot for it.

[simonmar]

To my mind this is stalled not just due to lack of effort, but there's some design work to be done - mainly around how distributions, and how Cabal and cabal-install will work with shared libraries. See my list of questions above ( http://hackage.haskell.org/trac/ghc/ticket/1876#comment:11) for a start.

The other thing that worries me is that without having support for fixed ABIs, we could have more problems with shared libraries, since when someone reinstalls a library they will break not only dependent packages, but also existing dynamically-linked binaries.

[clemens]

Simon, I didn't see any serious attempts to get buildbots going. Iirc I fixed "make install" to that extent where regression tests could have been run.

Also I do not see any reason, how establishing base line quality is related to deployment upgrading and installation lifetime issues. If GHC doesn't have a way of regression testing shared libs in whatever conceptual broken they may be now than GHC will fail to deliver. That's simply because shared libs will bitrot in the ongoing development.

As I can't put this bug into an unassigned status, I hope simon you don't mind if I reassign it to you.

[simonmar]

Replying to clemens:

Simon, I didn't see any serious attempts to get buildbots going. Iirc I fixed "make install" to that extent where regression tests could have been run. Also I do not see any reason, how establishing base line quality is related to deployment upgrading and installation lifetime issues. If GHC doesn't have a way of regression testing shared libs in whatever conceptual broken they may be now than GHC will fail to deliver. That's simply because shared libs will bitrot in the ongoing development.

You're right, nothing prevents us from setting up buildbots (except lack of time). But I would really like to have a clear plan of action for getting us to the goal of having shared libs deployed. It's probably not difficult, but someone/people need to sit down and seriously think about how all this is going to work.

[juhpetersen]

Replying to simonmar:

Grab the GHC sources, configure with --enable-shared, build as normal and report problems that you encounter.

Ok, sorry for the delay. I installed libffi-devel on Fedora rawhide and got quite far this time with stable:

: : /home/petersen/ghc-6.10.1.20090303/utils/installPackage/install-inplace/bin/installPackage install '/home/petersen/ghc-6.10.1.20090303/utils/ghc-pkg/install-inplace/bin/ghc-pkg' 'XXX/package.conf' "" \

/home/petersen/ghc-6.10.1.20090303/ghc/stage2-inplace \ /home/petersen/ghc-6.10.1.20090303/ghc/stage2-inplace \ '$prefix' \ '/home/petersen/ghc-6.10.1.20090303/inplace-datadir' \ '$prefix/libexec' \ '$prefix/dynlib' \ '/home/petersen/ghc-6.10.1.20090303/inplace-datadir' \ '$prefix/doc' \ '$prefix/html' \ '$prefix/haddock' \ --distpref dist-stage2 \ --disable-executable-stripping \ --enable-shell-wrappers

Installing executable(s) in /home/petersen/ghc-6.10.1.20090303/ghc/stage2-inplace make[3]: Leaving directory `/home/petersen/ghc-6.10.1.20090303/ghc' make[2]: Leaving directory `/home/petersen/ghc-6.10.1.20090303/compiler' make -C utils with-stage-2 make[2]: Entering directory `/home/petersen/ghc-6.10.1.20090303/utils' make -C installPackage with-stage-2 make[3]: Entering directory `/home/petersen/ghc-6.10.1.20090303/utils/installPackage' /home/petersen/ghc-6.10.1.20090303/libraries/cabal-bin /usr/bin/ghc /home/petersen/ghc-6.10.1.20090303/libraries/bootstrapping.conf 1.6.0.2 configure --distpref dist-install \

--prefix=/NONEXISTENT --bindir=/NONEXISTENT --libdir=/NONEXISTENT --libexecdir=/NONEXISTENT --datadir=/NONEXISTENT --docdir=/NONEXISTENT --haddockdir=/NONEXISTENT --htmldir=/NONEXISTENT \ --with-compiler=/home/petersen/ghc-6.10.1.20090303/ghc/stage2-inplace/ghc --with-hc-pkg=/home/petersen/ghc-6.10.1.20090303/utils/ghc-pkg/install-inplace/bin/ghc-pkg --package-db /home/petersen/ghc-6.10.1.20090303/stage3.package.conf \ --libsubdir='$pkgid' --with-gcc=gcc --with-ld=/usr/bin/ld --with-happy="/usr/bin/happy" --configure-option='--enable-shared' --configure-option=--with-cc="gcc" --with-hsc2hs=/home/petersen/ghc-6.10.1.20090303/utils/hsc2hs/install-inplace/bin/hsc2hs \ --constraint="Cabal == 1.6.0.2"

Configuring installPackage-1.0... cabal-bin: ghc version >=6.4 is required but the version of /home/petersen/ghc-6.10.1.20090303/ghc/stage2-inplace/ghc could not be determined. make[3]: *** [with-stage-2] Error 1 make[3]: Leaving directory `/home/petersen/ghc-6.10.1.20090303/utils/installPackage' make[2]: *** [with-stage-2.installPackage] Error 2 make[2]: Leaving directory `/home/petersen/ghc-6.10.1.20090303/utils' make[1]: *** [stage2] Error 2 make[1]: Leaving directory `/home/petersen/ghc-6.10.1.20090303' make: *** [bootstrap2] Error 2

[juhpetersen]

end of above buildlog

[simonmar]

see also #3072

[juhpetersen]

build error without prof

[juhpetersen]

same error with HEAD without prof (in English;)

[ttuegel]

Since there hasn't been any chatter on this ticket in more than a month, I thought I would share some observations I made today testing shared library support on x86_64 Linux (I also have an x86 machine to test on later). I apologize in advance if this information is redundant; keep in mind it comes from an only moderately well-informed outsider.

1. ./configure --enable-shared && make successfully compiles a compiler! I nearly fell out of my chair in awe!

2. make -jN does not work for N > 1, but this may be a symptom of the build system in general, and not of shared library support.

3. Because we need binaries bootstrapped as part of the compile process to install everything, make install fails because the operating system linker cannot find the shared libraries.

I was able to use some LD_LIBRARY_PATH magic to include the relevant build directories and install anyway, but this only exposed the following:

4. We put shared libraries for most packages into ${prefix}/lib but the rts and ffi shared-libraries go into ${prefix}/lib/ghc-6.11.20090409 where the dynamic linker won't find them by default. Although we could use LD_LIBRARY_PATH to include the appropriate directory, that feels like a dirty hack: the standard Linux practice seems to be to put any shared libraries the linker needs to find where it will find them by default.

[juhpetersen]

Replying to ttuegel:

1. ./configure --enable-shared && make successfully compiles a compiler! I nearly fell out of my chair in awe!

Cool - that is indeed great news. Wow, wow! :-)

I managed to "make install" ghc-6.11.20090421 with fedora ghc-6.10.2.x86_64 with a small patch I am attaching next: install needs to use dist-inplace/ghc-pkg not dist-install, which might be what ttuegel also ran into in the proceeding comment.

[juhpetersen]

Use dist-inplace ghc-pkg for dist package.conf and cabal'ing

[Pete]

"Although we could use LD_LIBRARY_PATH to include the appropriate directory, that feels like a dirty hack: the standard Linux practice seems to be to put any shared libraries the linker needs to find where it will find them by default."

Most packages I've seen install shared libraries in $PREFIX/lib. If you accept the default $PREFIX, this means they will go in /usr/local/lib, and the linker is normally set so it will find them there. If a package (deb, rpm or whatever) is being built, then $PREFIX will usually be set to /usr. In this case, the libraries will go in /usr/lib and again the linker will find them.

If you decide to install the package in a private directory, the linker won't find the libraries. In this case you can set the executable's rpath to the directory where the libraries are to be found. The downside is that this makes the executables non-portable. If Fred installs the libraries in ~fred/lib and Jim installs them in ~jim/lib, they won't be able to run each other's executables. The rpath will be set correctly for the person who compiled the executable, but no one else. At this point you have to set LD_LIBRARY_PATH and work round it that way.

The Debian package of ALSA takes a different approach. It installs its libraries in /usr/lib/alsa-lib/ and installs /etc/ld.so.conf.d/libasound2.conf which points to them. (This file contains a single line, '/usr/lib/alsa-lib'.) This approach makes sense, I imagine, if you are installing a lot of libraries and don't want to clutter up /usr/lib.

I was just doing some experiments to make sure I'd got this right. I created foo.c:

#include <stdio.h>

void blah() {

printf("Hello World!\n");

}

and bar.c:

extern void blah();

main() {

blah();

}

First of all I build foo.c into a shared library:

$ gcc -fPIC -shared foo.c -o libfoo.so

I can then link bar.c in two ways. Here is the first option:

$ gcc bar.c -o bar -L. -lfoo $ ./bar ./bar: error while loading shared libraries: libfoo.so: cannot open shared object file: No such file or directory $ LD_LIBRARY_PATH=. ./bar Hello World!

Second option:

$ gcc bar.c -o bar -Wl,--rpath=/home/pc -L. -lfoo $ ./bar Hello World!

I hope this is helpful. I'm not a functional programming expert, so I can't implement this myself. On the other hand, I'm happy to help with any of these operating system issues. My email is 1@… if you want anything from me.

[simonmar]

Shared libs are working on Linux in 6.12.1. We should open separate tickets for other tasks remaining to be done.

A big thanks to everyone who has worked on getting shared library support into GHC over the years, particularly Wolfgang Thaller, Clemens Fruhwirth and Duncan Coutts.