Dynamically link GHCi (and use system linker) on platforms that support it

[simonmar]

In 6.14.1 we should switch to shipping a dynamically linked GHCi binary, on those platforms for which dynamic linking is supported (currently Linux; MacOS X and Windows support is in progress).

Advantages

• The GHCi binary is smaller
• some packages don't need to be loaded on startup: lower memory use
• GHCi startup might be quicker (or it might not)
• some hacks due to having two copies of the base package are not necessary (see rts/Globals.c)
• We might save some space in the distributions.
• It takes us a step closer to not needing the RTS linker at all
• It takes us a step closer to using dynamic linking by default, which is where we want to go ultimately

Potential Issues

• Do we run into any problems with GHCi and the user program sharing the same stdin/stdout/stderr handles? Do we need to virtualise these explicitly in the GHCi front end?

• We cannot revert CAFs in packages that are shared by GHC and the user program. There are some old non-working hacks related to reverting CAFs when GHCi is dynamically linked (see KeepCAFsForGHCi) that need to be cleaned out. CAFs can only be reverted in code loaded by the RTS linker. We need to think about whether this is a necessary feature or not: we have never supported CAF reverting for interpreted code anyway. One reason to have it was so that you can recover after saying getContents at the GHCi prompt, but we can provide other ways to work around that.

• There will be installation/binary-dist issues to resolve; currently we don't install any dynamically-linked binaries.

Open questions

• Whether we continue to use the same binary for GHC and GHCi is an open question: it would be possible to provide a separate statically-linked GHC binary if performance of the dynamically-linked version was an issue.

• We might as well dynamically-link Haddock, and the other tools that come with GHC too.

[simonmar]

Some progress towards this:

Wed Apr 28 21:52:41 BST 2010  Simon Marlow <marlowsd@gmail.com>
  * Add $(GhcDynamic) knob, set to YES to get stage2 linked with -dynamic
  Default currently NO.
  
  Validate passed with GhcDynamic=YES on x86/Linux here.
  
  The compiler is currently slower on x86 when linked -dynamic,
  because the GC inner loop has been adversely affected by -fPIC, I'm
  looking into how to fix it.

The compiler really is slower when linked dynamically, at least on x86, because of the overhead of -fPIC - the GC is significantly slower. I looked into it but didn't find a way to fix it. We might have to ship separate GHC and GHCi binaries on x86; I expect this to be better on x86_64, but haven't measured it yet.

Currently, Cabal doesn't build packages dynamically by default, and this will become an accute issue when GHCi is linked dynamically. We need to do the package system changes needed to register each way separately, so that 'cabal install --enable-dynamic' works properly. Either that or Cabal has to enable dynamic by default.

GHCi is quicker to start up when dynamically linked: 0.1s vs. 0.23s.

[simonpj]

Bumping this to 7.2. We just aren't going to get this done for 7.0.

[simonmar]

(In #781) To fix this we need to use -fPIC by default on x86_64, which is also what we need for #3658. So let's plan to do this in 7.2, when we've thought through the implications of moving to dynamic linking by default.

[simonmar]

This has just become more important since we ran into an arbitrary limit in the OS X object file format:  http://www.haskell.org/pipermail/cvs-ghc/2010-September/055908.html.

Here's a rough plan of action:

• measure the performance impact of dynamic linking. If the impact is too much, fixing that is a blocking issue. We can't ship a separate static GHC binary, because GHC needs to use TH, so GHC itself has to be dynamic.
• make a build system switch to enable dynamic-by-default
• make GHC report whether it is a dynamic-default compiler, in the --info output.
• modify Cabal to use --enable-*-dynamic and --disable-library-vanilla, if GHC is dynamic.
• implement loading of object files by first calling 'gcc -shared' (or equiv. ld command line), and then dlopen(). Measure the performance impact of that. Figure out whether we need to cache the results of gcc -shared or regenerate them into $TMPDIR each time.
• enable dynamic-by-default for some platforms.

• Later: we need to track ways properly in Cabal and the package database.

[simonmar]

Furthermore:

• If the user mentions a .c file on the command line, we have to compile that with -fPIC too. The same goes for our _stub files.

[igloo]

Punting.

[igloo]

I think this is where we are:

• Dynamically linking GHC is relatively easy (compared to dynamically linking executables in general). GHC already ships with the actual dynamic libraries, so we can use assemblies on Windows, $ORIGIN on ELF and @loader_path on OS X to find the dynlibs with a relative path.
• To load .o files in the working directory compiled by the user, ghci can link them into a DLL in /tmp and then dlopen that DLL
• Problem 1: The .o files in the working directory would need to be built the dyn way
• Problem 2: #5987: There are currently too many symbols to make a ghc DLL on Windows

One solution (easy to implement, but not very user-friendly) to problem 1 would be to require that users use -dynamic when compiling things they want ghci to be able to use.

Perhaps another would be for ghc to make both .o and .dyn_o by default. I think the two ways are identical apart from the final codegen phase.

[simonmar]

Problem 3 is that Cabal doesn't track ways properly yet, although hopefully the Commentary/GSoCMultipleInstances project will address that.

I quite like the idea of building .o and .dyn_o at the same time, and sharing the .hi file between the two. That would at least reduce the pain of having to build things twice. This could then be the default mode of compilation in Cabal.

I still think the right way to fix this is to enable -dynamic by default on Unix systems (pending some performance measurements), after we have fixed Cabal to track ways. On Windows, Ian's idea of building with -dynamic but linking statically sounds interesting.

[igloo]

(In #7357) I can't reproduce this for some reason.

Nevertheless, it would be fixed by #3658.

[igloo]

(In #7299) I can reproduce this.

However, it doesn't happen with a dynamic-by-default build.

[igloo]

(In #7056) This would be fixed by dynamic-by-default.

[igloo]

(In #7103) This would be fixed by dynamic-by-default.

[igloo]

(In #7043) This doesn't happen if dynamic-by-default is enabled.

[igloo]

(In #7475) Thanks for the report. This looks like it's probably a duplicate of #7043, although that ticket claims to only affect 32bit platforms.

[igloo]

(In #7389) Thanks for the report.

I'm a little confused about why the ghc linker is being used in this case, but the error comes from it, so it must be.

In any case, this therefore ought to be fixed by #3658.

[igloo]

(In #7687) Thanks for the report. This will be fixed by using the system linker in ghci.

[igloo]

(In #7316) This should be fixed by using dynamic libraries to implement GHCi.

[igloo]

(In #7889) Thanks for the report. I'm hoping to fix this by fixing #3658.