USING EXTERNAL CORE FOR GREAT JUSTICE

The procedure for setting up your environment to write code that
manipulates External Core is pretty involved. That's mostly because you'll
have to modify GHC's libraries.

Why? Because GHC's optimizer inlines aggressively across module boundaries,
meaning that if you just want to go so far as parsing and typechecking the
Core version of the "hello world" program, you're going to have to implement
quite a few I/O primitives and foreign function calls. You might want to
get started writing your Core back-end by implementing just a very simple
I/O interface, which means you won't be able to compile all Core programs
produced by GHC. In short, you want to replace some of GHC's I/O library
functions with simpler versions, which rely on foreign functions that you
will define in your Core back-end.

I've provided a simple library patch that redefines just *two* functions,
hPutStr and hPutChar, to use a foreign function that prints one character
to stdout. This should serve as an example of how to modify the GHC
libraries so as to define I/O functions on top of the interface of your choice.

The steps described below are a prerequisite for running the Core Linker code
that turns multiple GHC modules (for example, one module plus all the base
library modules it depends on) into a single External Core file. You need
External Core source code for all the libraries in order to do this.

You can run the Core Linker and *typecheck* the results without modifying
the GHC libraries at all. I haven't explained here in detail how to do that.

The steps described below are also a prerequisite for running the stand-alone Core interpreter
so you can evaluate simple programs. This interpreter is meant to serve as an example
of how to write code that manipulates Core, as well as a specification of
Core's semantics.

This may seem complicated, but the alternative would be, in effect, to re-implement
GHC's base of primitive operations in your own back-end.

I hope that before too long, the syntax of External Core will change to use
a fixed set of primitives, obviating the need for library hacks.

========= Instructions ===========

You're going to need to make a directory in which to do step 1 and step 2.
In what follows, I'll refer to that directory as $TOP.

1. darcs get --partial http://darcs.haskell.org/ghc-6.10/ghc

(It's important that you get the stable branch of ghc 6.10.)

And you need to get the libraries too:
1a. $ cd ghc
$ chmod +x darcs-all
$ ./darcs-all get

Don't build ghc yet! Keep reading...

2. Execute the following:

$ ls
ghc
$ mkdir ghc_stage2
$ ls
ghc ghc_stage2
$ cd ghc_stage2
$ lndir ../ghc .

You've just created a shadow directory of the ghc source tree. In the original ghc source tree,
you'll build a stage 1 version of ghc. In the shadow directory, which you've called ghc_stage2,
you'll build a stage2 version of ghc.

Next, apply the library patch.

3. Get the extcore tarball
http://hackage.haskell.org/packages/archive/extcore/0.3/extcore-0.3.tar.gz

and then:

$ tar xzvf extcore-0.3.tar.gz

Don't install it just yet! If you ls at the top level in extcore-0.3:

$ ls
... simple_io.dpatch ...

you will see a file called simple-io.dpatch. In the next step, you're going to apply this
patch to the library directory you created in step 2.

4. Do:
$ cd $TOP/ghc/libraries/base
$ darcs apply $TOP/extcore-0.2/simple_io.dpatch

5. Now that you've patched the libraries, you're going to have to build ghc.

The first step is to set up your build.mk file:
$ cd $TOP/ghc/mk
$ cp build.mk.sample build.mk

Now edit your build.mk. The upshot is that you want to build the libraries with -O, -fext-core,
and -DSIMPLE_IO. I suggest you build ghc itself with no optimization, at least to get started.
Here's what I suggest:

* Uncomment the line beginning BuildFlavour = quickest
* Edit the line in the "quickest" section beginning with "GhcLibHcOpts =":
* * delete the flag "-O0" and add the flags "-O", "-fext-core", and "-DSIMPLE_IO".

6. In $TOP/ghc, do:
$ sh boot; ./configure; make -j2
(-j2 only helps if you have a dual-core machine, of course.)
Go get some coffee or cook a three-course meal or something while this is running.

The point of this step is to generate External Core files for all the GHC libraries.

(By the way, if you need more help building ghc, the GHC developer wiki is
very helpful:
http://hackage.haskell.org/trac/ghc/)
 
Eventually, the build will fail when you get to the stage2 point, because the libraries
that you've patched can't bootstrap GHC. But that's OK!  

7. Okay, now you're going to do the same thing over again in the stage2 library! Only,
make sure your build.mk in $TOP/ghc_stage2/mk does NOT include the -DSIMPLE_IO and
-fext-core options. It *must* build the libraries with -O, though.

The difference is, this time, the stage2 build will succeed.

In $TOP/ghc_stage2 do:
$ sh boot; ./configure; make -j2

7a. The External Core library has some prerequisites: mtl and parsec.
You need to install these packages in a place where your new stage2 compiler can see them.
You can get mtl and parsec from Hackage, and install them in the usual way. The important thing
is to make sure you build them with $TOP/ghc_stage2/ghc/stage2-inplace/ghc and install them where
your inplace GHC can see them. The steps for doing so are roughly the same as step 8 (next).

8. Now go back to the extcore-0.3 directory and do:
$ runhaskell -f $TOP/ghc_stage_2/ghc/stage2-inplace/ghc Setup configure --user --with-ghc=$TOP/ghc_stage_2/ghc/stage2-inplace/ghc --with-ghc-pkg=$TOP/ghc_stage_2/utils/ghc-pkg/install-inplace/bin/ghc-pkg
$ runhaskell -f $TOP/ghc_stage_2/ghc/stage2-inplace/ghc Setup build
$ runhaskell -f $TOP/ghc_stage_2/ghc/stage2-inplace/ghc Setup install

9. Now you can build the Core Linker.

Get the Core Linker from:
http://hackage.haskell.org/package/linkcore
and unzip it somewhere.

runhaskell -f $TOP/ghc_stage_2_build/ghc/stage2-inplace/ghc Setup.lhs configure --user --with-ghc=$TOP/ghc_stage_2/ghc/stage2-inplace/ghc --with-ghc-pkg=$TOP/ghc_stage_2/utils/ghc-pkg/install-inplace/bin/ghc-pkg --configure-option=--topdir=/Users/tjc/ExtCoreTestingSandbox/ghc_stage_2_build/inplace-datadir/ --configure-option=--pkgconf=/Users/tjc/ExtCoreTestingSandbox/ghc/inplace-datadir/package.conf
runhaskell -f $TOP/ghc_stage_2/ghc/stage2-inplace/ghc Setup build
runhaskell -f $TOP/ghc_stage_2/ghc/stage2-inplace/ghc Setup install # As superuser, if necessary

Now you have an executable "linkcore" installed... well, wherever your Cabal installs things.
(Since you configured with --user (this is necessary because you installed the prerequisite libraries with --user), you
may want to change the --prefix that you pass to configure.)

Finally, the moment you've been waiting for:
$ cat > hello.hs
module Main where
  main = putStrLn "hello world!"
^D

$ linkcore --package-root=$TOP/ghc/libraries --package-root=$TOP/ghc/libraries/base/dist/build --package-root=$TOP/ghc/libraries/integer-gmp hello.hs -o hello_out.hcr

What are all these flags doing? They're telling the Core linker to look for External Core source code in the library
trees you compiled to External Core in an earlier step. Also, the -o flag tells the linker to put the Core output in hello.hcr.

This will take a few minutes and use quite a lot of heap.

10. Now you have a single Core file hello_out.hcr, which you can parse, typecheck, and interpret with the stand-alone Core library.
See the Driver.hs file included in the source distribution for the External Core library for an example of how to do that.

11. (optional)
Using the Driver.hs file that came with the External Core library tarball you downloaded before:
  $TOP/[etc.]/ghc --make -package extcore -o Driver Driver.hs
  ./Driver [etc.]/hello_out.hcr # wherever you generated the hello_out.hcr file before