GF Developers Guide Authors: Björn Bringert and Krasimir Angelov Last update: %%mtime(%c) % NOTE: this is a txt2tags file. % Create an html file from this file using: % txt2tags -t html --toc darcs.txt %!target:html %!options(html): --toc %!encoding:utf-8 = Setting up your system for building GF = Before to build GF from sources you need to install some tools on your system. GF is written in Haskell, so first of all you need recent version of the Haskell compiler GHC. Currently we use GHC 6.10.3 and we recommend that you should use the same version as well. This version is not backward compatible with the previous major releases so you cannot use previous versions. GHC is available from here: http://www.haskell.org/ghc/ Once you have installed GHC, open a terminal (Command Prompt on Windows) and try to execute the following command: ``` $ ghc --version ``` This command should show you which version of GHC you have. If the installation of GHC was successful you should see message like: ``` The Glorious Glasgow Haskell Compilation System, version 6.10.3 ``` The other two tools that we use are the lexer generator for Haskell - Alex: http://www.haskell.org/alex/ and the parser generator - Happy: http://www.haskell.org/happy/. Again after the installation check that the tools are available from the terminal. If they are not then probably you have to update the current search path in your system. It is also a good idea to have either readline, editline or haskeline installed. This are libraries for user friendly command line editing. On Linux, without some of this libraries, the command line editor is very basic. Actually the only key for editing that you can use is backspace. On Windows you get much more user friendly editor by default but with it you cannot use the GF specific tab completion. In any case if you plan to use GF for continuous development then it is recomended to install some of these libraries. The GF configuration script checks the libraries in the following order: - haskeline - readline - editline the first that is found will be used in the compilation. The libraries are also written in Haskell and could be found on Hackage: http://hackage.haskell.org/packages/archive/pkg-list.html. If you want to check whether, you already have some of these you can use the following command: ``` $ ghc-pkg list ``` which shows the list of all installed libraries. Haskeline is the easiest to install because it is a pure Haskell library but currently with this editor GF doesn't provide word completion. With editline we provide word completion but the library is harder to install because it is a Haskell binding to a library with the same name written in C. If you do not have the C library you will have to install it first. Unfortunately editline does not have good support for Unicode. This will be a problem if you tend to work on non-Latin language. Finaly readline supports both word completion and Unicode. Currently this is the best supported library. Before to get the GF sources you also need Darcs, version 2 or later. Darcs is a decentralized revision control system, see http://darcs.net/ for more information. There are precompiled packages for many platforms available at http://darcs.net/DarcsWiki/CategoryBinaries. There is also source code if you want to compile it yourself. Darcs is also written in Haskell and so you can use GHC to compile it. = Getting the sources = Once you have all tools in place you can get the GF sources. If you just want to compile and use GF then it is enough to have read-only access. It is also possible to make changes in the sources but if you want these changes to be applied back to the main sources you will have to send the changes to us. If you plan to work continuously on GF then you should consider to get read-write access. == Read-only access == === Getting a fresh copy for read-only access === Anyone can get the latest development version of GF by running (all on one line): ``` $ darcs get --lazy --set-scripts-executable http://code.haskell.org/gf/ ``` This will create a directory called ``gf`` in the current directory. === Updating your copy === To get all new patches from the main repo: ``` $ darcs pull -a ``` This can be done anywhere in your local repository, i.e. in the ``gf`` directory, or any of its subdirectories. Without ``-a``, you can choose which patches you want to get. === Recording local changes === Since every copy is a repository, you can have local version control of your changes. If you have added files, you first need to tell your local repository to keep them under revision control: ``` $ darcs add file1 file2 ... ``` To record changes, use: ``` $ darcs record ``` This creates a patch against the previous version and stores it in your local repository. You can record any number of changes before pushing them to the main repo. In fact, you don't have to push them at all if you want to keep the changes only in your local repo. If you think there are too many questions about what to record, you can use the ``-a`` flag to ``record``. Or answer ``a`` to the first question. Both of these record all the changes you have in your local repository. === Submitting patches === If you are using read-only access, send your patches by email to someone with write-access. First record your changes in your local repository, as described above. You can send any number of recorded patches as one patch bundle. You create the patch bundle with: ``` $ darcs send -o mypatch.patch $ gzip mypatch.patch ``` (where ``mypatch`` is hopefully replaced by a slightly more descriptive name). Since some e-mail setups change text attachments (most likely by changing the newline characters) you need to send the patch in some compressed format, such as GZIP, BZIP2 or ZIP. Send it as an e-mail attachment. If you have sendmail or something equivalent installed, it is possible to send the patch directly from darcs. If so, replace ``-o mypatch.patch`` with ``--to=EMAIL`` where ``EMAIL`` is the address to send it to. == Read-write access == If you have a user account on code.haskell.org, you can get read-write access over SSH to the GF repository. To get an account, [fill out this form http://community.haskell.org/admin/account_request.html]. Once you have an account, ask to add you to the ``GF`` project. === Getting a fresh copy === Get your copy with (all on one line), replacing ``bringert`` with your own username on code.haskell.org: ``` $ darcs get --lazy --set-scripts-executable bringert@code.haskell.org:/srv/code/gf ``` The option ``--lazy`` means that darcs defers downloading all the history for the repository. This saves space, bandwidth and CPU time, and most people don't need the full history of all changes in the past. === Getting other people's changes? === Get all new patches from the main repo: ``` $ darcs pull -a ``` Without ``-a``, you can choose which patches you want to get. === Commit your changes === There are two steps to commiting a change to the main repo. First you have to record the changes that you want to commit, then you push them to the main repo. For instructions on recording your changes locally, see "Recording local changes" above. Then you can push the patch(es) to the main repo. If you are using ssh-access, all you need to do is: ``` $ darcs push ``` If you use the ``-a`` flag to push, all local patches which are not in the main repo are pushed. === Apply a patch from someone else === Use: ``` $ darcs apply < mypatch.patch ``` This applies the patch to your local repository. To commit it to the main repo, use ``darcs push``. == Further information about Darcs == For more info about what you can do with darcs, see http://darcs.net/manual/ = Compilation from sources = The build system of GF is based on Cabal (see http://www.haskell.org/cabal/ for more information). Cabal is installed by default together with the GHC compiler. This is actually a library which could be used from Haskell to compile projects written in Haskell. The entry point is a script called Setup.hs which is placed in the top directory of every project managed with Cabal. The three main steps that are needed for compilation are much like what you do in a project written in C, you have: configure, build and install. == Configure == During the configuration phase Cabal will check that you have all necessary tools and libraries needed for GF. The configuration is started by the command: ``` $ runghc Setup.hs configure ``` The command ```runghc``` comes with the GHC compiler and is batch interpreter which executes the specified script without the need to compile it advance. Setup.hs is our compilation driver which is based on Cabal. If you don't see any error message from the above command then you have everything that is needed for GF. You can also add the option ```-v``` to see more details about the configuration. == Build == The build phase does two things. First it builds the GF compiler from the Haskell sources and after that it builds the GF Resource Grammar Library using the already build compiler. The simplest command is: ``` $ runghc Setup.hs build ``` Again you can add the option ```-v``` if you want to see more details. Sometimes you just want to work on the GF compiler and don't want to recompile the resource library after each change. In this case use this extended command: ``` $ runghc Setup.hs build rgl-none ``` The resource library could also be compiled in two modes: with present tense only and with all tenses. By default it is compiled with all tenses. If you want to use the library with only present tense you can compile it in this special mode with the command: ``` $ runghc Setup.hs build present ``` Before to use this command make sure that the script lib/src/mkPresent has executable permissions on Linux. You could also control which languages you want to be recompiled by adding the option ```langs=list```. For example the following command will compile only the English and the Swedish language: ``` $ runghc Setup.hs build langs=Eng,Swe ``` == Install == After you have compiled GF you can install the binaries to make the system usable. On Linux you will need root privileges to do this. Use the command: ``` $ su ``` and enter the root password. This step should be skipped on Windows. The installation itself is started with the command: ``` $ runghc Setup.hs install ``` This command installs the GF compiler in the default place for executable files in your system. For example on Linux this is usualy /usr/local/bin and on Windows this is c:\Program Files\Haskell\bin. If you want to install in some other place then use the ```--prefix``` option during the configuration phase. The compiled GF Resource Grammar Library will be installed in /usr/local/share/gf-3.1/lib on Linux and in c:\Program Files\Haskell\gf-3.1\lib on Windows. Again the location could be changed using the ```--prefix``` option. == Clean == Sometimes you want to clean up the compilation and start again from clean sources. Use the clean command for this purpose: ``` $ runghc Setup.hs clean ``` == SDist == You can use the command: ``` $ runghc Setup.hs sdist ``` to prepare archive with all source codes needed to compile GF. = Compilation with make = If you feel more comfortable with Makefiles then there is a thin Makefile wrapper arround Cabal for you. If you just type: ``` $ make ``` the configuration phase will be run automatically if needed and after that the sources will be compiled. If you don't want to compile the resource library every time then you can use: ``` $ make gf ``` For installation use: ``` $ make install ``` For cleaning: ``` $ make clean ``` and to build source distribution archive run: ``` $ make sdist ``` = Running the testsuite = GF has testsuite. It is run with the following command: ``` $ runghc Setup.hs test ``` The testsuite architecture for GF is very simple but still very flexible. GF by itself is an interpreter and could execute commands in batch mode. This is everything that we need to organize a testsuite. The root of the testsuite is the testsuite/ directory. It contains subdirectories which themself contain GF batch files (with extension .gfs). The above command searches the subdirectories of the testsuite/ directory for files with extension .gfs and when it finds one it is executed with the GF interpreter. The output of the script is stored in file with extension .out and is compared with the content of the corresponding file with extension .gold, if there is one. If the contents are identical the command reports that the test was passed successfully. Otherwise the test had failed. Every time when you make some changes to GF that have to be tested, instead of writing the commands by hand in the GF shell, add them to one .gfs file in the testsuite and run the test. In this way you can use the same test later and we will be sure that we will not incidentaly break your code later. If you don't want to run the whole testsuite you can write the path to the subdirectory in which you are interested. For example: ``` $ runghc Setup.hs test testsuite/compiler ``` will run only the testsuite for the compiler.