cabal assumes -x option for linking

This ticket needs a desperate fix for the ghc-6.8.1 release! Furthermore the generated file libraries/base/GNUMakefile contained "xargs -s 30000" which is too big for Solaris (the maximal value is 2048)

this is how the error message looks like:

/usr/ccs/bin/ld: illegal option -- x
usage: ld [-6:abc:d:e:f:h:il:mo:p:rstu:z:B:CD:F:GI:L:M:N:P:Q:R:S:VY:?] file(s)
        [-64]           enforce a 64-bit link-edit
        [-a]            create an absolute file
        [-b]            do not do special PIC relocations in a.out
        [-B direct | nodirect]
                        establish direct bindings, or inhibit direct binding
                        to, the object being created
        [-B dynamic | static]
                        search for shared libraries|archives
        [-B eliminate]  eliminate unqualified global symbols from the
                        symbol table
        [-B group]      relocate object from within group
        [-B local]      reduce unqualified global symbols to local
        [-B reduce]     process symbol reductions
        [-B symbolic]   bind external references to definitions when creating
                        shared objects
        [-c name]       record configuration file `name'
        [-C]            demangle C++ symbol name diagnostics
        [-d y | n]      operate in dynamic|static mode
        [-D token,...]  print diagnostic messages
        [-e epsym]      use `epsym' as entry point address
        [-f name]       specify library for which this file is an auxiliary
                        filter
        [-F name]       specify library for which this file is a filter
        [-G]            create a shared object
        [-h name]       use `name' as internal shared object identifier
        [-i]            ignore LD_LIBRARY_PATH setting
        [-I name]       use `name' as path of interpreter
        [-l x]          search for libx.so or libx.a
        [-L path]       search for libraries in directory `path'
        [-m]            print memory map
        [-M mapfile]    use processing directives contained in `mapfile'
        [-N string]     create a dynamic dependency for `string'
        [-o outfile]    name the output file `outfile'
        [-p auditlib]   identify audit library to accompany this object
        [-P auditlib]   identify audit library for processing the dependencies
                        of this object
        [-Q y | n]      do|do not place version information in output file
        [-r]            create a relocatable object
        [-R path]       specify a library search path to be used at run time
        [-s]            strip any symbol and debugging information
        [-S supportlib]
                        specify a link-edit support library
        [-t]            do not warn of multiply-defined symbols that have
                        different sizes or alignments
        [-u symname]    create an undefined symbol `symname'
        [-V]            print version information
        [-Y P,dirlist]  use `dirlist' as a default path when searching for
                        libraries
        [-z absexec]    when building an executable absolute symbols
                        referenced in dynamic objects are promoted to
                        the executable
        [-z allextract | defaultextract | weakextract]
                        extract all member files, only members that resolve
                        undefined tor tentative symbols, or allow extraction of
                        archive members to resolvetweak references from
                        archive files
        [-z altexec64]  execute the 64-bit link-editor
        [-z combreloc]  combine multiple relocation sections
        [-z defs]       disallow undefined symbol references
        [-z direct | nodirect]
                        enable|disable direct binding to shared object
                        dependencies
        [-z endfiltee]  marks a filtee such that it will terminate a filters
                        search
        [-z finiarray=function]
                        name of function to be appended to the .finiarray
        [-z groupperm | nogroupperm]
                        enable|disable setting of group permissions
                        on dynamic dependencies
        [-z help ]      print this usage message
        [-z ignore | record]
                        ignore|record unused dynamic dependencies
        [-z initarray=function]
                        name of function to be appended to the .initarray
        [-z initfirst]  mark object to indicate that its .init section should
                        be executed before the .init section of any other
                        objects
        [-z interpose]  dynamic object is to be an `interposer' on direct
                        bindings
        [-z lazyload | nolazyload]
                        enable|disable delayed loading of shared object
                        dependencies
        [-z ld32=arg1,arg2,...]
                        define arguments applicable to the 32-bit class of ld(1)
        [-z ld64=arg1,arg2,...]
                        define arguments applicable to the 64-bit class of ld(1)
        [-z loadfltr]   mark filter as requiring immediate loading of its
                        filtees at runtime
        [-z muldefs]    allow multiply-defined symbols
        [-z nocompstrtab]
                        disable compression of string tables
        [-z nodefs]     allow undefined symbol references
        [-z nodefaultlib]
                        mark object to ignore any default library search path
        [-z nodelete]   mark object as non-deletable
        [-z nodlopen]   mark object as non-dlopen()'able
        [-z nodump]     mark object as non-dldump()'able
        [-z now]        mark object as requiring non-lazy binding
        [-z nopartial]  expand any partially initialized symbols
        [-z noversion]  don't record any version sections
        [-z origin]     mark object as requiring $ORIGIN processing
        [-z preinitarray=function]
                        name of function to be appended to the .preinitarray
        [-z redlocsym]  reduce local syms in .symtab to a minimum
        [-z rescan]     rescan archive list until no further member
                        extraction occurs
        [-z text]       disallow output relocations against text
        [-z textoff]    allow output relocations against text
        [-z textwarn]   warn if there are relocations against text
        [-z verbose]    generate warnings for suspicious processings
gmake[2]: *** [dist/build/HSbase-3.0.o] Error 1

I could continue with a gnu linker only after cleaning up completely.

No, I could not continue! When I put a gnu linker first in my path then ghc-inplace passes the -x option to the linker, which is wrong, because gcc is used as a linker which in turn is based on the Solaris linker.

The documentation for ld on Ubuntu reads:

-x
--discard-all
     Delete all local symbols.

That seems to be corresponding to

[-z redlocsym]  reduce local syms in .symtab to a minimum
}}}.
Maybe it could even be omitted.  There actually is a FIXMe in Distrubution/Simple/GHC.hs line 348.  Could you try removing that line, rebuild Cabal and try again?

Omitting this line works under Solaris. Maybe this explains why my binaries are about 20% bigger.

No, it did no work yet. It seems I have to edit libraries/Cabal/Distribution/Simple/GHC/Makefile.in, too!

See also http://hackage.haskell.org/trac/ghc/ticket/1786#comment:2 where I describe one route to a fix.

Maybe LDFLAGS should be appropriately set by configure and used in the sources instead of -x. Meanwhile the ghc-6.8.0.20071017 sources (without extra-libs) went through.

You misunderstand. The Makefile is generated by Cabal. So it's nothing to do with GHC's configure script and everything to do with Cabal deciding what the appropriate flags are.

As it happens because of the Makefile stuff we'll have to fix it in two places in Cabal, once for the ordinary Cabal build stuff and once for where Cabal generates the Makefile.

I quite understood that a Makefile.hs is generated from Makefile.in and that Cabal generates a GNUMakefile. I suggest that "Setup configure" should find out a proper value for a (new or further) LDFLAGS variable that is then used in conjunction with LD (instead of -x).

Latest cabal-1.2.x version has a fix. Can some Solaris user check that it works please?

Thanks, could you point me to a stable ghc-6.8.0 release candidate that I should try out? Christian

I could build ghc-6.8.0.20071025 from the sources, now (after skipping http://hackage.haskell.org/trac/ghc/ticket/1805). Christian

Replying to guest:

Furthermore the generated file libraries/base/GNUMakefile contained "xargs -s 30000" which is too big for Solaris (the maximal value is 2048)

I'd really like to get this increased. 2048 is ridiculously low for most platforms. We end up doing multiple ar and ld calls even for ordinary sized Haskell projects without even using split-objs. If people use split-objs we'll end up making 100's of ar/ld calls for larger libs.

According to POSIX.1-2001 the minimum value for ARG_MAX is _POSIX_ARG_MAX which is 4096. So surely even older versions of Solaris must have supported at least 4096?

I used the following C program to test on Solaris 9 and I discovered the value is 1048320.

#include <stdio.h>
#include <unistd.h>
#include <limits.h>
int main () {
  printf("ARG_MAX == %d, sysconf(_SC_ARG_MAX) == %ld\n",
          ARG_MAX,       sysconf(_SC_ARG_MAX));
  return 0;
}

On posix systems we could test dynamically using System.Posix.Unistd.getSysVar ArgumentLimit though it's a pain because we'd have to use cpp and add a conditional dependency on the unix package.

Christian, what version of Solaris are you using and what does the above C program report for you?

We have Solaris 10, you're program outputs the value 1048320, too. However, my problem was about the xargs -s value. So draw your own conclusions from my output below. Cheers Christian

-bash-3.00$ ./a.out
ARG_MAX == 1048320, sysconf(_SC_ARG_MAX) == 1048320
-bash-3.00$ xargs -s 30000
xargs: 0 < max-cmd-line-size <= 2048: 30000
xargs: Usage: xargs: [-t] [-p] [-e[eofstr]] [-E eofstr] [-I replstr] [-i[replstr]] [-L #] [-l[#]] [-n # [-x]] [-s size] [cmd [args ...]]
-bash-3.00$ uname -a
SunOS leo 5.10 Generic_127111-03 sun4u sparc SUNW,Sun-Fire-280R

Replying to guest:

We have Solaris 10, you're program outputs the value 1048320, too. However, my problem was about the xargs -s value. So draw your own conclusions from my output below. Cheers Christian

Thanks very much Christian, that's great. I don't think us cabal hackers realised it was just xargs the unix program that had the limit and assumed it was the process limit. There's also an xargs function in cabal and we mistakenly applied the limit to that too:

Thu Oct 18 15:05:00 BST 2007  Simon Marlow <simonmar@microsoft.com>
  * FIX GHC bug #1785: use 2048 as the maximum command-line size
  Apparently Solaris sets a limit of 2048 here

    hunk ./Distribution/Simple/GHC.hs 411
    -            maxCommandLineSize = 30 * 1024
    +            -- used to be 30k, but Solaris needs 2k (see GHC bug #1785)
    +            maxCommandLineSize = 2048
    hunk ./Distribution/Simple/GHC/Makefile.in 62
    -   (echo $(C_OBJS) `$(MKSTUBOBJS)`; find $(patsubst %.$(osuf),%_split,$(HS_OBJS)) -name '*.$(way_)o' -print) | xargs -s 30000 $(AR) q $(EXTRA_AR_ARGS) $@ $
    +   (echo $(C_OBJS) `$(MKSTUBOBJS)`; find $(patsubst %.$(osuf),%_split,$(HS_OBJS)) -name '*.$(way_)o' -print) | xargs -s 2048 $(AR) q $(EXTRA_AR_ARGS) $@ $

(Simon later applied another patch to remove the -s arg from the xargs program invocation)

So I'll change maxCommandLineSize back to 30k which seemed to be ok previously. I'm leaving the call to the xargs program well alone.