linker failures with foreign global data

[luite]

On OS X with X86_64 the attached example crashes (not reliably, might take a few tries, doesn't seem to crash in gdb). A compiled executable always works fine.

[luite]

crash in ghci (OS X X86_84), fine when compiled

[luite]

result:

# ghc -c global.c
# ghci global.o main.hs
GHCi, version 7.4.2: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
Loading object (static) global.o ... done
final link ... done
[1 of 1] Compiling Main             ( main.hs, interpreted )
Ok, modules loaded: Main.
*Main> main
Segmentation fault: 11
# ghc global.o main.hs
# ./main
x: 0
x: 1
#

[luite]

HEAD also crashes

[nus]

The segfault happens when OS X calloc allocates space from the region beyond limits imposed by the small C code model.

Program stopped at 0x10b0fd624.
It stopped with signal EXC_BAD_ACCESS, Could not access memory. 

(gdb) bt
#0  0x000000010b0fd624 in ?? ()
#1  0x000000010871043c in ffi_call_unix64 ()

(gdb) x/20i 0x10b0fd624
0x10b0fd624:	mov    0x7303d16(%rip),%eax        # 0x112401340
0x10b0fd62a:	xor    %cl,%cl
0x10b0fd62c:	lea    0x1c7(%rip),%rdx        # 0x10b0fd7fa
0x10b0fd633:	mov    %rdx,%rdi
0x10b0fd636:	mov    %eax,%esi
0x10b0fd638:	mov    %cl,%al
0x10b0fd63a:	callq  0x10b0fda70
0x10b0fd63f:	movl   $0x1,0x7303cf7(%rip)        # 0x112401340
0x10b0fd649:	mov    0x7303cf1(%rip),%ecx        # 0x112401340
0x10b0fd64f:	xor    %dl,%dl
0x10b0fd651:	lea    0x1a2(%rip),%rdi        # 0x10b0fd7fa
0x10b0fd658:	mov    %ecx,%esi
0x10b0fd65a:	mov    %dl,%al
0x10b0fd65c:	callq  0x10b0fda70
0x10b0fd661:	pop    %rbp
0x10b0fd662:	retq   
0x10b0fd663:	add    %dl,(%rax)
0x10b0fd665:	add    %al,(%rax)
0x10b0fd667:	add    %bh,%bh
0x10b0fd669:	(bad)  

(gdb) x/4w  0x10b0fd624
0x10b0fd624:	0x3d16058b	0xc9300730	0xc7158d48	0x48000001

The relevant allocation:

ALLOC 0x7fdd12401340-0x7fdd12401343 [size=4]: thread_7fff72dbe960 |s258_info | loadObj | loadOc | ocGetNames_MachO | stgCallocBytes | calloc | malloc_zone_calloc

rts/Linker.c truncates the pointer calloc returned (the offset is declared uint32_t):

        if((sections[i].flags & SECTION_TYPE) == S_ZEROFILL)
        {
            char * zeroFillArea = stgCallocBytes(1,sections[i].size,
                                      "ocGetNames_MachO(common symbols)");
            sections[i].offset = zeroFillArea - image;
        }

From /usr/include/mach-o/loader.h:

struct section { /* for 32-bit architectures */
        char            sectname[16];   /* name of this section */
        char            segname[16];    /* segment this section goes in */
        uint32_t        addr;           /* memory address of this section */
        uint32_t        size;           /* size in bytes of this section */
        uint32_t        offset;         /* file offset of this section */
        uint32_t        align;          /* section alignment (power of 2) */
        uint32_t        reloff;         /* file offset of relocation entries */
        uint32_t        nreloc;         /* number of relocation entries */
        uint32_t        flags;          /* flags (section type and attributes)*/
        uint32_t        reserved1;      /* reserved (for offset or index) */
        uint32_t        reserved2;      /* reserved (for count or sizeof) */
};

Even if it weren't truncated, the RIP-relative displacement is still required to fit into 32 bits. The analysis given in a comment in rts/Linker.c (see the bit starting "on x86_64 we have a problem with relocating symbol references") is also relevant to this situation.

[nus]

Err, rts/Linker.c actually uses section_64 in this case:

#if x86_64_HOST_ARCH || powerpc64_HOST_ARCH
#define mach_header mach_header_64
#define segment_command segment_command_64
#define section section_64
#define nlist nlist_64
#endif

struct section_64 { /* for 64-bit architectures */
        char            sectname[16];   /* name of this section */
        char            segname[16];    /* segment this section goes in */
        uint64_t        addr;           /* memory address of this section */
        uint64_t        size;           /* size in bytes of this section */
        uint32_t        offset;         /* file offset of this section */
        uint32_t        align;          /* section alignment (power of 2) */
        uint32_t        reloff;         /* file offset of relocation entries */
        uint32_t        nreloc;         /* number of relocation entries */
        uint32_t        flags;          /* flags (section type and attributes)*/
        uint32_t        reserved1;      /* reserved (for offset or index) */
        uint32_t        reserved2;      /* reserved (for count or sizeof) */
        uint32_t        reserved3;      /* reserved */
};

[nus]

This involves multiple archithectures/OS'es, changing the ticket properties. See the following example:

#include <stdio.h>
#include "global.h"

#if defined(__MINGW32__)
# if defined(__MINGW64_VERSION_MAJOR) && defined(__MINGW64__)
#    define ARR_SIZE 0x0fffffff /* works when linked by ghc, fails with ghci */
/* #    define ARR_SIZE 0x1fffffff */ /* bad executable when linked by ghc, fails to link with ghci */
# else
#    define ARR_SIZE 0x1 /* this, and anything bigger fails to link with ghci, works with ghc */
# endif
#elif defined(__APPLE__)
# if defined(__i386__)
#    define ARR_SIZE 0x0fffffff /* works */
/* #    define ARR_SIZE 0x1fffffff */ /* works when linked by ghc, causes failure in malloc at ghci run-time */
/* #    define ARR_SIZE 0x2fffffff */ /* refused by llvm-gcc as too large */
# elif defined(__x86_64__)
#    define ARR_SIZE 0xffffffff /* works when linked by ghc, causes failure in malloc at ghci run-time */
# endif
#elif defined(__linux__)
# if defined(__i386__)
 #    define ARR_SIZE 0x1fffffff
/* #    define ARR_SIZE 0x2fffffff */ /* refused by gcc as too large */
# elif defined(__x86_64__)
/* #    define ARR_SIZE 0x1fffffff */ /* works */
/* #    define ARR_SIZE 0x2fffffff */ /* works when linked by ghc, causes failure in malloc at ghci run-time */
/* #    define ARR_SIZE 0x3fffffff */ /* SIGKILL'ed when linked by ghc, causes failure in malloc at ghci run-time */
# endif
#endif

int x[ARR_SIZE];
void printx() {
  printf("ARR_SIZE: %x\n", ARR_SIZE);
  x[0] = 0;
  printf("x: %d\n", x[0]);
  x[0] = 1;
  printf("x: %d\n", x[0]);
}

The versions the example was tested on:

Windows 7 SP1 x64.
  x32:
    CC: i686-w64-mingw32-gcc-4.6.3-release-win32_rubenvb.7z
    GHC: ghc-7.4.1-i386-windows.exe
  x64:
    CC: x86_64-w64-mingw32-gcc-4.6.3-release-win64_rubenvb.7z
    GHC: ghc-7.5.20120617-i386-windows.exe; HEAD("Project version","7.5.20120705")("Booter version","7.5.20120617")

Mac OS X 10.7.3.
  x32:
    CC: gcc -m32 (i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.9.00))
    GHC: ghc-7.4.2-i386-apple-darwin.tar.bz2
  x64:
    CC: gcc (i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.9.00))
    GHC: ghc-7.4.2-x86_64-apple-darwin.tar.bz2

Linux Ubuntu 12.04 LTS x64.
  x32:
    CC: gcc -m32 (gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3)
    GHC: ghc-7.4.2-i386-unknown-linux.tar.bz2
  x64:
    CC: gcc (gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3)
    GHC: ghc-7.4.2-x86_64-unknown-linux.tar.bz2

[simonmar]

We should probably be using mmap to allocate the bss on Linux & OS X. We already have a bag of tricks to try to persuade mmap to give us memory in the low 2Gb on a 64-bit platform; see mmapForLinker() in Linker.c.

[luite]

this fixes it on my system (OS X 10.8 x86_64)

[luite]

the above patch uses mmap to allocate the section on OS X and fixes the crash here. It will likely still crash if there are multiple huge global arrays like in the example by nus (that exhaust the lower 2GB of mem), but that scenario is probably not exactly common.

It works on my system with Mountain Lion (10.8), i can't test with older systems unfortunately.

[simonmar]

Looks good; I will test and push.

[luite]

Any news on this? Looks like the patch hasn't been applied yet. I'd really appreciate it if this could make it into 7.6.1. Anything i can do to help?

[marlowsd@…]

commit e590ad77f9596a8389409ae56ea902c97e5dbfb0

Author: Simon Marlow <marlowsd@gmail.com>
Date:   Mon Aug 20 13:14:47 2012 +0100

    OS X: use mmap() instead of malloc for allocating the bss (#7040)

 rts/Linker.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

[simonmar]

let's get this merged into 7.6.1

[pcapriotti]

Merged as 29ec96c89d19c1b40a8990466424ff35be096780.