ghc can't load files with selinux Enforcing

Is this at all related to #703?

I have no idea what SELinux "enforcing" mode does. It looks like SELinux doesn't like us using mprotect() to make dynamically-allocated memory executable. This is required for things like 'foreign import "wrapper"', because we have to generate dynamic code.

It's possible we could mmap() instead, I suppose.

Can anyone shed any more light here?

More comments from Jon:

Is this at all related to #703?

No idea.

I have no idea what SELinux "enforcing" mode does.

It enforces the policies... I think permissive mode just logs things, but enforcing mode actually stops them.

It looks like SELinux doesn't like us using mprotect() to make dynamically-allocated memory executable. This is required for things like 'foreign import "wrapper"', because we have to generate dynamic code.

The audit log entry in Enforcing mode is this:

type=AVC msg=audit(1144148747.937:6073): avc:  denied  { execheap } for  pid=18253 comm="ghc-6.4.1" scontext=user_u:system_r:unconfined_t:s0 tcontext=user_u:system_r:unconfined_t:s0 tclass=process

whereas in Permissive mode I find this:

type=AVC msg=audit(1144148449.336:5974): avc:  denied  { execheap } for  pid=18056 comm="ghc-6.4.1" scontext=user_u:system_r:unconfined_t:s0 tcontext=user_u:system_r:unconfined_t:s0 tclass=process

ie the same, except that ghci loads the file OK.

Can anyone shed any more light here?

Not much; I can't say I understand SELinux, but I think the answer is probably in here:

http://people.redhat.com/drepper/selinux-mem.html

It's possible we could mmap() instead, I suppose.

It looks like you have to do that, and even so will need to take steps to avaid getting an execmem denial.

Workaround from the mailing list (volodimir.rudenko@gmail.com):

In GUI:

• Menu System -> Administration -> Security Level and Firewall -> tab SELinux,
• in the tree control open an item Other
• turn on allow_execheap

On command line (as root):

       setsebool -P allow_execheap 1

There are three related "booleans" to try (just in case the trick above does not help)

 allow_execmem
 allow_execmod
 allow_execstack

PS. This works for the targeted selinux policy, which is default in Fedora 5. There are also strict and mls policies. I am not selinux guru -I do not know if my workaround works for those policies.

Fixed in the HEAD and the 6.4 branch, but I haven't been able to test (don't have SE Linux on a local machine). Please download a snapshot and try it out, if possible.

The same (or at least similar) occurs on x86 with GHC 6.6.1. The compiler cannot even start.

$ ghc
ghc-6.6.1: internal error: getMBlock: mmap: Permission denied
    (GHC version 6.6.1 for i386_unknown_linux)
    Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug
$ strace ghc
... /* skipped */ ...
mmap2(NULL, 2097152, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 EACCES (Permission denied)
write(2, "ghc-6.6.1: internal error: ", 27ghc-6.6.1: internal error: ) = 27

GHC 6.6.1 from Debian package

Distro: Debian GNU/Linux Unstable (Lenny)

Arch: x86

Kernel: vanilla 2.6.22.4 Linux kernel

SELinux policy: refpolicy-targeted (from Debian package)

I would like to provide additional details, but I can't understand how to debug GHC. gdb shows completely brain-damaged stack dump.

The error I mentioned was caused by mmap-ing memory with PROT_EXEC and PROT_WRITE permissions simultaneously, which security people consider as bad thing.

This mmap resides at rts/MBlock.c:194

   ret = mmap(addr, size, PROT_READ | PROT_WRITE | PROT_EXEC,
MAP_ANON | MAP_PRIVATE, -1, 0);

Just for experiment, I removed PROT_EXEC from this line and tried to recompile GHC (this time at x86_64 platform). To my suprise it compiled well. I've installed the compiled GHC and tried to build some programs using it. It compiled all of my programs and they also worked fine.

So, the question is, does GHC really need executable memory here?

However, I found that GHCi become broken (but in a different way). At starts it says:

$ ghci
   ___         ___ _
  / _ \ /\  /\/ __(_)
 / /_\// /_/ / /  | |      GHC Interactive, version 6.6.1, for Haskell 98.
/ /_\\/ __  / /___| |      http://www.haskell.org/ghc/
\____/\/ /_/\____/|_|      Type :? for help.

ghc-6.6.1: internal error: loadObj: can't map `/usr/lib/ghc-6.6.1/HSbase.o'
    (GHC version 6.6.1 for x86_64_unknown_linux)
    Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug
Aborted
$

Running under strace shows following:

$ strace /usr/lib/ghc-6.6.1/ghc-6.6.1 -B/usr/lib/ghc-6.6.1 --interactive
/* ... some info skipped ... */
stat("/usr/lib/ghc-6.6.1/HSbase.o", {st_mode=S_IFREG|0644, st_size=10789086, ...}) = 0
open("/usr/lib/ghc-6.6.1/HSbase.o", O_RDONLY) = 5
mmap(NULL, 10792960, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|0x40, 5, 0) = -1 EACCES (Permission denied)
write(2, "ghc-6.6.1: internal error: ", 27ghc-6.6.1: internal error: ) = 27
write(2, "loadObj: can\'t map `/usr/lib/ghc"..., 48loadObj: can't map `/usr/lib/ghc-6.6.1/HSbase.o') = 48

The offending mmap is located rts/Linker.c:1354

   oc->image = mmap(map_addr, n, PROT_EXEC|PROT_READ|PROT_WRITE,
                    MAP_PRIVATE|EXTRA_MAP_FLAGS, fd, 0);
   if (oc->image == MAP_FAILED)
      barf("loadObj: can't map `%s'", path);

Obviously PROT_EXEC and PROT_WRITE is the thing SELinux doesn't like. Does this code really need PROT_EXEC here?

May be it possible to map image with PROT_WRITE and without PROT_EXEC, fix up all relocation/jumps/PLT/all-this-linker-stuff, and then remap it with PROT_EXEC but without PROT_WRITE?

Another solution is mentioned at http://people.redhat.com/drepper/selinux-mem.html (see section "Example code to avoid execmem violations").

All major Linux distributions are slowly adopting SELinux, so the problem may become quite annoying.

Remember that at some point we *do* need writeable and executable memory (for Haskell callbacks, see createAdjustor in Adjustor.c). There have been various attempts to fix problems related to newer security concepts, but obviously we are still not where we want to be. So the basic question is: How can we tell the underlying OS "It is OK for the GHC RTS to generate executable code?". How do other language implementations with e.g. JITs solve this problem?

We have already worked around this problem in libffi. The full gory details are not appropriate here, but there's a patch at

http://gcc.gnu.org/ml/java-patches/2007-q1/msg00567.html

I don't know if you are using libffi or using your own Foreign Function library to generate your callbacks. If you use libffi it's just a matter of upgrading; if you don't use libffi, I strongly recommend that you consider doing so.

This will become important as more distributions turn on SELinux. #793 is the ticket for switching to libffi, currently pending someone doing a feasibilty study.

From the program point of view there is commentry here about how to avoid this by doing two separate mappings, one writeable and one executable.

http://people.redhat.com/drepper/selinux-mem.html

That supposedly increases security but looks a bit ugly to me but since I guess it's only needed in one place it might not be too bad???

From the local operating system point of view, you have to change the binary to have a context which allows executable memory. You can either look on a fedora system where darcs works or you can find an appropriate context with.

sesearch --allow --target execmem

and what we work out is that probably we want unconfined_execmem_exec_t so the command

chcon -t unconfined_execmem_exec_t /usr/bin/darcs

fixes this problem. (experiment with doing "chcon -t bin_t /usr/bin/darcs" to break it again)

Michael De La Rue; posting as guest till I have better working private mail.

Here's a link to the upstream bug documenting this issue in Fedora Linux: https://bugzilla.redhat.com/show_bug.cgi?id=451877

Just a general question on this issue - what is the value of SELinux's execmem policy, given that it can be worked around? The workaround is extremely annoying, because we have to map the same memory twice. Even libffi can't abstract the details, because we have to manage two addresses to the same memory.

I think the point is that there is no area of memory that is both writable and executable at the same time. Apparently common stack/heap overflow exploits rely on this. I guess the idea is that some exploits rely on being able to trick the program they are attacking into writing some bit of memory and then running from it, but the attacker is not able to trick the program into making system calls to change memory protection.

There is still the issue that the attacker could work out the two addresses where the same physical memory is available with different protections but this is probably harder, especially with mmap address randomisation.

Other systems that use libffi will also have this problem with selinux so it might be worth seeing how they do it.

Couple of notes:

• Fedora is shipping with allow_execmem turned on these days, so the issue only affects people that turn it off manually.

• In order to reproduce the problem, you need to turn off both allow_execmem and allow_execstack using setsebool.

Ok, it turned out to be not as hard as I thought it might be. But libffi doesn't appear to support doing executable allocation on Windows, so we had to retain the old mechanism for that. I wasn't sure whether it would work on the other Unix-like OSs either, so I've enabled the libffi allocator only for Linux at the moment.

Fri Sep 19 14:46:02 BST 2008  Simon Marlow <marlowsd@gmail.com>            
  * On Linux use libffi for allocating executable memory (fixed #738)