GHC's runtime never terminates unused worker threads

[Remi]

When concurrently calling safe FFI functions, worker OS threads are created. These threads then never quit.

The following toy program creates 30k OS threads (which is fine because that's exactly what it asks for) which are then never "garbage collected": 30k threads and over 230g of VM are hanging around until the program exits.

{-# LANGUAGE ForeignFunctionInterface #-}
module Main where

import Control.Concurrent
import Control.Monad
import Foreign.C.Types
import System.Mem

foreign import ccall safe sleep :: CUInt -> IO ()

main = do
    replicateM_ 30000 $ forkIO $ sleep 2
    getLine
    -- do other stuff

P.S. Of course I should simply use threadDelay in this case. The real program performs up to a few hundred concurrent fdatasync calls.

[ezyang]

Here is a somewhat refined test-case, though POSIX only.

{-# LANGUAGE ForeignFunctionInterface #-}
module Main where

import Control.Concurrent
import Control.Monad
import Foreign.C.Types
import System.Mem
import System.Posix.Process
import System.Directory
import Control.Concurrent.QSem

foreign import ccall safe sleep :: CUInt -> IO ()

main = do
    let amount = 200
    qsem <- newQSem 0
    replicateM_ amount . forkIO $ (sleep 2 >> signalQSem qsem)
    replicateM_ amount $ waitQSem qsem
    -- POSIX only: check thread usage manually
    pid <- getProcessID
    let dir = "/proc/" ++ show pid ++ "/task"
    contents <- getDirectoryContents dir
    let status = length contents - 2 -- . and ..
    print status

[ezyang]

See  http://www.haskell.org/pipermail/glasgow-haskell-users/2010-November/019503.html

[simonmar]

Thanks for the patch, I'll review.

[simonmar]

Thanks. I reworked the patch a bit, and pushed my version today:

Thu Nov 25 05:57:29 PST 2010  Simon Marlow <marlowsd@gmail.com>
  * Keep a maximum of 6 spare worker threads per Capability (#4262)