hGetBufSome blocks when all available input is buffered (on Windows only)

[joeyadams]

See the attached program. On Windows, it blocks on hGetSome.

If I take out the hGetLine and hWaitForInput, hGetSome does not block. If I add <code>hSetBuffering h NoBuffering?</code>, hGetBufSome still blocks after hWaitForInput because it does some buffering when it calls hLookAhead_.

[joeyadams]

Sorry, I pressed Create Ticket when I meant to press Preview.

I did some debug traces, and found that the flow of calls is as follows:

hGetSome hGetBufSome bufReadNBNonEmpty (second occurrence) bufReadNBEmpty Buffered.fillReadBuffer0

The Buffered.fillReadBuffer0 call blocks on Windows, even though it shouldn't:

  -- | reads bytes into the buffer without blocking.  Returns the
  -- number of bytes read (Nothing indicates end-of-file), and the new
  -- buffer.
  fillReadBuffer0   :: dev -> Buffer Word8 -> IO (Maybe Int, Buffer Word8)

Digging deeper, I think fillReadBuffer0 ultimately boils down to a call to fdReady. But I'm not 100% sure.

hWaitForInput, which also calls fdReady, does in fact work on Windows. Perhaps readRawBufferPtrNoBlock isn't calling fdReady correctly. Here is how GHC.IO.FD.ready calls it (for FD):

#if defined(mingw32_HOST_OS)
                          (fromIntegral $ fromEnum $ fdIsSocket fd)
#else
                          0
#endif

And here is how readRawBufferPtrNoBlock calls it:

  | otherwise    = do r <- unsafe_fdReady (fdFD fd) 0 0 0

[simonmar]

Thanks for the report.

[duncan]

See GHC/IO/FD.hs, in the "FD operations" section, in the ! mingw32_HOST_OS case:

readRawBufferPtrNoBlock :: String -> FD -> Ptr Word8 -> Int -> CSize -> IO CInt
readRawBufferPtrNoBlock = readRawBufferPtr

writeRawBufferPtrNoBlock :: String -> FD -> Ptr Word8 -> Int -> CSize -> IO CInt
writeRawBufferPtrNoBlock = writeRawBufferPtr

There does not seem to be any non-blocking impl here for windows (where there is one for unix just above).

[simonmar]

Duncan: we don't have a way to do non-blocking I/O on Windows, but hGetSome is supposed to work around that.

There is probably some fault in the logic in bufReadNBNonEmpty, because it is not supposed to try to read any more data when called by hGetBufSome, see the comment:

                                       bufReadNBNonEmpty h_ buf' (castPtr ptr) 0 (min r count)
                                        -- new count is  (min r count), so
                                        -- that bufReadNBNonEmpty will not
                                        -- issue another read.

[simonmar]

I see it: the second call to bufReadNBNonEmpty in hGetBufSome is passing count along unchanged, it should use the minimum of the available buffer elements and count. I don't have a handy Windows build to test this; Paulo, could you try?

[pcapriotti]

Thanks Simon, that does indeed fix the problem, at least in a simple test-case with stdin.

I'll test it on the original socket example before pushing.

[pcapriotti]

Pushed as 370fc0b455f6a03283fbd5c0baa5d08d9115379d.

[pcapriotti]

Merged as bd399cef9d91060a3d733be4c18cb901a2efbbc8.