hSetBuffering stdin NoBuffering doesn't work on Windows

Thanks for the report!

It works for me on Linux; I can't test on Windows at the moment.

OK, I can reproduce the problem. At the Haskell level the problem is that asyncRead# isn't returning when a key is pressed; I haven't followed all the RTS stuff through to see where it goes wrong. Could be related to #806. I doubt we'll get to this for 6.8.3, so I'm marking it as low priority.

This is important to folks here at Galois; I'll try to look at it before 6.8.3.

There is code in base/cbits/consUtils.c:set_console_buffering to disable Windows line buffering (by calling the Win32 function SetConsoleMode). However, the code in GHC.Handle which is supposed to call that function was #ifdef'ed out on mingw in the following commit:

http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/libraries/base/GHC/Handle.hs#rev1.14

From the comments in System.Posix.Internals:

-- 'raw' mode for Win32 means turn off 'line input' (=> buffering and
-- character translation for the console.) The Win32 API for doing
-- this is GetConsoleMode(), which also requires echoing to be disabled
-- when turning off 'line input' processing. Notice that turning off
-- 'line input' implies enter/return is reported as '\r' (and it won't
-- report that character until another character is input..odd.) This
-- latter feature doesn't sit too well with IO actions like IO.hGetLine..

I'm not that experienced with Windows programming, but maybe instead of changing the ConsoleMode, we could call the lower-level ReadConsoleInput/PeekConsoleInput when we want to read only one character at a time.

See also #2568.

This is also very important for those of us working on FRP at Anygma since it precludes simple console-based FRP examples.

I'd also find this useful

Alistar Bayley noted this workaround in haskell-cafe

{-# LANGUAGE ForeignFunctionInterface #-}
import Data.Char
import Control.Monad (liftM)
import Foreign.C.Types

getHiddenChar = liftM (chr.fromEnum) c_getch
foreign import ccall unsafe "conio.h getch"
  c_getch :: IO CInt

Unfortunately conio doesn't mix with ordinary IO, as I demonstrated in a response to that thread on glasgow-haskell-users.

As for the original problem, I think this'd take some work to solve: GHC would have to convert to the Win32 API for all of its IO on Windows. (That'd also help with #806.)

What currently happens is the following call chain, starting from hGetChar (a bit of documentation for any would-be fixers):

System.IO.hGetChar stdin

-- meanings of numbers: stdin FD, not a socket, offset 0, length 1
GHC.Handle.readRawBuffer "hGetChar" 0 0 <buffer> 0 1

GHC.Handle.asyncReadRawBuffer 0 0 <buffer> 0 1

GHC.Conc.asyncReadBA 0 0 1 0 <buffer>

-- offset got applied
GHC.Conc.asyncRead 0 0 1 <buffer>

-- asyncReadzh_fast in rts/PrimOps.cmm
asyncRead# 0 0 1 <buffer>

-- in rts/win32/AsyncIO.c
-- the new 0 signifies that this is a read and not a write
addIORequest(0, 0, 0, 1, <buffer>)

From there it ends up into the asynchronous IO work queue, whence it eventually gets picked up by IOWorkerProc (in rts/win32/IOManager.c). It notices that the workKind is WORKER_READ but not WORKER_FOR_SOCKET, so it does a plain read() call.

In the current situation that's basically fine, since the hSetBuffering never did anything and we're still in line buffered mode. In unbuffered mode, that'd be a problem, as the comment from System.Posix.Internals (which judah posted above) asserts: enter needs to be pressed twice. The fact that it gives '\r' instead of '\n' isn't such a big problem since it can be easily modified.

I'm attaching a C program which shows how the problem is avoided by using the Win32 API directly (in this case, ReadFile is fine) instead of the POSIX read: the latter requires two presses of enter, the former only one.

Since there seems to be no way of getting a Win32 HANDLE object from a C FILE* let alone a POSIX file descriptor, I believe that the only reasonable way of getting this and #806 to work reliably is to convert the whole IO subsystem to use the Windows API directly—starting from changing GHC.IOBase.Handle__.haFD from an FD to a HANDLE on Windows.

read vs. ReadFile?: handling of enter varies

First, I completely agree that we should be using the Win32 API directly instead of the POSIX compatibility layer. It's that way for historical reasons - I think it was easier to port the IO library to Windows in the first place by using the POSIX layer. As you say, #806 would be helped by that, but also the Windows side of #635, and #989.

The call sequence you posted only applies to the non-threaded RTS. With the threaded RTS, everything is in the IO library. We still end up calling read(), but directly from Haskell.

Incedentally, there is a way to get a HANDLE from a file descriptor: _get_osfhandle(fd). You might want to look at libraries\base\cbits\inputReady.c for some serious console hackery.

Heh, I just noticed that _get_osfhandle is used about 30 lines down from the code I was looking at last time. Thanks for the heads up. Using it this might be fixable with a lot less work by just changing the read call in IOWorkerProc to a ReadFile.

The threaded RTS seems to end up in __hscore_PrelHandle_read in libraries/base/include/HsBase.h which is again just a read call and might be convertable to ReadFile without too much trouble.

We'd like to see this solved, but it looks as if it'd be a lot of work for a relatively narrow case. Can anyone else help? Meanwhile, reducing priority.

Simon & Simon

I believe I have a trivial fix for this, and can forward for testing & commit, if still of interest.

Replying to sof:

I believe I have a trivial fix for this, and can forward for testing & commit, if still of interest.

Yes please!

ok, not futzing with making this a proper patch:

--- old-base/cbits/consUtils.c  2009-02-25 10:06:10.631125000 -0800
+++ new-base/cbits/consUtils.c  2009-02-25 10:06:10.646750000 -0800
@@ -25,10 +25,13 @@
     DWORD flgs = ENABLE_LINE_INPUT | ENABLE_ECHO_INPUT;

     if ( (h = (HANDLE)_get_osfhandle(fd)) != INVALID_HANDLE_VALUE ) {
+      /* Only for console-connected Handles */
+      if ( GetFileType(h) == FILE_TYPE_CHAR ) {
        if ( GetConsoleMode(h,&st) &&
-            SetConsoleMode(h, cooked ? (st | ENABLE_LINE_INPUT) : st & ~flgs) ) {
+            SetConsoleMode(h, cooked ? (st | flgs) : st & ~flgs)  ) {
            return 0;
        }
+      }
     }
     return -1;
 }

--- old-base/GHC/Handle.hs      2009-02-25 10:06:10.631125000 -0800
+++ new-base/GHC/Handle.hs      2009-02-25 10:06:10.646750000 -0800
@@ -1374,13 +1374,10 @@
           is_tty <- fdIsTTY (haFD handle_)
           when (is_tty && isReadableHandleType (haType handle_)) $
                 case mode of
-#ifndef mingw32_HOST_OS
-        -- 'raw' mode under win32 is a bit too specialised (and troublesome
-        -- for most common uses), so simply disable its use here.
+                   -- Note: we used to disable 'cooked' mode setting
+                   -- for mingw / win32 here, but it is now back on (and well
+                   -- behaved for Console-connected Handles.)
                   NoBuffering -> setCooked (haFD handle_) False
-#else
-                  NoBuffering -> return ()
-#endif
                   _           -> setCooked (haFD handle_) True

           -- throw away spare buffers, they might be the wrong size

i.e., reliably (un)setting the line buffering flags of the underlying Console handle (if there's one) takes care of this.

Patch applied:

Thu Mar  5 03:33:23 PST 2009  Simon Marlow <marlowsd@gmail.com>
  * FIX #2189: re-enabled cooked mode for Console-connected Handles on Windows

Merged

We rolled back this patch: With it, it is not possible to type in ghci running from a cygwin window or a cmd window (but an msys window or SSHing into cygwin works fine).

Sigbjorn, any ideas on what the problem is and how to solve it?