Ticket #3808: 3808.dpatch

2 patches for repository http://code.haskell.org/~dons/code/bytestring:

Mon Mar 22 15:39:14 GMT 2010  Simon Marlow <marlowsd@gmail.com>
  * hGetContents: use hGet instead of hGetNonBlocking + hWaitForInput + hIsEOF
  
  Not only is this cleaner, but it fixes a problem with read binary data
  over a pipe, see
    http://hackage.haskell.org/trac/ghc/ticket/3808
  
  The problem is that bytestring normally works fine when the Handle is
  not in binary mode, because it uses hGetBuf/hPutBuf which bypass the
  encoding layer.  That is, except in this one particular case:
  hWaitForInput might do some decoding, because it has to determine
  whether there are any characters (not bytes) available to be read.
  
  I imagine hGetNonBlocking was used due to concerns that hGet might
  block if there is insufficient data, but that's not the case; hGet
  returns a short read if it would otherwise block, and only blocks if
  there is no data to read.

Mon Mar 22 15:41:03 GMT 2010  Simon Marlow <marlowsd@gmail.com>
  * update docs for hGet, hGetNonBlocking

New patches:

[hGetContents: use hGet instead of hGetNonBlocking + hWaitForInput + hIsEOF
Simon Marlow <marlowsd@gmail.com>**20100322153914
 Ignore-this: 43bac03fc6d7de74564f5cd608ed9f17
 
 Not only is this cleaner, but it fixes a problem with read binary data
 over a pipe, see
   http://hackage.haskell.org/trac/ghc/ticket/3808
 
 The problem is that bytestring normally works fine when the Handle is
 not in binary mode, because it uses hGetBuf/hPutBuf which bypass the
 encoding layer.  That is, except in this one particular case:
 hWaitForInput might do some decoding, because it has to determine
 whether there are any characters (not bytes) available to be read.
 
 I imagine hGetNonBlocking was used due to concerns that hGet might
 block if there is insufficient data, but that's not the case; hGet
 returns a short read if it would otherwise block, and only blocks if
 there is no data to read.
] {
hunk ./Data/ByteString/Lazy.hs 216
 import Data.Word                (Word8)
 import Data.Int                 (Int64)
 import System.IO                (Handle,stdin,stdout,openBinaryFile,IOMode(..)
-                                ,hClose,hWaitForInput,hIsEOF)
-import System.IO.Error          (mkIOError, ioError, illegalOperationErrorType)
+                                ,hClose)
+import System.IO.Error          (mkIOError, illegalOperationErrorType)
 import System.IO.Unsafe
 #ifndef __NHC__
 import Control.Exception        (bracket)
hunk ./Data/ByteString/Lazy.hs 1177
     lazyRead = unsafeInterleaveIO loop
 
     loop = do
-        c <- S.hGetNonBlocking h k
-        --TODO: I think this should distinguish EOF from no data available
-        -- the underlying POSIX call makes this distincion, returning either
-        -- 0 or EAGAIN
+        c <- S.hGet h k -- only blocks if there is no data available
         if S.null c
hunk ./Data/ByteString/Lazy.hs 1179
-          then do eof <- hIsEOF h
-                  if eof then hClose h >> return Empty
-                         else hWaitForInput h (-1)
-                           >> loop
+          then hClose h >> return Empty
           else do cs <- lazyRead
                   return (Chunk c cs)
 
}
[update docs for hGet, hGetNonBlocking
Simon Marlow <marlowsd@gmail.com>**20100322154103
 Ignore-this: b7b1986f438421615985a4587f97a58c
] {
hunk ./Data/ByteString.hs 1914
 -- is far more efficient than reading the characters into a 'String'
 -- and then using 'pack'. First argument is the Handle to read from, 
 -- and the second is the number of bytes to read. It returns the bytes
--- read, up to n, or EOF.
+-- read, up to n, or 'null' if EOF has been reached.
+--
+-- If there is any data to read, then 'hGet' will not block, instead
+-- it will return whatever data is available without blocking.  It
+-- only blocks if there is no data available to read.
 --
 -- 'hGet' is implemented in terms of 'hGetBuf'.
 --
hunk ./Data/ByteString.hs 1931
     | i == 0    = return empty
     | otherwise = illegalBufferSize h "hGet" i
 
--- | hGetNonBlocking is identical to 'hGet', except that it will never block
--- waiting for data to become available, instead it returns only whatever data
--- is available.
+-- | hGetNonBlocking is identical to 'hGet', except that it will never
+-- block waiting for data to become available.  If there is no data
+-- available to be read, 'hGetNonBlocking' returns 'null'.
 --
 hGetNonBlocking :: Handle -> Int -> IO ByteString
 #if defined(__GLASGOW_HASKELL__)
}

Context:

[Check for negative lengths in packCStringLen
Duncan Coutts <duncan@haskell.org>**20090915234111
 Ignore-this: a388aeeee7cd0b6df06c61d88a97588a
] 
[Check for negative lengths in hGet and hGetNonBlocking
Duncan Coutts <duncan@haskell.org>**20090915234011
 Ignore-this: c9e33f05ef3cabfe12c1d8fb509f4726
 See ghc ticket #3514
] 
[Fix elemIndices and split
Duncan Coutts <duncan@haskell.org>**20090912203950
 Ignore-this: 69ae945462b5638c99cc1824ed329e90
 They were using inlinePerformIO to lazily delay IO actions within the
 scope of a withForeignPtr. Thus the ptr was still in use after the
 ForeignPtr went out of scope. This lead to dangling pointers, incorrect
 results and segfaults. Nasty. See ghc tickets #3486 and #3487.
 Audited the rest of the code base for the same anit-pattern.
 Thanks to nwn and people on the haskell-jp mailing list for reporting
 the bug with nice test cases and also to Ian, Bertram and Don for
 diagnosing the source of the problem.
] 
[Update "tested-with" list
Duncan Coutts <duncan@haskell.org>**20090823195516
 Ignore-this: 873e64dd5c978cc12751d3b404e3619a
 It builds with ghc-6.4 through to 6.11 (23/08/09)
 It builds with no warnings with ghc 6.8, 6.10 and 6.11.
] 
[Clean up the language extension pragmas
Duncan Coutts <duncan@haskell.org>**20090823194920
 Ignore-this: c7fb9daf30b639d0bee909fd396de241
 We cannot actually specify them correctly, see ghc ticket #3457.
 We can at least simplify things and make it clear it does not work,
 rather than having misleading code that looks like it might work.
] 
[Fix import warnings
Duncan Coutts <duncan@haskell.org>**20090823194733
 Ignore-this: 2db4b086627d428de7add92566df9eb
 Patches merged from ghc's fork:
   Tue Jul  7 12:58:17 BST 2009  Ian Lynagh <igloo@earth.li>
     * Remove unused imports
 
   Tue Jul  7 14:35:56 BST 2009  Ian Lynagh <igloo@earth.li>
     * Fix unused import warnings
 
   Wed Jul 22 13:56:04 BST 2009  Simon Marlow <marlowsd@gmail.com>
     * fix compilation with GHC 6.10
] 
[Update for new IO library
Duncan Coutts <duncan@haskell.org>**20090823175302
 Ignore-this: 539da275e4528629baa6d9f66c2f3335
 Merge of patch from ghc's fork:
   Thu Jun 11 15:09:37 BST 2009  Simon Marlow <marlowsd@gmail.com>
     * Update for new IO library
] 
[Fix "cabal check" warnings
Duncan Coutts <duncan@haskell.org>**20090823145438
 Ignore-this: f6a17b693d9216b840bf01cf55a2fa7d
 Merge of patch from ghc's fork:
   Tue Aug 11 22:58:58 BST 2009  Ian Lynagh <igloo@earth.li>
     * Fix "Cabal check" warnings
] 
[Remove INLINE pragmas on recursive functions
Duncan Coutts <duncan@haskell.org>**20090823145127
 Ignore-this: 40b0b5e7a1c274b78231517736ab74c7
 Merge of patch from ghc's fork:
   Fri Dec  5 17:04:52 GMT 2008  simonpj@microsoft.com
     * Remove INLINE pragmas on recursive functions
 
     GHC now complains about INLINE pragmas on recursive functions,
     and they were no-ops before in any case
] 
[Fix some "warn-unused-do-bind" warnings where we want to ignore the value
Ian Lynagh <igloo@earth.li>**20090710202441] 
[Hide the .Internal modules in the haddock docs
Duncan Coutts <duncan@haskell.org>**20090702000819
 This should stop users accidentally using the semi-public
 modules while still leaving them available for extension
 packages. Also means that references to 'ByteString' go to
 a sensible place rather than the .Internal modules.
] 
[Fix docs for unsafePackAddressLen, it doesn't need null-termination
Duncan Coutts <duncan@haskell.org>**20090629192811] 
[Make docs for fromForeignPtr point to suitable public alternatives
Duncan Coutts <duncan@haskell.org>**20090629191336] 
[Improve the top-level documentation for the Unsafe and Internal modules
Duncan Coutts <duncan@haskell.org>**20090629191017
 Clarify the purpose and properties of the functions in the two modules.
 In particular try to disuade people from using the Internal functions
 unnecessarily.
] 
[Use comma separated lists in the LANGUAGE pragmas
Duncan Coutts <duncan@haskell.org>**20090629190950
 Haddock complains.
] 
[bump versoin
Don Stewart <dons@galois.com>**20090108182815] 
[Add a test for the laziness of readInt and readInteger over lazy ByteStrings
Bryan O'Sullivan <bos@serpentine.com>**20081231070009] 
[Fix strictness bugs in readInt and readInteger.
Bryan O'Sullivan <bos@serpentine.com>**20081231064853] 
[Add TODO item to eliminate -fno-warn-orphans
Duncan Coutts <duncan@haskell.org>**20081101190200
 Orphan instances in libraries are not a good thing.
] 
[Make everything build with ghc-6.6 and 6.4
Duncan Coutts <duncan@haskell.org>**20081101190029
 No code changes, just cpp for new LANGUAGE pragmas
 and not using a RULE with a funky LHS for ghc-6.4 which didn't allow it.
] 
[avoid import renaming errors in nhc98
Malcolm.Wallace@cs.york.ac.uk**20081008092026] 
[Import fix for nhc98
Duncan Coutts <duncan@haskell.org>**20081006211934] 
[Bump version to 0.9.1.4
Duncan Coutts <duncan@haskell.org>**20081006210002] 
[Drop dependency on syb with base 4
Duncan Coutts <duncan@haskell.org>**20081006205908] 
[fix import of Control.Exception for nhc98
Malcolm.Wallace@cs.york.ac.uk**20081006102248] 
[TAG 0.9.1.3
Duncan Coutts <duncan@haskell.org>**20081004225748] 
[Bump version to 0.9.1.3
Duncan Coutts <duncan@haskell.org>**20081004225716] 
[Whitespace changes to the package description
Duncan Coutts <duncan@haskell.org>**20081004223529] 
[Drop -fglasgow-exts, use LANGUAGE pragmas
Duncan Coutts <duncan@haskell.org>**20081004223518] 
[Build with both base-3 and 4
Duncan Coutts <duncan@haskell.org>**20081004221008] 
[Drop unrecognised and unnecessary pragma
Duncan Coutts <duncan@haskell.org>**20081004220034] 
[stupid ghc-prim
Don Stewart <dons@galois.com>**20080825214146] 
[Only a minor version bump
Don Stewart <dons@galois.com>**20080825205906] 
[clarify comments on hGet and EOF
Don Stewart <dons@galois.com>**20080809233325] 
[typo in comment
Don Stewart <dons@galois.com>**20080716181103] 
[notes, and undo -fcpr-off
Don Stewart <dons@galois.com>**20080520001839] 
[clean up flags. building on 6.9
Don Stewart <dons@galois.com>**20080519215317] 
[more aggressive inlining on lazy bytestring readInt. Performance wins for sum-file
Don Stewart <dons@galois.com>**20080519215149] 
[make hackage happy
Don Stewart <dons@galois.com>**20080420211741] 
[Point to HPC coverage data
Don Stewart <dons@galois.com>**20080420210524] 
[extra-source-files
Don Stewart <dons@galois.com>**20080420205553] 
[Add properties for rewrite rules, compiled sans -fhpc
Don Stewart <dons@galois.com>**20080419025720
 
 Following an idea of Andy Gill's, compile just some rewrite rules code
 without -fhpc, link it into a hpc app, and get coverage / rule firing
 data for code run by rewrite rules. Library now 93% covered by QuickCheck.
 
] 
[fix warnings
Don Stewart <dons@galois.com>**20080417232307] 
[Remove old fusion mechanism. Data.ByteString.Fusion is a place holder
Don Stewart <dons@galois.com>**20080417230308
 for streams now.
 
 And while we're at it, reimplement mapAccumL/R and scanl 
 directly. Remove mapIndexed, as its trivial to write with mapAccumL.
 
] 
[more tests
Don Stewart <dons@galois.com>**20080417162926] 
[tune deprecation string
Don Stewart <dons@galois.com>**20080417162305] 
[some more tests
Don Stewart <dons@galois.com>**20080417011226] 
[pointless filter rule
Don Stewart <dons@galois.com>**20080417011025] 
[more QuickCheck properties, 88% covered now
Don Stewart <dons@galois.com>**20080417010205] 
[Use -fglasgow-exts to re-enable rewrite rules
Don Stewart <dons@galois.com>**20080417010144] 
[Remove obscure filter (==) rewrite rule
Don Stewart <dons@galois.com>**20080417010057] 
[add a cunning test for packMallocCString
Don Stewart <dons@galois.com>**20080416171625] 
[typos in comment
Don Stewart <dons@galois.com>**20080416161143] 
[more testing
Don Stewart <dons@galois.com>**20080414065202] 
[normalise rule names
Don Stewart <dons@galois.com>**20080414064537] 
[More testing. 80% of bytestring is now tested with QuickCheck
Don Stewart <dons@galois.com>**20080413221600] 
[tweaks
Don Stewart <dons@galois.com>**20080413190445] 
[More coverage
Don Stewart <dons@galois.com>**20080413190419] 
[better clean target
Don Stewart <dons@galois.com>**20080413135043] 
[tweak makefile
Don Stewart <dons@galois.com>**20080413134800] 
[properties for breakSubstring
Don Stewart <dons@galois.com>**20080413134635] 
[Add breakSubstring, split a bytestring on another bytestring. More idiomatic approach than findSubstring
Don Stewart <dons@galois.com>**20080413134332] 
[faster findSubstrings
Don Stewart <dons@galois.com>**20080413103033] 
[more tests
Don Stewart <dons@galois.com>**20080413103014] 
[tweaks
Don Stewart <dons@galois.com>**20080413102931] 
[no longer depend on array
Don Stewart <dons@galois.com>**20080413102910] 
[require Instances
Don Stewart <dons@galois.com>**20080413102859] 
[add cheaper hpc test target
Don Stewart <dons@galois.com>**20080413090044] 
[comments
Don Stewart <dons@galois.com>**20080413090026] 
[use "ByteString" prefix for rules
Don Stewart <dons@galois.com>**20080413090006] 
[return failures
Don Stewart <dons@galois.com>**20080413085955] 
[actually fail if tests don't pass
Don Stewart <dons@galois.com>**20080413084834] 
[more test coverage
Don Stewart <dons@galois.com>**20080412192702] 
[improve performance of findSubstrings on small strings
Don Stewart <dons@galois.com>**20080412192633] 
[Use -fno-ignore-asserts when testing
Don Stewart <dons@galois.com>**20080411160639] 
[coverage for Monoid instance
Don Stewart <dons@galois.com>**20080411154716] 
[reuse internal invariant test
Don Stewart <dons@galois.com>**20080411154700] 
[ghc uses -XUnliftedFFITypes -XMagicHash -XUnboxedTuples -XDeriveDataTypeable
Don Stewart <dons@galois.com>**20080411154632] 
[use appropriate -X flags
Don Stewart <dons@galois.com>**20080411144604] 
[bump version number
Don Stewart <dons@galois.com>**20080411114047] 
[note how to run testsuite with hugs
Don Stewart <dons@galois.com>**20080411114030] 
[document second argument of hGet
Don Stewart <dons@galois.com>**20080406034554] 
[normalise rewrite rule names
Don Stewart <dons@galois.com>**20080321165326] 
[tweaks
Don Stewart <dons@galois.com>**20080321005940] 
[standardise rule names with   "ByteString" prefix
Don Stewart <dons@galois.com>**20080320204043] 
[standardise rules firing with prefix "ByteString"
Don Stewart <dons@galois.com>**20080320170005] 
[dead code
Don Stewart <dons@galois.com>**20080320165557] 
[Document IsString ByteString, and how to enable it
Don Stewart <dons@galois.com>**20080320165445] 
[remove deprecated 'join' -- people should have transitioned to intercalate in the past 12 months
Don Stewart <dons@galois.com>**20080320165136] 
[more tests
Don Stewart <dons@galois.com>**20080318171700] 
[Add rewrite rule for Char8, matching the Word8 one, specialising break (==x)
Don Stewart <dons@galois.com>**20080318171256] 
[erroneos inline [1] on Char8.dropWhile
Don Stewart <dons@galois.com>**20080318170525] 
[portably use unsafeDupablePerformIO
Don Stewart <dons@galois.com>**20080317014444] 
[respect warnings
Don Stewart <dons@galois.com>**20080317012323] 
[clean up some things
Don Stewart <dons@galois.com>**20080317011200] 
[Use unsafeDupablePerformIO as a cheaper unsafePerformIO, fixes stack overflow issues in stack squeezing cases
Don Stewart <dons@galois.com>**20080314002323] 
[Documentations typos (filterByte instead of filterChar).
nicolas.pouillard@gmail.com**20080311011358] 
[bump version
Don Stewart <dons@galois.com>**20080222173300] 
[some tests on Ord speed
Don Stewart <dons@galois.com>**20080221200459] 
[Remove pessimistic INLINE on compareBytes. 5x speedup for some Ord-heavy tests
Don Stewart <dons@galois.com>**20080221200424] 
[typo in doc. spotted by Johan Tibell
Don Stewart <dons@galois.com>**20080112222204] 
[todo, find the memcmp length threshodl
Don Stewart <dons@galois.com>**20080109185141] 
[bump version
Don Stewart <dons@galois.com>**20080105050338] 
[fix laziness issue with Lazy.lines
Don Stewart <dons@galois.com>**20080105050319] 
[instance IsString for strict and lazy bytestrings. Use -XOverloadedStrings
Don Stewart <dons@galois.com>**20080105045950] 
[note that Lazy.lines is too strict, and sketch lazy imlementation
Don Stewart <dons@galois.com>**20080104000246] 
[bump version
Don Stewart <dons@galois.com>**20071220073828] 
[TAG 0.9.0.3
Don Stewart <dons@galois.com>**20071220073602] 
Patch bundle hash:
9722266b6ca5842f395d3011b226eca6d5741e08