Streamly.Internal.FileSystem.FD

This module is a an experimental replacement for Streamly.FileSystem.Handle. The former module provides IO facilities based on the GHC Handle type. The APIs in this module avoid the GHC handle layer and provide more explicit control over buffering.

Read and write data as streams and arrays to and from files.

This module provides read and write APIs based on handles. Before reading or writing, a file must be opened first using openFile. The Handle returned by openFile is then used to access the file. A Handle is backed by an operating system file descriptor. When the Handle is garbage collected the underlying file descriptor is automatically closed. A handle can be explicitly closed using closeFile.

Reading and writing APIs are divided into two categories, sequential streaming APIs and random or seekable access APIs. File IO APIs are quite similar to Streamly.Data.Array.Foreign read write APIs. In that regard, arrays can be considered as in-memory files or files can be considered as on-disk arrays.

import qualified Streamly.Internal.FileSystem.FD as FD

File Handles

data Handle

A Handle is returned by openFile and is subsequently used to perform read and write operations on a file.

stdin :: Handle

File handle for standard input

stdout :: Handle

File handle for standard output

stderr :: Handle

File handle for standard error

openFile :: FilePath -> IOMode -> IO Handle

Open a file that is not a directory and return a file handle. openFile enforces a multiple-reader single-writer locking on files. That is, there may either be many handles on the same file which manage input, or just one handle on the file which manages output. If any open handle is managing a file for output, no new handle can be allocated for that file. If any open handle is managing a file for input, new handles can only be allocated if they do not manage output. Whether two files are the same is implementation-dependent, but they should normally be the same if they have the same absolute path name and neither has been renamed, for example.

Streaming IO

Streaming APIs read or write data to or from a file or device sequentially, they never perform a seek to a random location. When reading, the stream is lazy and generated on-demand as the consumer consumes it. Read IO requests to the IO device are performed in chunks of 32KiB, this is referred to as defaultChunkSize in the documentation. One IO request may or may not read the full chunk. If the whole stream is not consumed, it is possible that we may read slightly more from the IO device than what the consumer needed. Unless specified otherwise in the API, writes are collected into chunks of defaultChunkSize before they are written to the IO device.

Read File to Stream

read :: (IsStream t, MonadIO m) => Handle -> t m Word8

Generate a stream of elements of the given type from a file Handle. The stream ends when EOF is encountered.

Since: 0.7.0

readInChunksOf :: (IsStream t, MonadIO m) => Int -> Handle -> t m Word8

readInChunksOf chunkSize handle reads a byte stream from a file handle, reads are performed in chunks of up to chunkSize. The stream ends as soon as EOF is encountered.

readArrays :: (IsStream t, MonadIO m) => Handle -> t m (Array Word8)

readArrays h reads a stream of arrays from file handle h. The maximum size of a single array is limited to defaultChunkSize. readArrays ignores the prevailing TextEncoding and NewlineMode on the Handle.

readArrays = readArraysOfUpto defaultChunkSize

Since: 0.7.0

readArraysOfUpto :: (IsStream t, MonadIO m) => Int -> Handle -> t m (Array Word8)

Write File from Stream

write :: MonadIO m => Handle -> SerialT m Word8 -> m ()

Write a byte stream to a file handle. Combines the bytes in chunks of size up to defaultChunkSize before writing. Note that the write behavior depends on the IOMode and the current seek position of the handle.

Since: 0.7.0

writeInChunksOf :: MonadIO m => Int -> Handle -> SerialT m Word8 -> m ()

Like write but provides control over the write buffer. Output will be written to the IO device as soon as we collect the specified number of input elements.

Since: 0.7.0

writeArrays :: (MonadIO m, Storable a) => Handle -> SerialT m (Array a) -> m ()

Write a stream of arrays to a handle.

Since: 0.7.0

writeArraysPackedUpto :: (MonadIO m, Storable a) => Int -> Handle -> SerialT m (Array a) -> m ()

Write a stream of arrays to a handle after coalescing them in chunks of specified size. The chunk size is only a maximum and the actual writes could be smaller than that as we do not split the arrays to fit them to the specified size.

Since: 0.7.0