Streamly.Internal.Data.Array

Setup

>>> :m
>>> :set -XFlexibleContexts
>>> :set -XMagicHash
>>> import Data.Function ((&))
>>> import Data.Functor.Identity (Identity(..))
>>> import System.IO.Unsafe (unsafePerformIO)

>>> import Streamly.Data.Array (Array)
>>> import Streamly.Data.Stream (Stream)

>>> import qualified Streamly.Data.Array as Array
>>> import qualified Streamly.Data.Fold as Fold
>>> import qualified Streamly.Data.ParserK as ParserK
>>> import qualified Streamly.Data.Stream as Stream
>>> import qualified Streamly.Data.StreamK as StreamK

For APIs that have not been released yet.

>>> import qualified Streamly.Internal.Data.Array as Array
>>> import qualified Streamly.Internal.Data.Stream as Stream

Design Notes

To summarize:

• Arrays are finite and fixed in size
• provide O(1) access to elements
• store only data and not functions
• provide efficient IO interfacing

Foldable instance is not provided because the implementation would be much less efficient compared to folding via streams. Semigroup and Monoid instances should be used with care; concatenating arrays using binary operations can be highly inefficient. Instead, use toArray to concatenate N arrays at once.

Each array is one pointer visible to the GC. Too many small arrays (e.g. single byte) are only as good as holding those elements in a Haskell list. However, small arrays can be compacted into large ones to reduce the overhead. To hold 32GB memory in 32k sized buffers we need 1 million arrays if we use one array for each chunk. This is still significant to add pressure to GC.

The Array Type

Type

We can use an Unbox constraint in the Array type and the constraint can be automatically provided to a function that pattern matches on the Array type. However, it has huge performance cost, so we do not use it. Investigate a GHC improvement possiblity.

data Array a

Constructors

Array

Fields

• arrContents :: !MutByteArray
• arrStart :: !Int
• arrEnd :: !Int

Instances

a ~ Char => IsString (Array a)

Defined in Streamly.Internal.Data.Array.Type

Methods

fromString :: String -> Array a

Unbox a => Monoid (Array a)

Defined in Streamly.Internal.Data.Array.Type

Methods

mempty :: Array a

mappend :: Array a -> Array a -> Array a

mconcat :: [Array a] -> Array a

Unbox a => Semigroup (Array a)

This should not be used for combining many or N arrays as it would copy the two arrays everytime to a new array. For coalescing multiple arrays use fromChunksK instead.

Defined in Streamly.Internal.Data.Array.Type

Methods

(<>) :: Array a -> Array a -> Array a

sconcat :: NonEmpty (Array a) -> Array a

stimes :: Integral b => b -> Array a -> Array a

Unbox a => IsList (Array a)

Defined in Streamly.Internal.Data.Array.Type

Associated Types

type Item (Array a)

Methods

fromList :: [Item (Array a)] -> Array a

fromListN :: Int -> [Item (Array a)] -> Array a

toList :: Array a -> [Item (Array a)]

(Unbox a, Read a, Show a) => Read (Array a)

Defined in Streamly.Internal.Data.Array.Type

Methods

readsPrec :: Int -> ReadS (Array a)

readList :: ReadS [Array a]

readPrec :: ReadPrec (Array a)

readListPrec :: ReadPrec [Array a]

(Show a, Unbox a) => Show (Array a)

Defined in Streamly.Internal.Data.Array.Type

Methods

showsPrec :: Int -> Array a -> ShowS

show :: Array a -> String

showList :: [Array a] -> ShowS

Eq (Array Int16)

Defined in Streamly.Internal.Data.Array.Type

Methods

(==) :: Array Int16 -> Array Int16 -> Bool

(/=) :: Array Int16 -> Array Int16 -> Bool

Eq (Array Int32)

Defined in Streamly.Internal.Data.Array.Type

Methods

(==) :: Array Int32 -> Array Int32 -> Bool

(/=) :: Array Int32 -> Array Int32 -> Bool

Eq (Array Int64)

Defined in Streamly.Internal.Data.Array.Type

Methods

(==) :: Array Int64 -> Array Int64 -> Bool

(/=) :: Array Int64 -> Array Int64 -> Bool

Eq (Array Int8)

Defined in Streamly.Internal.Data.Array.Type

Methods

(==) :: Array Int8 -> Array Int8 -> Bool

(/=) :: Array Int8 -> Array Int8 -> Bool

Eq (Array Word16)

Defined in Streamly.Internal.Data.Array.Type

Methods

(==) :: Array Word16 -> Array Word16 -> Bool

(/=) :: Array Word16 -> Array Word16 -> Bool

Eq (Array Word32)

Defined in Streamly.Internal.Data.Array.Type

Methods

(==) :: Array Word32 -> Array Word32 -> Bool

(/=) :: Array Word32 -> Array Word32 -> Bool

Eq (Array Word64)

Defined in Streamly.Internal.Data.Array.Type

Methods

(==) :: Array Word64 -> Array Word64 -> Bool

(/=) :: Array Word64 -> Array Word64 -> Bool

Eq (Array Word8)

Defined in Streamly.Internal.Data.Array.Type

Methods

(==) :: Array Word8 -> Array Word8 -> Bool

(/=) :: Array Word8 -> Array Word8 -> Bool

Eq (Array Char)

Defined in Streamly.Internal.Data.Array.Type

Methods

(==) :: Array Char -> Array Char -> Bool

(/=) :: Array Char -> Array Char -> Bool

Eq (Array Int)

Defined in Streamly.Internal.Data.Array.Type

Methods

(==) :: Array Int -> Array Int -> Bool

(/=) :: Array Int -> Array Int -> Bool

(Unbox a, Eq a) => Eq (Array a)

If the type allows a byte-by-byte comparison this instance can be overlapped by a more specific instance that uses byteCmp. Byte comparison can be significantly faster.

Defined in Streamly.Internal.Data.Array.Type

Methods

(==) :: Array a -> Array a -> Bool

(/=) :: Array a -> Array a -> Bool

(Unbox a, Ord a) => Ord (Array a)

Defined in Streamly.Internal.Data.Array.Type

Methods

compare :: Array a -> Array a -> Ordering

(<) :: Array a -> Array a -> Bool

(<=) :: Array a -> Array a -> Bool

(>) :: Array a -> Array a -> Bool

(>=) :: Array a -> Array a -> Bool

max :: Array a -> Array a -> Array a

min :: Array a -> Array a -> Array a

Serialize (Array a)

Defined in Streamly.Internal.Data.Serialize.Type

Methods

addSizeTo :: Int -> Array a -> Int

deserializeAt :: Int -> MutByteArray -> Int -> IO (Int, Array a)

serializeAt :: Int -> MutByteArray -> Array a -> IO Int

type Item (Array a)

Defined in Streamly.Internal.Data.Array.Type

type Item (Array a) = a

Conversion

Mutable and Immutable

unsafeFreeze :: MutArray a -> Array a

Makes an immutable array using the underlying memory of the mutable array.

Please make sure that there are no other references to the mutable array lying around, so that it is never used after freezing it using unsafeFreeze. If the underlying array is mutated, the immutable promise is lost.

Pre-release

unsafeFreezeWithShrink :: Unbox a => MutArray a -> Array a

Similar to unsafeFreeze but uses rightSize on the mutable array first.

unsafeThaw :: Array a -> MutArray a

Makes a mutable array using the underlying memory of the immutable array.

Please make sure that there are no other references to the immutable array lying around, so that it is never used after thawing it using unsafeThaw. If the resulting array is mutated, any references to the older immutable array are mutated as well.

Pre-release

Pinned and Unpinned

pin :: Array a -> IO (Array a)

Return a copy of the Array in pinned memory if unpinned, else return the original array.

unpin :: Array a -> IO (Array a)

Return a copy of the Array in unpinned memory if pinned, else return the original array.

isPinned :: Array a -> Bool

Return True if the array is allocated in pinned memory.

Casting

unsafePinnedAsPtr :: MonadIO m => Array a -> (Ptr a -> m b) -> m b

Use an Array a as Ptr a.

See unsafePinnedAsPtr in the Mutable array module for more details.

Unsafe

Pre-release

Construction

empty :: Array a

Cloning

clone :: MonadIO m => Array a -> m (Array a)

pinnedClone :: MonadIO m => Array a -> m (Array a)

Slicing

Get a subarray without copying

splitAt :: Unbox a => Int -> Array a -> (Array a, Array a)

Create two slices of an array without copying the original array. The specified index i is the first index of the second slice.

breakOn :: MonadIO m => Word8 -> Array Word8 -> m (Array Word8, Maybe (Array Word8))

Stream Folds

unsafeMakePure :: Monad m => Fold IO a b -> Fold m a b

Fold "step" has a dependency on "initial", and each step is dependent on the previous invocation of step due to state passing, finally extract depends on the result of step, therefore, as long as the fold is driven in the correct order the operations would be correctly ordered. We need to ensure that we strictly evaluate the previous step completely before the next step.

To not share the same array we need to make sure that the result of "initial" is not shared. Existential type ensures that it does not get shared across different folds. However, if we invoke "initial" multiple times for the same fold, there is a possiblity of sharing the two because the compiler would consider it as a pure value. One such example is the chunksOf combinator, or using an array creation fold with foldMany combinator. Is there a proper way in GHC to tell it to not share a pure expression in a particular case?

For this reason array creation folds have a MonadIO constraint. Pure folds could be unsafe and dangerous. This is dangerous especially when used with foldMany like operations.

>>> unsafePureWrite = Array.unsafeMakePure Array.write

createOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)

createOf n folds a maximum of n elements from the input stream to an Array.

pinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)

Like createOf but creates a pinned array.

unsafeCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)

Like createOf but does not check the array bounds when writing. The fold driver must not call the step function more than n times otherwise it will corrupt the memory and crash. This function exists mainly because any conditional in the step function blocks fusion causing 10x performance slowdown.

unsafePinnedCreateOf :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)

create :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)

Fold the whole input to a single array.

Caution! Do not use this on infinite streams.

pinnedCreate :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)

Like create but creates a pinned array.

createWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)

From containers

fromListN :: Unbox a => Int -> [a] -> Array a

Create an Array from the first N elements of a list. The array is allocated to size N, if the list terminates before N elements then the array may hold less than N elements.

pinnedFromListN :: Unbox a => Int -> [a] -> Array a

Like fromListN but creates a pinned array.

fromList :: Unbox a => [a] -> Array a

Create an Array from a list. The list must be of finite size.

pinnedFromList :: Unbox a => [a] -> Array a

Like fromList but creates a pinned array.

fromListRevN :: Unbox a => Int -> [a] -> Array a

Create an Array from the first N elements of a list in reverse order. The array is allocated to size N, if the list terminates before N elements then the array may hold less than N elements.

Pre-release

fromListRev :: Unbox a => [a] -> Array a

Create an Array from a list in reverse order. The list must be of finite size.

Pre-release

fromStreamN :: (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)

Create an Array from the first N elements of a stream. The array is allocated to size N, if the stream terminates before N elements then the array may hold less than N elements.

>>> fromStreamN n = Stream.fold (Array.writeN n)

Pre-release

fromStream :: (MonadIO m, Unbox a) => Stream m a -> m (Array a)

Create an Array from a stream. This is useful when we want to create a single array from a stream of unknown size. writeN is at least twice as efficient when the size is already known.

>>> fromStream = Stream.fold Array.write

Note that if the input stream is too large memory allocation for the array may fail. When the stream size is not known, chunksOf followed by processing of indvidual arrays in the resulting stream should be preferred.

Pre-release

fromPureStreamN :: Unbox a => Int -> Stream Identity a -> Array a

fromPureStream :: Unbox a => Stream Identity a -> Array a

Convert a pure stream in Identity monad to an immutable array.

Same as the following but with better performance:

>>> fromPureStream = Array.fromList . runIdentity . Stream.toList

fromByteStr# :: Addr# -> Array Word8

Copy a null terminated immutable Addr# Word8 sequence into an array.

Unsafe: The caller is responsible for safe addressing.

Note that this is completely safe when reading from Haskell string literals because they are guaranteed to be NULL terminated:

>>> Array.toList $ Array.fromByteStr# "\1\2\3\0"#
[1,2,3]

Note that this should be evaluated strictly to ensure that we do not hold the reference to the pointer in a lazy thunk.

fromByteStr :: Ptr Word8 -> Array Word8

Note that this should be evaluated strictly to ensure that we do not hold the reference to the pointer in a lazy thunk.

fromPtrN :: Int -> Ptr Word8 -> Array Word8

Copy an immutable 'Ptr Word8' sequence into an array.

Unsafe: The caller is responsible for safe addressing.

Note that this should be evaluated strictly to ensure that we do not hold the reference to the pointer in a lazy thunk.

fromChunks :: (MonadIO m, Unbox a) => Stream m (Array a) -> m (Array a)

Given a stream of arrays, splice them all together to generate a single array. The stream must be finite.

fromChunksK :: (MonadIO m, Unbox a) => StreamK m (Array a) -> m (Array a)

Convert an array stream to an array. Note that this requires peak memory that is double the size of the array stream.

Reading

Indexing

unsafeIndexIO :: forall a. Unbox a => Int -> Array a -> IO a

Return element at the specified index without checking the bounds.

Unsafe because it does not check the bounds of the array.

getIndexUnsafe :: forall a. Unbox a => Int -> Array a -> a

Return element at the specified index without checking the bounds.

To Streams

read :: (Monad m, Unbox a) => Array a -> Stream m a

Convert an Array into a stream.

Pre-release

readRev :: (Monad m, Unbox a) => Array a -> Stream m a

Convert an Array into a stream in reverse order.

Pre-release

toStreamK :: forall m a. (Monad m, Unbox a) => Array a -> StreamK m a

toStreamKRev :: forall m a. (Monad m, Unbox a) => Array a -> StreamK m a

To Containers

toList :: Unbox a => Array a -> [a]

Convert an Array into a list.

Unfolds

producer :: forall m a. (Monad m, Unbox a) => Producer m (Array a) a

readerUnsafe :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a

Unfold an array into a stream, does not check the end of the array, the user is responsible for terminating the stream within the array bounds. For high performance application where the end condition can be determined by a terminating fold.

Written in the hope that it may be faster than "read", however, in the case for which this was written, "read" proves to be faster even though the core generated with unsafeRead looks simpler.

Pre-release

reader :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a

Unfold an array into a stream.

readerRev :: forall m a. (Monad m, Unbox a) => Unfold m (Array a) a

Unfold an array into a stream in reverse order.

Size

length :: Unbox a => Array a -> Int

O(1) Get the length of the array i.e. the number of elements in the array.

byteLength :: Array a -> Int

O(1) Get the byte length of the array.

Folding

foldl' :: forall a b. Unbox a => (b -> a -> b) -> b -> Array a -> b

foldr :: Unbox a => (a -> b -> b) -> b -> Array a -> b

byteCmp :: Array a -> Array a -> Ordering

Byte compare two arrays. Compare the length of the arrays. If the length is equal, compare the lexicographical ordering of two underlying byte arrays otherwise return the result of length comparison.

Unsafe: Note that the Unbox instance of sum types with constructors of different sizes may leave some memory uninitialized which can make byte comparison unreliable.

Pre-release

byteEq :: Array a -> Array a -> Bool

Byte equality of two arrays.

>>> byteEq arr1 arr2 = (==) EQ $ Array.byteCmp arr1 arr2

Unsafe: See byteCmp.

Appending

splice :: MonadIO m => Array a -> Array a -> m (Array a)

Copy two immutable arrays into a new array. If you want to splice more than two arrays then this operation would be highly inefficient because it would make a copy on every splice operation, instead use the fromChunksK operation to combine n immutable arrays.

Streams of arrays

Chunk

Group a stream into arrays.

chunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)

chunksOf n stream groups the elements in the input stream into arrays of n elements each.

Same as the following but may be more efficient:

>>> chunksOf n = Stream.foldMany (Array.writeN n)

Pre-release

pinnedChunksOf :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> Stream m (Array a)

Like chunksOf but creates pinned arrays.

buildChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))

Split

Split an array into slices.

Concat

Append the arrays in a stream to form a stream of elements.

concat :: (Monad m, Unbox a) => Stream m (Array a) -> Stream m a

Convert a stream of arrays into a stream of their elements.

>>> concat = Stream.unfoldMany Array.reader

concatRev :: forall m a. (Monad m, Unbox a) => Stream m (Array a) -> Stream m a

Convert a stream of arrays into a stream of their elements reversing the contents of each array before flattening.

>>> concatRev = Stream.unfoldMany Array.readerRev

Compact

Append the arrays in a stream to form a stream of larger arrays.

fCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)

Fold fCompactGE n coalesces adjacent arrays in the input stream until the size becomes greater than or equal to n.

Generates unpinned arrays irrespective of the pinning status of input arrays.

fPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) (Array a)

PInned version of fCompactGE.

lCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()

Like compactGE but for transforming folds instead of stream.

>>> lCompactGE n = Fold.many (Array.fCompactGE n)

Generates unpinned arrays irrespective of the pinning status of input arrays.

lPinnedCompactGE :: (MonadIO m, Unbox a) => Int -> Fold m (Array a) () -> Fold m (Array a) ()

Pinned version of lCompactGE.

compactGE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)

compactGE n stream coalesces adjacent arrays in the stream until the size becomes greater than or equal to n.

>>> compactGE n = Stream.foldMany (Array.fCompactGE n)

Generates unpinned arrays irrespective of the pinning status of input arrays.

Deprecated

asPtrUnsafe :: MonadIO m => Array a -> (Ptr a -> m b) -> m b

Deprecated: Please use unsafePinnedAsPtr instead.

unsafeIndex :: forall a. Unbox a => Int -> Array a -> a

Deprecated: Please use getIndexUnsafe instead

bufferChunks :: (MonadIO m, Unbox a) => Stream m a -> m (StreamK m (Array a))

Deprecated: Please use buildChunks instead.

flattenArrays :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a

Deprecated: Please use "unfoldMany reader" instead.

flattenArraysRev :: forall m a. (MonadIO m, Unbox a) => Stream m (Array a) -> Stream m a

Deprecated: Please use "unfoldMany readerRev" instead.

fromArrayStreamK :: (Unbox a, MonadIO m) => StreamK m (Array a) -> m (Array a)

Deprecated: Please use fromChunksK instead.

fromStreamDN :: forall m a. (MonadIO m, Unbox a) => Int -> Stream m a -> m (Array a)

Deprecated: Please use fromStreamN instead.

fromStreamD :: forall m a. (MonadIO m, Unbox a) => Stream m a -> m (Array a)

Deprecated: Please use fromStream instead.

toStreamD :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a

Deprecated: Please use read instead.

toStreamDRev :: forall m a. (Monad m, Unbox a) => Array a -> Stream m a

Deprecated: Please use readRev instead.

toStream :: (Monad m, Unbox a) => Array a -> Stream m a

Deprecated: Please use read instead.

Same as read

toStreamRev :: (Monad m, Unbox a) => Array a -> Stream m a

Deprecated: Please use readRev instead.

Same as readRev

nil :: Array a

Deprecated: Please use empty instead.

writeWith :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)

Deprecated: Please use createWith instead.

writeN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)

pinnedWriteN :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)

Deprecated: Please use pinnedCreateOf instead.

writeNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)

Deprecated: Please use unsafeCreateOf instead.

pinnedWriteNUnsafe :: forall m a. (MonadIO m, Unbox a) => Int -> Fold m a (Array a)

Deprecated: Please use unsafePinnedCreateOf instead.

pinnedWriteNAligned :: forall m a. (MonadIO m, Unbox a) => Int -> Int -> Fold m a (Array a)

Deprecated: To be removed.

pinnedWriteNAligned alignment n folds a maximum of n elements from the input stream to an Array aligned to the given size.

Pre-release

write :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)

pinnedWrite :: forall m a. (MonadIO m, Unbox a) => Fold m a (Array a)

Deprecated: Please use pinnedCreate instead.

Construction

writeLastN :: (Storable a, Unbox a, MonadIO m) => Int -> Fold m a (Array a)

writeLastN n folds a maximum of n elements from the end of the input stream to an Array.

Random Access

getIndex :: forall a. Unbox a => Int -> Array a -> Maybe a

O(1) Lookup the element at the given index. Index starts from 0.

getIndexRev :: forall a. Unbox a => Int -> Array a -> Maybe a

Like getIndex but indexes the array in reverse from the end.

Pre-release

last :: Unbox a => Array a -> Maybe a

>>> last arr = Array.getIndexRev arr 0

Pre-release

indexReader :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a

Given a stream of array indices, read the elements on those indices from the supplied Array. An exception is thrown if an index is out of bounds.

This is the most general operation. We can implement other operations in terms of this:

read =
     let u = lmap (arr -> (0, length arr - 1)) Unfold.enumerateFromTo
      in Unfold.lmap f (indexReader arr)

readRev =
     let i = length arr - 1
      in Unfold.lmap f (indexReaderFromThenTo i (i - 1) 0)

Pre-release

indexReaderFromThenTo :: Unfold m (Int, Int, Int, Array a) a

Unfolds (from, then, to, array) generating a finite stream whose first element is the array value from the index from and the successive elements are from the indices in increments of then up to to. Index enumeration can occur downwards or upwards depending on whether then comes before or after from.

getIndicesFromThenTo =
    let f (from, next, to, arr) =
            (Stream.enumerateFromThenTo from next to, arr)
     in Unfold.lmap f getIndices

Unimplemented

Size

null :: Array a -> Bool

>>> null arr = Array.byteLength arr == 0

Pre-release

Search

binarySearch :: a -> Array a -> Maybe Int

Given a sorted array, perform a binary search to find the given element. Returns the index of the element if found.

Unimplemented

findIndicesOf :: (a -> Bool) -> Array a -> Stream Identity Int

Unimplemented

indexFinder :: (a -> Bool) -> Unfold Identity (Array a) Int

Perform a linear search to find all the indices where a given element is present in an array.

Unimplemented

Casting

cast :: forall a b. Unbox b => Array a -> Maybe (Array b)

Cast an array having elements of type a into an array having elements of type b. The length of the array should be a multiple of the size of the target element otherwise Nothing is returned.

asBytes :: Array a -> Array Word8

Cast an Array a into an Array Word8.

castUnsafe :: Array a -> Array b

Cast an array having elements of type a into an array having elements of type b. The array size must be a multiple of the size of type b otherwise accessing the last element of the array may result into a crash or a random value.

Pre-release

asCStringUnsafe :: Array a -> (CString -> IO b) -> IO b

Convert an array of any type into a null terminated CString Ptr. If the array is unpinned it is first converted to a pinned array which requires a copy.

Unsafe

O(n) Time: (creates a copy of the array)

Pre-release

Subarrays

getSliceUnsafe

Arguments

:: forall a. Unbox a

=> Int

starting index

-> Int

length of the slice

-> Array a

-> Array a

O(1) Slice an array in constant time.

Caution: The bounds of the slice are not checked.

Unsafe

Pre-release

sliceIndexerFromLen

Arguments

:: forall m a. (Monad m, Unbox a)

=> Int

from index

-> Int

length of the slice

-> Unfold m (Array a) (Int, Int)

slicerFromLen

Arguments

:: forall m a. (Monad m, Unbox a)

=> Int

from index

-> Int

length of the slice

-> Unfold m (Array a) (Array a)

Generate a stream of slices of specified length from an array, starting from the supplied array index. The last slice may be shorter than the requested length.

Pre-release/

splitOn :: (Monad m, Unbox a) => (a -> Bool) -> Array a -> Stream m (Array a)

Split the array into a stream of slices using a predicate. The element matching the predicate is dropped.

Pre-release

Streaming Operations

streamTransform :: forall m a b. (MonadIO m, Unbox a, Unbox b) => (Stream m a -> Stream m b) -> Array a -> m (Array b)

Transform an array into another array using a stream transformation operation.

Pre-release

Folding

streamFold :: (Monad m, Unbox a) => (Stream m a -> m b) -> Array a -> m b

Fold an array using a stream fold operation.

Pre-release

fold :: forall m a b. (Monad m, Unbox a) => Fold m a b -> Array a -> m b

Fold an array using a Fold.

Pre-release

Stream of Arrays

interpose :: (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a

Insert the given element between arrays and flatten.

>>> interpose x = Stream.interpose x Array.reader

interposeSuffix :: forall m a. (Monad m, Unbox a) => a -> Stream m (Array a) -> Stream m a

Insert the given element after each array and flatten. This is similar to unlines.

>>> interposeSuffix x = Stream.interposeSuffix x Array.reader

intercalateSuffix :: (Monad m, Unbox a) => Array a -> Stream m (Array a) -> Stream m a

Insert the given array after each array and flatten.

>>> intercalateSuffix = Stream.intercalateSuffix Array.reader

compactLE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)

compactLE n coalesces adjacent arrays in the input stream only if the combined size would be less than or equal to n.

Generates unpinned arrays irrespective of the pinning status of input arrays.

pinnedCompactLE :: (MonadIO m, Unbox a) => Int -> Stream m (Array a) -> Stream m (Array a)

Pinned version of compactLE.

compactOnByte :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)

Split a stream of arrays on a given separator byte, dropping the separator and coalescing all the arrays between two separators into a single array.

compactOnByteSuffix :: MonadIO m => Word8 -> Stream m (Array Word8) -> Stream m (Array Word8)

Like compactOnByte considers the separator in suffix position instead of infix position.

foldBreakChunks :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m (b, Stream m (Array a))

foldChunks :: (MonadIO m, Unbox a) => Fold m a b -> Stream m (Array a) -> m b

Fold a stream of arrays using a Fold. This is equivalent to the following:

>>> foldChunks f = Stream.fold f . Stream.unfoldMany Array.reader

foldBreakChunksK :: forall m a b. (MonadIO m, Unbox a) => Fold m a b -> StreamK m (Array a) -> m (b, StreamK m (Array a))

Fold a stream of arrays using a Fold and return the remaining stream.

The following alternative to this function allows composing the fold using the parser Monad:

foldBreakStreamK f s =
      fmap (first (fromRight undefined))
    $ StreamK.parseBreakChunks (ParserK.adaptC (Parser.fromFold f)) s

We can compare perf and remove this one or define it in terms of that.

parseBreakChunksK :: forall m a b. (MonadIO m, Unbox a) => Parser a m b -> StreamK m (Array a) -> m (Either ParseError b, StreamK m (Array a))

Parse an array stream using the supplied Parser. Returns the parse result and the unconsumed stream. Throws ParseError if the parse fails.

The following alternative to this function allows composing the parser using the parser Monad:

>>> parseBreakStreamK p = StreamK.parseBreakChunks (ParserK.adaptC p)

We can compare perf and remove this one or define it in terms of that.

Internal

Serialization

encodeAs :: forall a. Serialize a => PinnedState -> a -> Array Word8

serialize :: Serialize a => a -> Array Word8

Properties: 1. Identity: deserialize . serialize == id 2. Encoded equivalence: serialize a == serialize a

pinnedSerialize :: Serialize a => a -> Array Word8

Serialize a Haskell type to a pinned byte array. The array is allocated using pinned memory so that it can be used directly in OS APIs for writing to file or sending over the network.

Properties: 1. Identity: deserialize . pinnedSerialize == id 2. Encoded equivalence: pinnedSerialize a == pinnedSerialize a

deserialize :: Serialize a => Array Word8 -> a

Decode a Haskell type from a byte array containing its serialized representation.

Deprecated

genSlicesFromLen

Arguments

:: forall m a. (Monad m, Unbox a)

=> Int

from index

-> Int

length of the slice

-> Unfold m (Array a) (Int, Int)

Deprecated: Please use sliceIndexerFromLen instead.

getSlicesFromLen

Arguments

:: forall m a. (Monad m, Unbox a)

=> Int

from index

-> Int

length of the slice

-> Unfold m (Array a) (Array a)

Deprecated: Please use slicerFromLen instead.

getIndices :: (Monad m, Unbox a) => Stream m Int -> Unfold m (Array a) a

Deprecated: Please use getIndices instead.