Data.Repa.Array.Material

Material arrays are represented as concrete data in memory.

For performance reasons, random access indexing into these layouts is not bounds checked. However, all bulk operators like map and concat are guaranteed to be safe.

• A -- Type directed automatic layout.
• F -- Foreign memory buffers.
• N -- Nested arrays.
• B -- Boxed vectors, via Data.Vector.Boxed
• U -- Adaptive unboxed vectors, via Data.Vector.Unboxed

Documentation

type Material l a = (Bulk l a, Windowable l a, Target l a)

Classes supported by all material representations.

We can index them in a random-access manner, window them in constant time, and use them as targets for a computation.

In particular, delayed arrays are not material as we cannot use them as targets for a computation.

Auto arrays

data A

Arrays where the elements that are automatically layed out into some efficient runtime representation.

The implementation uses type families to chose unboxed representations for all elements that can be unboxed. In particular: arrays of unboxed tuples are represented as tuples of unboxed arrays, and nested arrays are represented using a segment descriptor and a single single flat vector containing all the elements.

Constructors

Auto

Fields

autoLength :: Int

Instances

Eq A

Show A

Layout A

Bulk A Char

Bulk A Double

Bulk A Float

Bulk A Int

Bulk A Int8

Bulk A Int16

Bulk A Int32

Bulk A Int64

Bulk A Word8

Bulk A Date32

Windowable A Char

Windowable A Double

Windowable A Float

Windowable A Int

Windowable A Int8

Windowable A Int16

Windowable A Int32

Windowable A Int64

Windowable A Word8

Windowable A Date32

Target A Char

Target A Double

Target A Float

Target A Int

Target A Int8

Target A Int16

Target A Int32

Target A Int64

Target A Word8

Target A Date32

Convert F Char A Char

Convert F Double A Double

Convert F Float A Float

Convert F Int A Int

Convert F Int8 A Int8

Convert F Int16 A Int16

Convert F Int32 A Int32

Convert F Int64 A Int64

Convert F Word8 A Word8

Convert A Char F Char

Convert A Double F Double

Convert A Float F Float

Convert A Int F Int

Convert A Int8 F Int8

Convert A Int16 F Int16

Convert A Int32 F Int32

Convert A Int64 F Int64

Convert A Word8 F Word8

Bulk A a => Bulk A [a]

Bulk A a => Windowable A [a]

Target A [a]

(Bulk A a, Bulk A b) => Bulk A (a, b)

(Bulk A a, Bulk A b) => Bulk A ((:*:) a b)

(Bulk A a, Windowable r a, (~) * (Index r) Int) => Bulk A (Array r a)

(Windowable A a, Windowable A b) => Windowable A (a, b)

(Windowable A a, Windowable A b) => Windowable A ((:*:) a b)

(Bulk A a, Windowable l a, (~) * (Index l) Int) => Windowable A (Array l a)

(Target A a, Target A b) => Target A (a, b)

(Target A a, Target A b) => Target A ((:*:) a b)

(Bulk l a, Target l a, (~) * (Index l) Int) => Target A (Array l a)

Convert r1 a1 r2 a2 => Convert N (Array r1 a1) A (Array r2 a2)

Convert r1 a1 r2 a2 => Convert A (Array r1 a1) A (Array r2 a2)

Convert r1 a1 r2 a2 => Convert A (Array r1 a1) N (Array r2 a2)

(Convert A a1 A a2, Convert A b1 A b2) => Convert A (a1, b1) (T2 A A) (a2, b2)

Eq (Name A)

Show (Name A)

Eq (Array A Char)

Eq a => Eq (Array A [a])

Eq (Array (T2 A A) (a, b)) => Eq (Array A (a, b))

(Eq (Array A a), Eq (Array A b)) => Eq (Array A ((:*:) a b))

Show (Array A Char)

Show (Array A Double)

Show (Array A Float)

Show (Array A Int)

Show (Array A Int8)

Show (Array A Int16)

Show (Array A Int32)

Show (Array A Int64)

Show a => Show (Array A [a])

Show (Array A Word8)

Show (Array (T2 A A) (a, b)) => Show (Array A (a, b))

(Show (Array A a), Show (Array A b)) => Show (Array A ((:*:) a b))

Show (Array A a) => Show (Array A (Array A a))

Show (Array A Date32)

Unpack (Array F Char) t => Unpack (Array A Char) t

Unpack (Buffer F Char) t => Unpack (Buffer A Char) t

Unpack (Buffer F Double) t => Unpack (Buffer A Double) t

Unpack (Buffer F Float) t => Unpack (Buffer A Float) t

Unpack (Buffer F Int) t => Unpack (Buffer A Int) t

Unpack (Buffer F Int8) t => Unpack (Buffer A Int8) t

Unpack (Buffer F Int16) t => Unpack (Buffer A Int16) t

Unpack (Buffer F Int32) t => Unpack (Buffer A Int32) t

Unpack (Buffer F Int64) t => Unpack (Buffer A Int64) t

Unpack (Buffer F Word8) t => Unpack (Buffer A Word8) t

Unpack (Buffer (T2 A A) (a, b)) t => Unpack (Buffer A (a, b)) t

Unpack (Buffer N (Array l a)) t => Unpack (Buffer A (Array l a)) t

Unpack (Buffer F Date32) t => Unpack (Buffer A Date32) t

Unpack (Buffer A [a]) (Buffer A [a])

(Unpack (Buffer A a) tA, Unpack (Buffer A b) tB) => Unpack (Buffer A ((:*:) a b)) (tA, tB)

(Convert A a1 A a2, Convert A b1 A b2) => Convert (T2 A A) (a1, b1) A (a2, b2)

data Name A = A

type Index A = Int

data Array A Char = AArray_Char !(Array F Char)

data Array A Double = AArray_Double !(Array F Double)

data Array A Float = AArray_Float !(Array F Float)

data Array A Int = AArray_Int !(Array F Int)

data Array A Int8 = AArray_Int8 !(Array F Int8)

data Array A Int16 = AArray_Int16 !(Array F Int16)

data Array A Int32 = AArray_Int32 !(Array F Int32)

data Array A Int64 = AArray_Int64 !(Array F Int64)

data Array A Word8 = AArray_Word8 !(Array F Word8)

data Array A Date32 = AArray_Date32 !(Array F Date32)

data Buffer A Char = ABuffer_Char !(Buffer F Char)

data Buffer A Double = ABuffer_Double !(Buffer F Double)

data Buffer A Float = ABuffer_Float !(Buffer F Float)

data Buffer A Int = ABuffer_Int !(Buffer F Int)

data Buffer A Int8 = ABuffer_Int8 !(Buffer F Int8)

data Buffer A Int16 = ABuffer_Int16 !(Buffer F Int16)

data Buffer A Int32 = ABuffer_Int32 !(Buffer F Int32)

data Buffer A Int64 = ABuffer_Int64 !(Buffer F Int64)

data Buffer A Word8 = ABuffer_Word8 !(Buffer F Word8)

data Buffer A Date32 = ABuffer_Date32 !(Buffer F Date32)

data Array A [a] = AArray_List !(Array B [a])

data Buffer A [a] = ABuffer_List !(Buffer B [a])

data Array A (a, b) = AArray_T2 !(Array (T2 A A) (a, b))

data Array A ((:*:) a b) = AArray_Prod !(Array A a) !(Array A b)

data Array A (Array r a) = AArray_Array !(Array N (Array r a))

data Buffer A (a, b) = ABuffer_T2 !(Buffer (T2 A A) (a, b))

data Buffer A ((:*:) a b) = ABuffer_Prod !(Buffer A a) !(Buffer A b)

data Buffer A (Array l a) = ABuffer_Array !(Buffer N (Array l a))

Foreign arrays

data F

Layout for dense Foreign arrays.

UNSAFE: Indexing into raw material arrays is not bounds checked. You may want to wrap this with a Checked layout as well.

Constructors

Foreign

Fields

foreignLength :: Int

Instances

Eq F

Show F

Layout F

Foreign arrays.

Storable a => Bulk F a

Foreign arrays.

Storable a => Windowable F a

Windowing Foreign arrays.

Storable a => Target F a

Foreign buffers

Convert F Char A Char

Convert F Double A Double

Convert F Float A Float

Convert F Int A Int

Convert F Int8 A Int8

Convert F Int16 A Int16

Convert F Int32 A Int32

Convert F Int64 A Int64

Convert F Word8 A Word8

Convert A Char F Char

Convert A Double F Double

Convert A Float F Float

Convert A Int F Int

Convert A Int8 F Int8

Convert A Int16 F Int16

Convert A Int32 F Int32

Convert A Int64 F Int64

Convert A Word8 F Word8

Eq (Name F)

Show (Name F)

(Eq a, Storable a) => Eq (Array F a)

(Storable a, Show a) => Show (Array F a)

Unpack (Array F a) (Vector a)

Unpack (Buffer F a) (IOVector a)

Unpack Foreign buffers

data Name F = F

type Index F = Int

data Array F = FArray !(Vector a)

data Buffer F = FBuffer !(IOVector a)

fromForeignPtr :: Storable a => Int -> ForeignPtr a -> Array F a

O(1). Wrap a ForeignPtr as an array.

toForeignPtr :: Storable a => Array F a -> (Int, Int, ForeignPtr a)

O(1). Unwrap a ForeignPtr from an array.

fromByteString :: ByteString -> Array F Word8

O(1). Convert a ByteString to an foreign Array.

toByteString :: Array F Word8 -> ByteString

O(1). Convert a foreign Vector to a ByteString.

fromStorableVector :: Vector a -> Array F a

O(1). Convert a storable Vector to a foreign Array

toStorableVector :: Array F a -> Vector a

O(1). Convert a foreign array to a storable Vector.

Nested arrays

data N

Nested array represented as a flat array of elements, and a segment descriptor that describes how the elements are partitioned into the sub-arrays. Using this representation for multidimentional arrays is significantly more efficient than using a boxed array of arrays, as there is no need to allocate the sub-arrays individually in the heap.

With a nested type like: Array N (Array N (Array U Int)), the concrete representation consists of five flat unboxed vectors: two for each of the segment descriptors associated with each level of nesting, and one unboxed vector to hold all the integer elements.

UNSAFE: Indexing into raw material arrays is not bounds checked. You may want to wrap this with a Checked layout as well.

Constructors

Nested

Fields

nestedLength :: !Int

Instances

Eq N

Show N

Layout N

Nested arrays.

(BulkI l a, Windowable l a) => Bulk N (Array l a)

Nested arrays.

(BulkI l a, Windowable l a) => Windowable N (Array l a)

Windowing Nested arrays.

(Bulk l a, Target l a, (~) * (Index l) Int) => Target N (Array l a)

Convert r1 a1 r2 a2 => Convert N (Array r1 a1) A (Array r2 a2)

Convert r1 a1 r2 a2 => Convert A (Array r1 a1) N (Array r2 a2)

Eq (Name N)

Show (Name N)

Show (Array l a) => Show (Array N (Array l a))

Unpack (Buffer N (Array l a)) (IOVector (Array l a))

data Name N = N

type Index N = Int

data Array N (Array l a) = NArray {

• nArrayStarts :: !(Vector Int)
• nArrayLengths :: !(Vector Int)
• nArrayElems :: !(Array l a)

}

data Buffer N (Array l a) = NBuffer !(IOVector (Array l a))

fromLists :: TargetI l a => Name l -> [[a]] -> Array N (Array l a)

O(size src) Convert some lists to a nested array.

fromListss :: TargetI l a => Name l -> [[[a]]] -> Array N (Array N (Array l a))

O(size src) Convert a triply nested list to a triply nested array.

Boxed arrays

data B

Layout an array as flat vector of boxed elements.

UNSAFE: Indexing into raw material arrays is not bounds checked. You may want to wrap this with a Checked layout as well.

Constructors

Boxed

Fields

boxedLength :: !Int

Instances

Eq B

Show B

Layout B

Boxed arrays.

Bulk B a

Boxed arrays.

Windowable B a

Boxed windows.

Target B a

Boxed buffers.

Eq (Name B)

Show (Name B)

Eq a => Eq (Array B a)

Show a => Show (Array B a)

Unpack (Buffer B a) (IOVector a)

data Name B = B

type Index B = Int

data Array B = BArray !(Vector a)

data Buffer B = BBuffer !(IOVector a)

fromBoxed :: Vector a -> Array B a

O(1). Wrap a boxed vector as an array.

toBoxed :: Array B a -> Vector a

O(1). Unwrap a boxed vector from an array.

Unboxed arrays

data U

Layout an array as a flat vector of unboxed elements.

This is the most efficient representation for numerical data.

The implementation uses Data.Vector.Unboxed which picks an efficient, specialised representation for every element type. In particular, unboxed vectors of pairs are represented as pairs of unboxed vectors.

UNSAFE: Indexing into raw material arrays is not bounds checked. You may want to wrap this with a Checked layout as well.

Constructors

Unboxed

Fields

unboxedLength :: !Int

Instances

Eq U

Show U

Layout U

Unboxed arrays.

Unbox a => Bulk U a

Unboxed arrays.

Unbox a => Windowable U a

Windowing Unboxed arrays.

Unbox a => Target U a

Unboxed buffers.

Eq (Name U)

Show (Name U)

(Unbox a, Eq a) => Eq (Array U a)

(Show a, Unbox a) => Show (Array U a)

Unpack (Array U a) (Vector a)

Unpack (Buffer U a) (IOVector a)

data Name U = U

type Index U = Int

data Array U = UArray !(Vector a)

data Buffer U = UBuffer !(IOVector a)

class (Vector Vector a, MVector MVector a) => Unbox a

Instances

Unbox Bool

Unbox Char

Unbox Double

Unbox Float

Unbox Int

Unbox Int8

Unbox Int16

Unbox Int32

Unbox Int64

Unbox Word

Unbox Word8

Unbox Word16

Unbox Word32

Unbox Word64

Unbox ()

(RealFloat a, Unbox a) => Unbox (Complex a)

(Unbox a, Unbox b) => Unbox (a, b)

(Unbox a, Unbox b) => Unbox ((:*:) a b)

(Unbox a, Unbox b, Unbox c) => Unbox (a, b, c)

(Unbox a, Unbox b, Unbox c, Unbox d) => Unbox (a, b, c, d)

(Unbox a, Unbox b, Unbox c, Unbox d, Unbox e) => Unbox (a, b, c, d, e)

(Unbox a, Unbox b, Unbox c, Unbox d, Unbox e, Unbox f) => Unbox (a, b, c, d, e, f)

fromUnboxed :: Unbox a => Vector a -> Array U a

O(1). Wrap an unboxed vector as an array.

toUnboxed :: Unbox a => Array U a -> Vector a

O(1). Unwrap an unboxed vector from an array.

Material operators

These operators work on particular material representations, rather than being generic like the ones in Data.Repa.Array.Generic

Mapping

mapElems :: (Array l1 a -> Array l2 b) -> Array N (Array l1 a) -> Array N (Array l2 b)

Apply a function to all the elements of a doubly nested array, preserving the nesting structure.

Filtering

decimate :: (a -> a -> Bool) -> Array B a -> Array B a

Scan through an array from front to back. For pairs of successive elements, drop the second one when the given predicate returns true.

This function can be used to remove duplicates from a sorted array.

TODO: generalise to other array types.

Slicing

slices

Arguments

:: Array F Int

Segment starting positions.

-> Array F Int

Segment lengths.

-> Array l a

Array elements.

-> Array N (Array l a)

O(1). Produce a nested array by taking slices from some array of elements.

This is a constant time operation, as the representation for nested vectors just wraps the starts, lengths and elements vectors.

Partitioning

partition

Arguments

:: (BulkI lSrc (Int, a), Target lDst a, Index lDst ~ Int, Elt a)

=> Name lDst

Name of destination layout.

-> Int

Total number of segments.

-> Array lSrc (Int, a)

Segment numbers and values.

-> Array N (Array lDst a)

Result array

Take a desired number of segments, and array of key value pairs where the key is the segment number. Partition the values into the stated number of segments, discarding values where the key falls outside the given range.

• This function operates by first allocating a buffer of size (segs * len src) and filling it with a default value. Both the worst case runtime and memory use will be poor for a large number of destination segments.

TODO: we need the pre-init because otherwise unused values in the elems array are undefined. We could avoid this by copying out the used elements after the partition loop finishes. Use a segmented extract function. This would also remove the dependency on the Elt class.

partitionBy

Arguments

:: (BulkI lSrc a, Target lDst a, Index lDst ~ Int, Elt a)

=> Name lDst

Name of destination layout.

-> Int

Total number of Segments.

-> (a -> Int)

Get the segment number for this element.

-> Array lSrc a

Source values.

-> Array N (Array lDst a)

Like partition but use the provided function to compute the segment number for each element.

partitionByIx

Arguments

:: (BulkI lSrc a, Target lDst a, Index lDst ~ Int, Elt a)

=> Name lDst

Name of destination layout.

-> Int

Total number of Segments.

-> (Int -> a -> Int)

Get the segment number for this element.

-> Array lSrc a

Source values.

-> Array N (Array lDst a)

Like partition but use the provided function to compute the segment number for each element. The function is given the index of the each element, along with the element itself.

Concatenation

concats :: Array N (Array N (Array l a)) -> Array N (Array l a)

Segmented concatenation. Concatenate triply nested vector, producing a doubly nested vector.

• Unlike the plain concat function, this operation is performed entirely on the segment descriptors of the nested arrays, and does not require the inner array elements to be copied.

> import Data.Repa.Nice
> nice $ concats $ fromListss U [["red", "green", "blue"], ["grey", "white"], [], ["black"]]
["red","green","blue","grey","white","black"]

Splitting

segment

Arguments

:: (BulkI l a, Unbox a)

=> (a -> Bool)

Detect the start of a segment.

-> (a -> Bool)

Detect the end of a segment.

-> Array l a

Vector to segment.

-> Array N (Array l a)

O(len src). Given predicates which detect the start and end of a segment, split an vector into the indicated segments.

segmentOn

Arguments

:: (BulkI l a, Eq a, Unbox a)

=> (a -> Bool)

Detect the end of a segment.

-> Array l a

Vector to segment.

-> Array N (Array l a)

O(len src). Given a terminating value, split an vector into segments.

The result segments do not include the terminator.

> import Data.Repa.Nice
> nice $ segmentOn (== ' ') (fromList U "fresh   fried fish  ") 
["fresh "," "," ","fried ","fish "," "]

dice

Arguments

:: (BulkI l a, Windowable l a, Unbox a)

=> (a -> Bool)

Detect the start of an inner segment.

-> (a -> Bool)

Detect the end of an inner segment.

-> (a -> Bool)

Detect the start of an outer segment.

-> (a -> Bool)

Detect the end of an outer segment.

-> Array l a

Array to dice.

-> Array N (Array N (Array l a))

O(len src). Like segment, but cut the source array twice.

diceSep

Arguments

:: (BulkI l a, Windowable l a, Unbox a, Eq a)

=> a

Terminating element for inner segments.

-> a

Terminating element for outer segments.

-> Array l a

Vector to dice.

-> Array N (Array N (Array l a))

O(len src). Given field and row terminating values, split an array into rows and fields.

Trimming

trims :: BulkI l a => (a -> Bool) -> Array N (Array l a) -> Array N (Array l a)

For each segment of a nested array, trim elements off the start and end of the segment that match the given predicate.

trimEnds :: BulkI l a => (a -> Bool) -> Array N (Array l a) -> Array N (Array l a)

For each segment of a nested array, trim elements off the end of the segment that match the given predicate.

trimStarts :: BulkI l a => (a -> Bool) -> Array N (Array l a) -> Array N (Array l a)

For each segment of a nested array, trim elements off the start of the segment that match the given predicate.

Transpose

ragspose3 :: Array N (Array N (Array l a)) -> Array N (Array N (Array l a))

Ragged transpose of a triply nested array.

• This operation is performed entirely on the segment descriptors of the nested arrays, and does not require the inner array elements to be copied.