Portability | GHC |
---|---|
Stability | experimental |
Maintainer | bos@serpentine.com, rtomharper@googlemail.com, duncan@haskell.org |
A time and space-efficient implementation of Unicode text using lists of packed arrays. This representation is suitable for high performance use and for streaming large quantities of data. It provides a means to manipulate a large body of text without requiring that the entire content be resident in memory.
Some operations, such as concat
, append
, reverse
and cons
,
have better complexity than their Data.Text equivalents, due to
optimisations resulting from the list spine structure. And for
other operations lazy Text
s are usually within a few percent of
strict ones, but with better heap usage. For data larger than
available memory, or if you have tight memory constraints, this
module will be the only option.
This module is intended to be imported qualified
, to avoid name
clashes with Prelude functions. eg.
import qualified Data.Text.Lazy as B
- data Text
- pack :: String -> Text
- unpack :: Text -> String
- singleton :: Char -> Text
- empty :: Text
- fromChunks :: [Text] -> Text
- toChunks :: Text -> [Text]
- cons :: Char -> Text -> Text
- snoc :: Text -> Char -> Text
- append :: Text -> Text -> Text
- uncons :: Text -> Maybe (Char, Text)
- head :: Text -> Char
- last :: Text -> Char
- tail :: Text -> Text
- init :: Text -> Text
- null :: Text -> Bool
- length :: Text -> Int64
- map :: (Char -> Char) -> Text -> Text
- intercalate :: Text -> [Text] -> Text
- intersperse :: Char -> Text -> Text
- transpose :: [Text] -> [Text]
- reverse :: Text -> Text
- replace :: Text -> Text -> Text -> Text
- toCaseFold :: Text -> Text
- toLower :: Text -> Text
- toUpper :: Text -> Text
- justifyLeft :: Int64 -> Char -> Text -> Text
- justifyRight :: Int64 -> Char -> Text -> Text
- center :: Int64 -> Char -> Text -> Text
- foldl :: (b -> Char -> b) -> b -> Text -> b
- foldl' :: (b -> Char -> b) -> b -> Text -> b
- foldl1 :: (Char -> Char -> Char) -> Text -> Char
- foldl1' :: (Char -> Char -> Char) -> Text -> Char
- foldr :: (Char -> b -> b) -> b -> Text -> b
- foldr1 :: (Char -> Char -> Char) -> Text -> Char
- concat :: [Text] -> Text
- concatMap :: (Char -> Text) -> Text -> Text
- any :: (Char -> Bool) -> Text -> Bool
- all :: (Char -> Bool) -> Text -> Bool
- maximum :: Text -> Char
- minimum :: Text -> Char
- scanl :: (Char -> Char -> Char) -> Char -> Text -> Text
- scanl1 :: (Char -> Char -> Char) -> Text -> Text
- scanr :: (Char -> Char -> Char) -> Char -> Text -> Text
- scanr1 :: (Char -> Char -> Char) -> Text -> Text
- mapAccumL :: (a -> Char -> (a, Char)) -> a -> Text -> (a, Text)
- mapAccumR :: (a -> Char -> (a, Char)) -> a -> Text -> (a, Text)
- replicate :: Int64 -> Text -> Text
- unfoldr :: (a -> Maybe (Char, a)) -> a -> Text
- unfoldrN :: Int64 -> (a -> Maybe (Char, a)) -> a -> Text
- take :: Int64 -> Text -> Text
- drop :: Int64 -> Text -> Text
- takeWhile :: (Char -> Bool) -> Text -> Text
- dropWhile :: (Char -> Bool) -> Text -> Text
- dropWhileEnd :: (Char -> Bool) -> Text -> Text
- dropAround :: (Char -> Bool) -> Text -> Text
- strip :: Text -> Text
- stripStart :: Text -> Text
- stripEnd :: Text -> Text
- splitAt :: Int64 -> Text -> (Text, Text)
- spanBy :: (Char -> Bool) -> Text -> (Text, Text)
- break :: Text -> Text -> (Text, Text)
- breakEnd :: Text -> Text -> (Text, Text)
- breakBy :: (Char -> Bool) -> Text -> (Text, Text)
- group :: Text -> [Text]
- groupBy :: (Char -> Char -> Bool) -> Text -> [Text]
- inits :: Text -> [Text]
- tails :: Text -> [Text]
- split :: Text -> Text -> [Text]
- splitBy :: (Char -> Bool) -> Text -> [Text]
- chunksOf :: Int64 -> Text -> [Text]
- lines :: Text -> [Text]
- words :: Text -> [Text]
- unlines :: [Text] -> Text
- unwords :: [Text] -> Text
- isPrefixOf :: Text -> Text -> Bool
- isSuffixOf :: Text -> Text -> Bool
- isInfixOf :: Text -> Text -> Bool
- filter :: (Char -> Bool) -> Text -> Text
- find :: Text -> Text -> (Text, [(Text, Text)])
- findBy :: (Char -> Bool) -> Text -> Maybe Char
- partitionBy :: (Char -> Bool) -> Text -> (Text, Text)
- index :: Text -> Int64 -> Char
- count :: Text -> Text -> Int64
- zip :: Text -> Text -> [(Char, Char)]
- zipWith :: (Char -> Char -> Char) -> Text -> Text -> Text
Documentation
Creation and elimination
Basic interface
O(1) Returns the first character of a Text
, which must be
non-empty. Subject to array fusion.
O(1) Returns the last character of a Text
, which must be
non-empty. Subject to array fusion.
O(1) Returns all characters after the head of a Text
, which
must be non-empty. Subject to array fusion.
O(1) Returns all but the last character of a Text
, which must
be non-empty. Subject to array fusion.
Transformations
intercalate :: Text -> [Text] -> TextSource
O(n) The intercalate
function takes a Text
and a list of
Text
s and concatenates the list after interspersing the first
argument between each element of the list.
intersperse :: Char -> Text -> TextSource
O(n) The intersperse
function takes a character and places it
between the characters of a Text
. Subject to array fusion.
O(m+n) Replace every occurrence of one substring with another.
Case conversion
With Unicode text, it is incorrect to use combinators like map
toUpper
to case convert each character of a string individually.
Instead, use the whole-string case conversion functions from this
module. For correctness in different writing systems, these
functions may map one input character to two or three output
characters.
toCaseFold :: Text -> TextSource
O(n) Convert a string to folded case. This function is mainly useful for performing caseless (or case insensitive) string comparisons.
A string x
is a caseless match for a string y
if and only if:
toCaseFold x == toCaseFold y
The result string may be longer than the input string, and may
differ from applying toLower
to the input string. For instance,
the Armenian small ligature men now (U+FB13) is case folded to the
bigram men now (U+0574 U+0576), while the micro sign (U+00B5) is
case folded to the Greek small letter letter mu (U+03BC) instead of
itself.
O(n) Convert a string to lower case, using simple case conversion. The result string may be longer than the input string. For instance, the Latin capital letter I with dot above (U+0130) maps to the sequence Latin small letter i (U+0069) followed by combining dot above (U+0307).
O(n) Convert a string to upper case, using simple case conversion. The result string may be longer than the input string. For instance, the German eszett (U+00DF) maps to the two-letter sequence SS.
Justification
justifyLeft :: Int64 -> Char -> Text -> TextSource
O(n) Left-justify a string to the given length, using the specified fill character on the right. Subject to fusion. Examples:
justifyLeft 7 'x' "foo" == "fooxxxx" justifyLeft 3 'x' "foobar" == "foobar"
justifyRight :: Int64 -> Char -> Text -> TextSource
O(n) Right-justify a string to the given length, using the specified fill character on the left. Examples:
justifyRight 7 'x' "bar" == "xxxxbar" justifyRight 3 'x' "foobar" == "foobar"
center :: Int64 -> Char -> Text -> TextSource
O(n) Center a string to the given length, using the specified fill character on either side. Examples:
center 8 'x' "HS" = "xxxHSxxx"
Folds
foldl' :: (b -> Char -> b) -> b -> Text -> bSource
O(n) A strict version of foldl
.
Subject to array fusion.
foldl1' :: (Char -> Char -> Char) -> Text -> CharSource
O(n) A strict version of foldl1
.
Subject to array fusion.
Special folds
Construction
Scans
Accumulating maps
Generation and unfolding
unfoldr :: (a -> Maybe (Char, a)) -> a -> TextSource
O(n), where n
is the length of the result. The unfoldr
function is analogous to the List unfoldr
. unfoldr
builds a
Text
from a seed value. The function takes the element and
returns Nothing
if it is done producing the Text
, otherwise
Just
(a,b)
. In this case, a
is the next Char
in the
string, and b
is the seed value for further production.
unfoldrN :: Int64 -> (a -> Maybe (Char, a)) -> a -> TextSource
O(n) Like unfoldr
, unfoldrN
builds a Text
from a seed
value. However, the length of the result should be limited by the
first argument to unfoldrN
. This function is more efficient than
unfoldr
when the maximum length of the result is known and
correct, otherwise its performance is similar to unfoldr
.
Substrings
Breaking strings
dropWhileEnd :: (Char -> Bool) -> Text -> TextSource
O(n) dropWhileEnd
p
t
returns the prefix remaining after
dropping characters that fail the predicate p
from the end of
t
.
Examples:
dropWhileEnd (=='.') "foo..." == "foo"
dropAround :: (Char -> Bool) -> Text -> TextSource
O(n) dropAround
p
t
returns the substring remaining after
dropping characters that fail the predicate p
from both the
beginning and end of t
. Subject to fusion.
O(n) Remove leading and trailing white space from a string. Equivalent to:
dropAround isSpace
stripStart :: Text -> TextSource
O(n) Remove leading white space from a string. Equivalent to:
dropWhile isSpace
stripEnd :: Text -> TextSource
O(n) Remove trailing white space from a string. Equivalent to:
dropWhileEnd isSpace
spanBy :: (Char -> Bool) -> Text -> (Text, Text)Source
O(n) spanBy
, applied to a predicate p
and text t
, returns
a pair whose first element is the longest prefix (possibly empty)
of t
of elements that satisfy p
, and whose second is the
remainder of the list.
break :: Text -> Text -> (Text, Text)Source
O(n+m) Find the first instance of needle
(which must be
non-null
) in haystack
. The first element of the returned tuple
is the prefix of haystack
before needle
is matched. The second
is the remainder of haystack
, starting with the match.
Examples:
break "::" "a::b::c" ==> ("a", "::b::c") break "/" "foobar" ==> ("foobar", "")
Laws:
append prefix match == haystack where (prefix, match) = break needle haystack
If you need to break a string by a substring repeatedly (e.g. you
want to break on every instance of a substring), use find
instead, as it has lower startup overhead.
This function is strict in its first argument, and lazy in its second.
In (unlikely) bad cases, this function's time complexity degrades towards O(n*m).
breakEnd :: Text -> Text -> (Text, Text)Source
O(n+m) Similar to break
, but searches from the end of the string.
The first element of the returned tuple is the prefix of haystack
up to and including the last match of needle
. The second is the
remainder of haystack
, following the match.
breakEnd "::" "a::b::c" ==> ("a::b::", "c")
The group
function takes a Text
and returns a list of Text
s
such that the concatenation of the result is equal to the argument.
Moreover, each sublist in the result contains only equal elements.
For example,
group "Mississippi" = ["M","i","ss","i","ss","i","pp","i"]
It is a special case of groupBy
, which allows the programmer to
supply their own equality test.
Breaking into many substrings
Splitting functions in this library do not perform character-wise
copies to create substrings; they just construct new Text
s that
are slices of the original.
O(m+n) Break a Text
into pieces separated by the first
Text
argument, consuming the delimiter. An empty delimiter is
invalid, and will cause an error to be raised.
Examples:
split "\r\n" "a\r\nb\r\nd\r\ne" == ["a","b","d","e"] split "aaa" "aaaXaaaXaaaXaaa" == ["","X","X","X",""] split "x" "x" == ["",""]
and
intercalate s . split s == id split (singleton c) == splitBy (==c)
This function is strict in its first argument, and lazy in its second.
In (unlikely) bad cases, this function's time complexity degrades towards O(n*m).
splitBy :: (Char -> Bool) -> Text -> [Text]Source
O(n) Splits a Text
into components delimited by separators,
where the predicate returns True for a separator element. The
resulting components do not contain the separators. Two adjacent
separators result in an empty component in the output. eg.
splitBy (=='a') "aabbaca" == ["","","bb","c",""] splitBy (=='a') [] == [""]
chunksOf :: Int64 -> Text -> [Text]Source
O(n) Splits a Text
into components of length k
. The last
element may be shorter than the other chunks, depending on the
length of the input. Examples:
chunksOf 3 "foobarbaz" == ["foo","bar","baz"] chunksOf 4 "haskell.org" == ["hask","ell.","org"]
Breaking into lines and words
Predicates
isPrefixOf :: Text -> Text -> BoolSource
O(n) The isPrefixOf
function takes two Text
s and returns
True
iff the first is a prefix of the second. This function is
subject to fusion.
isSuffixOf :: Text -> Text -> BoolSource
O(n) The isSuffixOf
function takes two Text
s and returns
True
iff the first is a suffix of the second.
Searching
find :: Text -> Text -> (Text, [(Text, Text)])Source
O(n+m) Find all non-overlapping instances of needle
in
haystack
. The first element of the returned pair is the prefix
of haystack
prior to any matches of needle
. The second is a
list of pairs.
The first element of each pair in the list is a span from the beginning of a match to the beginning of the next match, while the second is a span from the beginning of the match to the end of the input.
Examples:
find "::" "" ==> ("", []) find "/" "a/b/c/d" ==> ("a", [("/b","/b/c/d"), ("/c","/c/d"), ("/d","/d")])
This function is strict in its first argument, and lazy in its second.
In (unlikely) bad cases, this function's time complexity degrades towards O(n*m).
partitionBy :: (Char -> Bool) -> Text -> (Text, Text)Source
O(n) The partitionBy
function takes a predicate and a Text
,
and returns the pair of Text
s with elements which do and do not
satisfy the predicate, respectively; i.e.
partitionBy p t == (filter p t, filter (not . p) t)