!     2018 Luis Pedro CoelhoMITluis@luispedro.orgNone<NV]conduit-algorithmswrite a Storable vector&This uses the same format as in-memorySee conduit-algorithmsread a Storable vector5This expects the same format as the in-memory vector.This will break up the incoming data into vectors of the given size. The last vector may be smaller if there is not enough data. Any unconsumed Bytes will be leftover for the next conduit in the pipeline.See 2013-2017 Luis Pedro CoelhoMITluis@luispedro.orgNoneN]conduit-algorithms0Act on the next input (do nothing if no input).  awaitJust f is equivalent to h do next <- C.await case next of Just val -> f val Nothing -> return () 'This is a simple utility adapted from Ahttp://neilmitchell.blogspot.de/2015/07/thoughts-on-conduits.htmlconduit-algorithms/Conduit analogue to Python's enumerate functionconduit-algorithms%groupC yields the input as groups of n. elements. If the input is not a multiple of n%, the last element will be incompleteExample: 8 CC.yieldMany [0..10] .| groupC 3 .| CC.consumeList  results in &[ [0,1,2], [3,4,5], [6,7,8], [9, 10] ]!This function is deprecated; use 2013-2018 Luis Pedro CoelhoMITluis@luispedro.orgNone<NV]_aconduit-algorithms This is like E, except that each element is processed in a separate thread (up to  maxThreads can be queued up at any one time). Results are evaluated to normal form (not weak-head normal form!, i.e., the structure is deeply evaluated) to ensure that the computation is fully evaluated in the worker thread.}Note that there is some overhead in threading. It is often a good idea to build larger chunks of input before passing it to ' to amortize the costs. That is, when f" is not a lot of work, instead of  asyncMapC f, it is sometimes better to do = CC.conduitVector 4096 .| asyncMapC (V.map f) .| CC.concat where CC refers to   See conduit-algorithms A version of ( which can reorder results in the streamIf the order of the results is not important, this function can lead to a better use of resources if some of the chunks take longer to complete.See conduit-algorithmsH with error handling. The inner function can now return an error (as a $). When the first error is seen, it  s in the main monad. Note that f may be evaluated for arguments beyond the first error (as some threads may be running in the background and already processing elements after the first error).See  conduit-algorithmsconcatenates input into larger chunks and yields it. Its indended use is to build up larger blocks from smaller ones so that they can be sent across thread barriers with little overhead.the chunkSize parameter is a hint, not an exact element. In particular, larger chunks are not split up and smaller chunks can be yielded too.conduit-algorithms^A simple sink which performs gzip compression in a separate thread and writes the results to h. See also  conduit-algorithmsjCompresses the output and writes to the given file with compression being performed in a separate thread. See also  conduit-algorithmsA source which produces the ungzipped content from the the given handle. Note that this "reads ahead" so if you do not use all the input, the Handle will probably be left at an undefined position in the file. See also  conduit-algorithmsWOpen and read a gzip file with the uncompression being performed in a separate thread. See also  conduit-algorithms_A simple sink which performs bzip2 compression in a separate thread and writes the results to h. See also  conduit-algorithmsjCompresses the output and writes to the given file with compression being performed in a separate thread. See also  conduit-algorithmsA source which produces the bzipped2 content from the the given handle. Note that this "reads ahead" so if you do not use all the input, the Handle will probably be left at an undefined position in the file. See also conduit-algorithmsXOpen and read a bzip2 file with the uncompression being performed in a separate thread. See also conduit-algorithmsaA simple sink which performs lzma/xz compression in a separate thread and writes the results to h. See also conduit-algorithmsjCompresses the output and writes to the given file with compression being performed in a separate thread. See also conduit-algorithmsA source which produces the unxzipped content from the the given handle. Note that this "reads ahead" so if you do not use all the input, the Handle will probably be left at an undefined position in the file. See also conduit-algorithmsZOpen and read a lzma/xz file with the uncompression being performed in a separate thread. See also conduit-algorithms}If the filename indicates a supported compressed file (gzip, xz, and, on Unix, bzip2), then it reads it and uncompresses it.Usage G withPossiblyCompressedFile fname $ src -> src .| mySink Unlike M, this ensures that the file is closed even if the conduit terminates early.=On Windows, attempting to read from a bzip2 file, results in !.conduit-algorithmsoIf the filename indicates a gzipped file (or, on Unix, also a bz2 file), then it reads it and uncompresses it.YTo ensure that the file is closed even if the downstream finishes early, consider using .=On Windows, attempting to read from a bzip2 file, results in !.conduit-algorithmsIf the filename indicates a gzipped file (or, on Unix, also a bz2 file), then it compresses and write with the algorithm matching the filename<On Windows, attempting to write to a bzip2 file, results in !.conduit-algorithms Maximum number of worker threadsconduit-algorithmsFunction to executeconduit-algorithms Maximum number of worker threadsconduit-algorithmsFunction to execute"conduit-algorithms Maximum number of worker threadsconduit-algorithmsFunction to execute conduit-algorithms chunk hint  2018 Luis Pedro CoelhoMITluis@luispedro.orgNone<NV]qgconduit-algorithms#Apply a function to groups of linesNote that this is much more efficient than the (more or less equivalent, except that the intermediate lists can be of varying sizes): N CB.lines .| CC.conduitVector N .| CAlg.asyncMapC nthreads (f . V.toList) The reason being that splitting into lines then becomes the bottleneck and processing a single line is typically a tiny chunk of work so that the threading overhead overwhelms the advantage of using multiple cores. Instead, j will pass big chunks to the worker thread and perform most of the line splitting _in the worker thread_.Only Unix-style ASCII lines are supported (splitting at Bytes with value 10, i.e., \n). When Windows lines (\r\n) are passed to this function, this results in each element having an extra \r at the end.conduit-algorithms#Filter lines using multiple threadsIt is not clear from the types but the input is taken to unbroken lines, while the output will be yielded line by line. This conduit is equivalent to  CB.lines .| CL.filer f 2013-2018 Luis Pedro CoelhoMITluis@luispedro.orgNoneNQV]conduit-algorithmsUnique conduit./For each element, it checks its key (using the a -> b; key function) and yields it if it has not seen it before.Note that this conduit does not5 assume that the input is sorted. Instead it uses a   } to store previously seen elements. Thus, memory usage is O(N) and time is O(N log N). If the input is sorted, you can use conduit-algorithmsUnique conduitSee  and conduit-algorithmsRemoves repeated elements A yieldMany [0, 0, 1, 1, 1, 2, 2, 0] .| removeRepeatsC .| consume is equivalent to  [0, 1, 2, 0]See  and conduit-algorithmsBMerge a list of sorted sources to produce a single (sorted) source3This takes a list of sorted sources and produces a #- which outputs all elements in sorted order.See conduit-algorithms'Take two sorted sources and merge them.See $  !"#$%&'()*+,-./01+23456781conduit-algorithms-0.0.8.2-3sHDOh6OHyWGgJeoRPoKie Data.Conduit.Algorithms.StorableData.Conduit.Algorithms.UtilsData.Conduit.Algorithms.Async(Data.Conduit.Algorithms.Async.ByteStringData.Conduit.AlgorithmsData.Conduit.ListchunksOfmap Data.Conduit CombinatorsDataSetwriteStorableV readStorableV awaitJust enumerateCgroupC asyncMapCunorderedAsyncMapCasyncMapEitherC asyncGzipToasyncGzipToFile asyncGzipFromasyncGzipFromFile asyncBzip2ToasyncBzip2ToFileasyncBzip2FromasyncBzip2FromFile asyncXzTo asyncXzToFile asyncXzFromasyncXzFromFilewithPossiblyCompressedFileconduitPossiblyCompressedFileconduitPossiblyCompressedToFileasyncMapLineGroupsCasyncFilterLinesC uniqueOnCuniqueCremoveRepeatsCmergeCmergeC2base Data.EitherLeft mtl-2.2.2Control.Monad.Error.Class throwError bsConcatToGHC.ErrerrorasyncMapCHelper&conduit-1.3.0.3-8D9iaCe1leW2pfJ2Ava7pxData.Conduit.Internal.ConduitSource