Data.Conduit.Find

Introduction

• *find-conduit** is essentially a souped version of GNU find for Haskell, using a DSL to provide both ease of us, and extensive flexbility.

In its simplest form, let's compare some uses of find to find-conduit. Bear in mind that the result of the find function is a conduit, so you're expected to either sink it to a list, or operate on the file paths as they are yielded.

Basic comparison with GNU find

A typical find command:

find src -name '*.hs' -type f -print

Would in find-conduit be:

find src (glob *.hs <> regular) $$ mapM_C (liftIO . print)

The glob predicate matches the file basename against the globbing pattern, while the regular predicate matches plain files.

A more complicated example:

find . -size +100M -perm 644 -mtime 1

Now in find-conduit:

let megs = 1024 * 1024
    days = 86400
now <- liftIO getCurrentTime
find "." ( fileSize (> 100*megs)
        <> hasMode 0o644
        <> lastModified (> addUTCTime now (-(1*days)))
         )

Appending predicates like this expressing an and relationship. Use <|> to express or. You can also negate any predicate:

find "." (not_ (hasMode 0o644))

By default, predicates, whether matching or not, will allow recursion into directories. In order to express that matching predicate should disallow recursion, use prune:

find "." (prune (depth (> 2)))

This is the same as using '-maxdepth 2' in find.

find "." (prune (filename_ (== dist)))

This is the same as:

find . \( -name dist -prune \) -o -print

Performance

find-conduit strives to make file-finding a well performing operation. To this end, a composed Predicate will only call stat once per entry being considered; and if you prune a directory, it is not traversed at all.

By default, find calls stat for every file before it applies the predicate, in order to ensure that only one such call is needed. Sometimes, however, you know just from the FilePath that you don't want to consider a certain file, or you want to prune a directory tree.

To support these types of optimized queries, a variant of find is provided called findWithPreFilter. This takes two predicates: one that is applied to only the FilePath, before stat (or lstat) is called; and one that is applied to the full file information after the stat.

Other notes

Predicates form a Category and an Arrow, so you can use Arrow-style composition rather than Monoids if you wish. They also form an Applicative, a Monad and a MonadPlus.

In the Monad, the value bound over is whatever the predicate chooses to return (most Predicates return the same FilePath they examined, however, making the Monad less value). Here's an example Monad: If the find takes longer than 5 minutes, abort. We could have used timeout, but this is for illustration.

start <- liftIO getCurrentTime
find "." $ do
    glob "*.hs"

end <- liftIO getCurrentTime
    if diffUTCTIme end start > 300
        then ignoreAll
        else matchAll

The Predicate Monad is a short-circuiting monad, meaning we stop as soon as it can be determined that the user is not interested in a given file. To access the current file, simply bind the result value from any Predicate. To change the file being matched against,for whatever reason, use consider.

Finding functions

find :: (MonadIO m, MonadResource m) => FilePath -> Predicate m FileEntry -> Producer m FilePath

find' :: (MonadIO m, MonadResource m) => FilePath -> Predicate m FileEntry -> Producer m FileEntry

lfind :: (MonadIO m, MonadResource m) => FilePath -> Predicate m FileEntry -> Producer m FilePath

lfind' :: (MonadIO m, MonadResource m) => FilePath -> Predicate m FileEntry -> Producer m FileEntry

findWithPreFilter :: (MonadIO m, MonadResource m) => FilePath -> Bool -> Predicate m FileInfo -> Predicate m FileEntry -> Producer m FileEntry

readPaths :: (MonadIO m, MonadResource m) => FilePath -> Predicate m FilePath -> Producer m FilePath

stat :: MonadIO m => Looped m FileInfo FileEntry

lstat :: MonadIO m => Looped m FileInfo FileEntry

test :: MonadIO m => Predicate m FileEntry -> FilePath -> m Bool

Test a file path using the same type of Predicate that is accepted by find.

File path predicates

ignoreVcs :: (MonadIO m, HasFileInfo e) => Predicate m e

Return all entries, except for those within version-control metadata directories (and not including the version control directory itself either).

regex :: (Monad m, HasFileInfo e) => Text -> Predicate m e

glob :: (Monad m, HasFileInfo e) => Text -> Predicate m e

Find every entry whose filename part matching the given filename globbing expression. For example: glob *.hs.

filename_ :: (Monad m, HasFileInfo e) => (FilePath -> Bool) -> Predicate m e

filenameS_ :: (Monad m, HasFileInfo e) => (String -> Bool) -> Predicate m e

filepath_ :: (Monad m, HasFileInfo e) => (FilePath -> Bool) -> Predicate m e

filepathS_ :: (Monad m, HasFileInfo e) => (String -> Bool) -> Predicate m e

withPath :: HasFileInfo a => Monad m => (FilePath -> m Bool) -> Predicate m a

entryPath :: HasFileInfo a => a -> FilePath

File entry predicates (uses stat information)

regular :: Monad m => Predicate m FileEntry

hasMode :: Monad m => FileMode -> Predicate m FileEntry

executable :: Monad m => Predicate m FileEntry

depth :: (Monad m, HasFileInfo e) => (Int -> Bool) -> Predicate m e

lastAccessed :: Monad m => (UTCTime -> Bool) -> Predicate m FileEntry

lastModified :: Monad m => (UTCTime -> Bool) -> Predicate m FileEntry

withStatus :: Monad m => (FileStatus -> m Bool) -> Predicate m FileEntry

Predicate combinators

or_ :: MonadIO m => Looped m a b -> Looped m a b -> Looped m a b

and_ :: MonadIO m => Looped m a b -> Looped m a b -> Looped m a b

not_ :: MonadIO m => Predicate m a -> Predicate m a

not_ reverse the meaning of the given predicate, preserving recursion.

prune :: MonadIO m => Predicate m a -> Predicate m a

prune is much like not_, but does not preserve recursion.

matchAll :: Monad m => Predicate m a

matchAll is id in the Predicate Category.

ignoreAll :: Monad m => Looped m a b

ignoreAll rejects every entry, and does not recurse. This is the same as mzero.

consider :: a -> Looped m a b -> Looped m a b

Within a predicate block, consider a different item than what is currently being predicated upon. This makes it possible to write custom logic within the Monad instance for a predicate, such as in this contrived example:

   flip runLooped "bar.hs" $ do
       x <- filename_ (== "foo.hs")
       when (x /= "") $
           consider "baz.hs" $
               filename_ (== "baz.hs")

(=~) :: FilePath -> Text -> Bool

This is a re-export of =~, with the types changed for ease of use with this module. For example, you can simply say:

    filename_ (=~ \.hs$)

Which is the same thing as:

    regex \.hs$

Types and type classes

data FileEntry

Constructors

FileEntry

Fields

entryInfo :: FileInfo

entryStatus :: FileStatus

Instances

Show FileEntry

HasFileInfo FileEntry

type Predicate m a = Looped m a a

class HasFileInfo a where

Methods

getFileInfo :: a -> FileInfo

Instances

HasFileInfo FileEntry