Database.Postgres.Temp

This module provides functions for creating a temporary postgres instance. By default it will create a temporary data directory and a temporary directory for a UNIX domain socket for postgres to listen on in addition to listening on 127.0.0.1 and ::1.

Here is an example using the expection safe with function:

with $ \db -> bracket
   (connectPostgreSQL (toConnectionString db))
   close $
   \conn -> execute_ conn "CREATE TABLE foo (id int)"

To extend or override the defaults use withConfig (or startConfig).

tmp-postgres ultimately calls (optionally) initdb, postgres and (optionally) createdb.

All of the command line, environment variables and configuration files that are generated by default for the respective executables can be extended.

In general tmp-postgres is useful if you want a clean temporary postgres and do not want to worry about clashing with an existing postgres instance (or needing to ensure postgres is already running).

Here are some different use cases for tmp-postgres and their respective configurations:

• The default with and start functions can be used to make a sandboxed temporary database for testing.
• By disabling initdb one could run a temporary isolated postgres on a base backup to test a migration.
• By using the stopPostgres and withRestart functions one can test backup strategies.

WARNING!! Ubuntu's PostgreSQL installation does not put initdb on the PATH. We need to add it manually. The necessary binaries are in the /usr/lib/postgresql/VERSION/bin/ directory, and should be added to the PATH

echo "export PATH=$PATH:/usr/lib/postgresql/VERSION/bin/" >> /home/ubuntu/.bashrc

Start and Stop postgres

Exception safe interface

with

Arguments

:: (DB -> IO a)

action continuation.

-> IO (Either StartError a)

Default expectation safe interface. Equivalent to

  with = withConfig defaultConfig

Since: 1.21.0.0

withConfig

Arguments

:: Config

extra. Config combined with the generated Config. See startConfig for more info.

-> (DB -> IO a)

action continuation.

-> IO (Either StartError a)

Exception safe database create with options. See startConfig for more details. Calls stop even in the face of exceptions.

Since: 1.21.0.0

withRestart :: DB -> (DB -> IO a) -> IO (Either StartError a)

Exception safe version of restart.

Since: 1.12.0.0

Configuration

Defaults

defaultConfig :: Config

The default configuration. This will create a database called "postgres" via initdb (it's default behavior). It will create a temporary directory for the data and a temporary directory for a unix socket on a random port. Additionally it will use the following "postgresql.conf" which is optimized for performance.

   shared_buffers = 12MB
   fsync = off
   synchronous_commit = off
   full_page_writes = off
   log_min_messages = PANIC
   log_min_error_statement = PANIC
   log_statement = none
   client_min_messages = ERROR

defaultConfig also passes the --no-sync flag to initdb.

If you would like to customize this behavior you can start with the defaultConfig and overwrite fields or combine a defaultConfig with another Config using <> (mappend).

Alternatively you can eschew defaultConfig altogether, however your postgres might start and run faster if you use defaultConfig.

The defaultConfig redirects all output to /dev/null. See verboseConfig for a version that logs more output.

To append additional lines to "postgresql.conf" file create a custom Config like the following.

 custom = defaultConfig <> mempty
   { plan = mempty
     { postgresConfigFile =
         [ "wal_level = replica"
         , "archive_mode = on"
         , "max_wal_senders = 2"
         , "fsync = on"
         , "synchronous_commit = on"
         ]
     }
   }

Or using the provided lenses and your favorite lens library:

 custom = defaultConfig & planL . postgresConfigFileL <>~
   [ "wal_level = replica"
   , "archive_mode = on"
   , "max_wal_senders = 2"
   , "fsync = on"
   , "synchronous_commit = on"
   ]

This is common enough there is defaultPostgresConf which is a helper to do this.

As an alternative to using defaultConfig one could create a config from connections parameters using optionsToDefaultConfig.

Since: 1.21.0.0

defaultConfig_9_3_10 :: Config

Default configuration for PostgreSQL versions 9.3 and greater but less than 10.

If you get an error that "--no-sync" is an invalid parameter then you should use this config.

Since: 1.21.1.0

defaultPostgresConf :: [(String, String)] -> Config

mappend the defaultConfig with a Config that provides additional "postgresql.conf" lines. Equivalent to:

defaultPostgresConf extra = defaultConfig <> mempty
  { plan = mempty
    { postgresConfigFile = extra
    }
  }

or with lenses:

defaultPostgresConf extra = defaultConfig & planL . postgresConfigFileL <>~ extra

Since: 1.21.0.0

verboseConfig :: Config

The similar to defaultConfig log as much as possible.

Since: 1.21.0.0

Custom Config builder helpers

optionsToDefaultConfig :: Options -> Config

Attempt to create a Config from a Options. Useful if you want to create a database owned by a specific user you will also login with among other use cases.

Since: 1.21.0.0

General Configuration Types

module Database.Postgres.Temp.Config

Main resource handle

data DB

Handle for holding temporary resources, the postgres process handle and postgres connection information. The DB also includes the final plan used to start initdb, createdb and postgres. See toConnectionString or toConnectionOptions for converting a DB to postgresql connection string.

Since: 1.12.0.0

Instances

Pretty DB

Defined in Database.Postgres.Temp.Internal

Methods

pretty :: DB -> Doc

prettyList :: [DB] -> Doc

DB accessors

toConnectionString :: DB -> ByteString

Convert a DB to a connection string. Alternatively one can access the Options using toConnectionOptions.

Since: 1.12.0.0

toConnectionOptions :: DB -> Options

Convert a DB to a connection Options type.

Since: 1.12.0.0

toDataDirectory :: DB -> FilePath

Access the data directory. This was either generated or specified explicitly when creating the Config

Since: 1.12.0.0

toTemporaryDirectory :: DB -> FilePath

Get the directory that is used to create other temporary directories

Since: 1.12.0.0

toPostgresqlConf :: DB -> String

Get the final postgresql.conf

Since: 1.22.0.0

DB modifiers

makeDataDirPermanent :: DB -> DB

Make the data directory permanent. Useful for debugging. If you are using with or withConfig this function will not modify the DB that is passed for cleanup. You will need to setup your own bracket like

   bracket (fmap makeDataDirPermanent start) (either mempty stop)

Since: 1.12.0.0

reloadConfig :: DB -> IO ()

Reload the configuration file without shutting down. Calls pg_reload_conf().

Since: 1.12.0.0

DB debugging

prettyPrintDB :: DB -> String

Display a DB.

Since: 1.12.0.0

Separate start and stop interface.

start :: IO (Either StartError DB)

Default start behavior. Equivalent to calling startConfig with the defaultConfig.

Since: 1.21.0.0

startConfig

Arguments

:: Config

extra configuration that is mappended last to the generated Config. generated <> extra.

-> IO (Either StartError DB)

Create zero or more temporary resources and use them to make a Config.

The passed in config is inspected and a generated config is created. The final config is built by

  generated <> extra

Based on the value of socketDirectory a "postgresql.conf" is created with:

  listen_addresses = '127.0.0.1, ::1'
  unix_socket_directories = 'SOCKET_DIRECTORY'

Additionally the generated Config also does the following:

• Sets a connectionTimeout of one minute.
• Logs internal Events.
• Sets the processes to use the standard input and output handles.
• Sets the dataDirectoryString to file path generated from dataDirectory.

All of these values can be overrided by the extra config.

The returned DB requires cleanup. startConfig should be used with a bracket and stop, e.g.

  withConfig :: Config -> (DB -> IO a) -> IO (Either StartError a)
  withConfig plan f = bracket (startConfig plan) (either mempty stop) $
     either (pure . Left) (fmap Right . f)

or just use withConfig. If you are calling startConfig you probably want withConfig anyway.

Since: 1.15.0.0

stop :: DB -> IO ()

Stop the postgres process and cleanup any temporary resources that might have been created.

Since: 1.12.0.0

restart :: DB -> IO (Either StartError DB)

Restart the postgres from DB using the prior Plan.

Since: 1.12.0.0

stopPostgres :: DB -> IO ExitCode

Only stop the postgres process but leave any temporary resources. Useful for testing backup strategies when used in conjunction with restart or withRestart.

Since: 1.12.0.0

stopPostgresGracefully :: DB -> IO ExitCode

Only stop the postgres process but leave any temporary resources. In contrast to stopPostgres this function makes sure postgres has time to properly write files to the data directory.

Since: 1.16.1.0

Making Starting Faster

initdb Data Directory Caching

Exception safe interface

withDbCache :: (CacheResources -> IO a) -> IO a

Equivalent to withDbCacheConfig with the CacheConfig defaultCacheConfig makes.

Since: 1.20.0.0

withDbCacheConfig

Arguments

:: CacheConfig

Configuration

-> (CacheResources -> IO a)

action for which caching is enabled

-> IO a

Enable initdb data directory caching. This can lead to a 4x speedup.

Exception safe version of setupInitDbCache. Equivalent to

   withDbCacheConfig = bracket (setupInitDbCache config) cleanupInitDbCache

Since: 1.20.0.0

initdb cache configuration.

data CacheConfig

Configuration for the initdb data directory cache.

Since: 1.20.0.0

Constructors

CacheConfig

Fields

• cacheTemporaryDirectory :: FilePath

  Root temporary directory used if cacheDirectoryType is set to Temporary. /tmp is a good default.

• cacheDirectoryType :: DirectoryType

  Used to specify is a Permanent or Temporary directory should be used. defaultCacheConfig uses Permanent ~/.tmp-postgres by default.

• cacheUseCopyOnWrite :: Bool

  Some operatoring system versions support flags for cp that allow "copy on write" which is about 2x faster. defaultCacheConfig attempts to determine if the cp on the path supports copy on write and sets this to True if it does.

defaultCacheConfig :: CacheConfig

defaultCacheConfig attempts to determine if the cp on the path supports "copy on write" flags and if it does, sets cacheUseCopyOnWrite to True.

It sets cacheDirectoryType to Permanent ~/.tmp-postgres and cacheTemporaryDirectory to /tmp (but this is not used when Permanent is set).

Since: 1.19.0.0

initdb cache handle.

data CacheResources

A handle to cache temporary resources and configuration.

Since: 1.20.0.0

Instances

Generic CacheResources

Defined in Database.Postgres.Temp.Internal

Associated Types

type Rep CacheResources :: Type -> Type

Methods

from :: CacheResources -> Rep CacheResources x

to :: Rep CacheResources x -> CacheResources

NFData CacheResources

Defined in Database.Postgres.Temp.Internal

Methods

rnf :: CacheResources -> ()

type Rep CacheResources

Defined in Database.Postgres.Temp.Internal

type Rep CacheResources = D1 (MetaData "CacheResources" "Database.Postgres.Temp.Internal" "tmp-postgres-1.22.0.0-KC5ojUwcbReDdMM4TiGd2b" False) (C1 (MetaCons "CacheResources" PrefixI True) (S1 (MetaSel (Just "cacheResourcesCow") NoSourceUnpackedness NoSourceStrictness DecidedLazy) (Rec0 Bool) :*: S1 (MetaSel (Just "cacheResourcesDirectory") NoSourceUnpackedness NoSourceStrictness DecidedLazy) (Rec0 CompleteDirectoryType)))

cacheResourcesToConfig :: CacheResources -> Config

Helper to make a Config out of caching info.

Since: 1.22.0.0

Separate start and stop interface.

setupInitDbCache :: CacheConfig -> IO CacheResources

Setup the initdb cache folder.

cleanupInitDbCache :: CacheResources -> IO ()

Cleanup the cache directory if it was Temporary.

Since: 1.20.0.0

Data Directory Snapshots

Exception safe interface

withSnapshot :: DirectoryType -> DB -> (Snapshot -> IO a) -> IO (Either StartError a)

Exception safe method for taking a file system level copy of the database cluster.

Snapshots are useful if you would like to start every test from a migrated database and the migration process is more time consuming then copying the additional data.

Since: 1.20.0.0

Snapshot handle

data Snapshot

A type to track a possibly temporary snapshot directory

Since: 1.20.0.0

Instances

Generic Snapshot

Defined in Database.Postgres.Temp.Internal

Associated Types

type Rep Snapshot :: Type -> Type

Methods

from :: Snapshot -> Rep Snapshot x

to :: Rep Snapshot x -> Snapshot

NFData Snapshot

Defined in Database.Postgres.Temp.Internal

Methods

rnf :: Snapshot -> ()

type Rep Snapshot

Defined in Database.Postgres.Temp.Internal

type Rep Snapshot = D1 (MetaData "Snapshot" "Database.Postgres.Temp.Internal" "tmp-postgres-1.22.0.0-KC5ojUwcbReDdMM4TiGd2b" True) (C1 (MetaCons "Snapshot" PrefixI True) (S1 (MetaSel (Just "unSnapshot") NoSourceUnpackedness NoSourceStrictness DecidedLazy) (Rec0 CompleteDirectoryType)))

snapshotConfig :: Snapshot -> Config

Convert a snapshot into a Config that includes a copyConfig for copying the snapshot directory to a temporary directory.

Since: 1.20.0.0

Separate start and stop interface.

takeSnapshot

Arguments

:: DirectoryType

Either a Temporary or preexisting Permanent directory.

-> DB

The handle. The postgres is shutdown and the data directory is copied.

-> IO (Either StartError Snapshot)

Shutdown the database and copy the directory to a folder.

Since: 1.20.0.0

cleanupSnapshot :: Snapshot -> IO ()

Cleanup any temporary resources used for the snapshot.

Since: 1.20.0.0

Errors

data StartError

A list of failures that can occur when starting. This is not and exhaustive list but covers the errors that the system catches for the user.

Since: 1.12.0.0

Constructors

StartPostgresFailed ExitCode

postgres failed before a connection succeeded. Most likely this is due to invalid configuration

InitDbFailed

initdb failed. This can be from invalid configuration or using a non-empty data directory

Fields

• startErrorStdOut :: String
• startErrorStdErr :: String
• startErrorExitCode :: ExitCode

CreateDbFailed

createdb failed. This can be from invalid configuration or the database might already exist.

Fields

• startErrorStdOut :: String
• startErrorStdErr :: String
• startErrorExitCode :: ExitCode

CompletePlanFailed String [String]

The Plan was missing info and a complete CompletePlan could not be created.

CompleteProcessConfigFailed String [String]

The ProcessConfig was missing info and a CompleteProcessConfig could not be created.

ConnectionTimedOut

Timed out waiting for postgres to accept a connection

DeleteDbError SqlError

EmptyDataDirectory

This will happen if a Plan is missing a initDbConfig.

CopyCachedInitDbFailed String ExitCode

This is called if copying a folder cache fails.

FailedToFindDataDirectory String

Failed to find a data directory when trying to get a cached initdb folder.

SnapshotCopyFailed String ExitCode

We tried to copy a data directory to a snapshot folder and it failed

Instances

Eq StartError

Defined in Database.Postgres.Temp.Internal.Core

Methods

(==) :: StartError -> StartError -> Bool

(/=) :: StartError -> StartError -> Bool

Show StartError

Defined in Database.Postgres.Temp.Internal.Core

Methods

showsPrec :: Int -> StartError -> ShowS

show :: StartError -> String

showList :: [StartError] -> ShowS

Exception StartError

Defined in Database.Postgres.Temp.Internal.Core

Methods

toException :: StartError -> SomeException

fromException :: SomeException -> Maybe StartError

displayException :: StartError -> String