approx-rand-test-0.0.3: Approximate randomization test

Stabilityexperimental
MaintainerDaniël de Kok <me@danieldk.eu>
Safe HaskellSafe-Infered

Statistics.Test.ApproxRand

Contents

Description

This module provides functionality to perform approximate randomization tests (Noreen, 1989).

Synopsis

Description

Approximate randomization tests rely on a simple premise: given a test statistic, if the null-hypothesis (the samples do not differ) is true, we can randomly swap values between samples without an (extreme) impact on the test statistic. Otherwise, the null-hypothesis must be rejected.

The test works by generating a given number of sample shuffles and computing the test statistic for each shuffle. If r is the number of shuffled samples where the test statistic is at least as high as the test statistic applied on the original samples; and N the number of shuffles, then the null-hypothesis is rejected iff (r + 1):(N + 1) < p-value (for one-sided tests).

Two kinds of test are supported:

  • Paired sample (approxRandPairTest): values from samples are shuffled pair-wise. This requires the samples to have an equal length.
  • Unpaired sample (approxRandTest): values from samples are shuffled among both samples. Consequently the i-th element of one sample does not bear a relationship with the i-th element of the other sample. The shuffled samples retain the sizes of the original samples.

Both tests can be performed as a one-tailed or two-tailed test.

Examples

Both unpaired and paired sample tests use the Rand monad to obtain random numbers. We can obtain a pseudo-random number generator that is seeded using the system clock using the newPureMT function (please refer to the documentation of Pure64 for more information):

 prng <- newPureMT

Suppose that we have the samples s1 and s2. We could now perform a Two-Tailed randomization test with 10,000 shuffles and the mean difference as the test statistic, by running approxRandTest in the Rand monad (at the p = 0.01 level):

 evalRandom (approxRandTest TwoSided meanDifference 10000 0.01 s1 s2) prng

It is also possible to obtain the test scores of the shuffled samples directly (e.g. to inspect the distribution of test scores) using the 'approxRandScores'/'approxRandPiarScores' functions:

 evalRandom (approxRandScores meanDifference 10000 0.01 s1 s2) prng

Data types

data TestResult Source

The result of hypothesis testing.

Constructors

Significant Double

The null hypothesis should be rejected

NotSignificant Double

Data is compatible with the null hypothesis

type RandWithError a = ErrorT String Rand aSource

Computations with random numbers that can fail.

Approximate randomization tests

approxRandTestSource

Arguments

:: TestType

Type of test (OneTailed or TwoTailed)

-> TestStatistic

Test statistic

-> Int

Number of shuffled sample to create

-> Double

The p-value at which to test (e.g. 0.05)

-> Sample

First sample

-> Sample

Second sample

-> Rand TestResult

The test result

Apply an approximate randomization test.

In approximate randomization tests, the values of two samples are shuffled among those samples. A test statistic is calculated for the original samples and the shuffled samples, to detect whether the difference of the samples is extreme or not.

approxRandScoresSource

Arguments

:: TestStatistic

Test statistic

-> Int

Number of shuffled samples to create

-> Sample

First sample

-> Sample

Second sample

-> Rand [Double]

The scores of each shuffle

Generate a given number of shuffled samples, and calculate the test score for each shuffle.

This function does not require the samples to have an equal length.

approxRandPairTestSource

Arguments

:: TestType

Type of test (OneTailed or TwoTailed)

-> TestStatistic

Test statistic

-> Int

Number of shuffled samples to create

-> Double

The p-value at which to test (e.g. 0.05)

-> Sample

First sample

-> Sample

Second sample

-> RandWithError TestResult

The test result

Apply a pair-wise approximate randomization test.

In pair-wise approximate randomization tests the scores at a given index are swapped between samples with a probability of 0.5. Since swapping is pairwise, the samples should have the same length.

approxRandPairScoresSource

Arguments

:: TestStatistic

Test statistic

-> Int

Number of shuffled samples to create

-> Sample

First sample

-> Sample

Second sample

-> RandWithError [Double]

The scores of each shuffle

Generate a given number of pairwise shuffled samples, and calculate the test score for each shuffle.

Since the scores at a given index are swapped (with a probability of 0.5), the samples should have the same length.

Test statistics

type TestStatistic = Sample -> Sample -> DoubleSource

A test stastic calculates the difference between two samples. See meanDifference and varianceRatio for examples.

differenceMean :: TestStatisticSource

Calculates the difference mean of two samples (mean(s1 - s2)). When the two samples do not have an equal length, the trailing elements of the longer vector are ignored.

meanDifference :: TestStatisticSource

Calculates the mean difference of two samples (mean(s1) - mean(s2)).

varianceRatio :: TestStatisticSource

Calculate the ratio of sample variances (var(s1) : var(s2)).