Stability  experimental 

Maintainer  Daniël de Kok <me@danieldk.eu> 
Safe Haskell  None 
This module provides functionality to perform approximate randomization tests (Noreen, 1989).
 data TestOptions = TestOptions {}
 data TestResult = TestResult {}
 data Significance
 type RandWithError a = ErrorT String Rand a
 approxRandTest :: TestOptions > Sample > Sample > Rand TestResult
 approxRandStats :: TestStatistic > Int > Sample > Sample > Rand Sample
 approxRandPairTest :: TestOptions > Sample > Sample > RandWithError TestResult
 approxRandPairStats :: TestStatistic > Int > Sample > Sample > RandWithError Sample
 type TestStatistic = Sample > Sample > Double
 differenceMean :: TestStatistic
 meanDifference :: TestStatistic
 varianceRatio :: TestStatistic
Description
Approximate randomization tests rely on a simple premise: given a test statistic, if the nullhypothesis (the samples do not differ) is true, we can randomly swap values between samples without an (extreme) impact on the test statistic. Otherwise, the nullhypothesis must be rejected.
The test works by generating a given number of sample shuffles and computing the test statistic for each shuffle. If r is the number of shuffled samples where the test statistic is at least as high as the test statistic applied on the original samples; and N the number of shuffles, then the nullhypothesis is rejected iff (r + 1):(N + 1) < pvalue (for onesided tests).
Two kinds of test are supported:
 Paired sample (
approxRandPairTest
): values from samples are shuffled pairwise. This requires the samples to have an equal length.  Unpaired sample (
approxRandTest
): values from samples are shuffled among both samples. Consequently the ith element of one sample does not bear a relationship with the ith element of the other sample. The shuffled samples retain the sizes of the original samples.
Both tests can be performed as a onetailed or twotailed test.
Examples
Both unpaired and paired sample tests use the Rand
monad to obtain
random numbers. We can obtain a pseudorandom number generator that
is seeded using the system clock using the
newPureMT
function (please refer to
the documentation of Pure64
for more
information):
prng < newPureMT
Suppose that we have the samples s1
and s2
. We could now perform
a TwoTailed randomization test with 10,000 shuffles and the mean
difference as the test statistic, by running approxRandTest
in the Rand
monad (at the p = 0.01 level):
evalRandom (approxRandTest TwoTailed meanDifference 10000 0.01 s1 s2) prng
It is also possible to obtain the test statistics of the shuffled samples directly (e.g. to inspect the distribution of test statistics) using the 'approxRandStats'/'approxRandPiarStats' functions:
evalRandom (approxRandStats meanDifference 10000 0.01 s1 s2) prng
Data types
data TestOptions Source
Options for randomization tests
TestOptions  

data TestResult Source
The result of hypothesis testing.
TestResult  

data Significance Source
Significance.
Significant Double  The null hypothesis should be rejected 
NotSignificant Double  Data is compatible with the null hypothesis 
type RandWithError a = ErrorT String Rand aSource
Computations with random numbers that can fail.
Approximate randomization tests
:: TestOptions  Options for the test 
> Sample  First sample 
> Sample  Second sample 
> Rand TestResult  The test result 
Apply an approximate randomization test.
In approximate randomization tests, the values of two samples are shuffled among those samples. A test statistic is calculated for the original samples and the shuffled samples, to detect whether the difference of the samples is extreme or not.
:: TestStatistic  Test statistic 
> Int  Number of shuffled samples to create 
> Sample  First sample 
> Sample  Second sample 
> Rand Sample  The statistics of the shuffles 
Generate a given number of shuffled samples, and calculate the test statistic for each shuffle.
This function does not require the samples to have an equal length.
:: TestOptions  Options for the test 
> Sample  First sample 
> Sample  Second sample 
> RandWithError TestResult  The test result 
Apply a pairwise approximate randomization test.
In pairwise approximate randomization tests the data points at a given index are swapped between samples with a probability of 0.5. Since swapping is pairwise, the samples should have the same length.
:: TestStatistic  Test statistic 
> Int  Number of shuffled samples to create 
> Sample  First sample 
> Sample  Second sample 
> RandWithError Sample  The statistics of the shuffles 
Generate a given number of pairwise shuffled samples, and calculate the test statistic for each shuffle.
Since the data points at a given index are swapped (with a probability of 0.5), the samples should have the same length.
Test statistics
type TestStatistic = Sample > Sample > DoubleSource
A test stastic calculates the difference between two samples. See
meanDifference
and varianceRatio
for examples.
differenceMean :: TestStatisticSource
Calculates the difference mean of two samples (mean(s1  s2)). When the two samples do not have an equal length, the trailing elements of the longer vector are ignored.
meanDifference :: TestStatisticSource
Calculates the mean difference of two samples (mean(s1)  mean(s2)).
varianceRatio :: TestStatisticSource
Calculate the ratio of sample variances (var(s1) : var(s2)).