Numeric.Datasets

The datasets package defines two different kinds of datasets:

• small data sets which are directly (or indirectly with `file-embed`) embedded in the package as pure values and do not require network or IO to download the data set.
• other data sets which need to be fetched over the network with getDataset and are cached in a local temporary directory

This module defines the getDataset function for fetching datasets and utilies for defining new data sets. It is only necessary to import this module when using fetched data sets. Embedded data sets can be imported directly.

Using datasets

getDataset :: Dataset a -> IO [a]

Load a dataset, using the system temporary directory as a cache

type Dataset a

Arguments

 = FilePath

Directory for caching downloaded datasets

-> IO [a]

A dataset is defined as a function from the caching directory to the IO action that loads the data

Defining datasets

data Source

Constructors

URL String

csvDatasetPreprocess :: FromRecord a => (ByteString -> ByteString) -> Source -> Dataset a

Define a dataset from a pre-processing function and a source for a CSV file

csvDataset :: FromRecord a => Source -> Dataset a

Define a dataset from a source for a CSV file

jsonDataset :: FromJSON a => Source -> Dataset a

getFileFromSource :: FilePath -> Source -> IO ByteString

Get a ByteString from the specified Source

parseCSV :: FromRecord a => (ByteString -> ByteString) -> ByteString -> [a]

parseJSON :: FromJSON a => ByteString -> [a]

Helper functions for parsing

dashToCamelCase :: String -> String

Turn dashes to CamlCase

parseDashToCamelField :: Read a => Field -> Parser a

Parse a field, first turning dashes to CamlCase

parseReadField :: Read a => Field -> Parser a

parse somethign, based on its read instance

dropLines :: Int -> ByteString -> ByteString

Drop lines from a bytestring

fixAmericanDecimals :: ByteString -> ByteString

Turn US-style decimals starting with a period (e.g. .2) into something Haskell can parse (e.g. 0.2)

fixedWidthToCSV :: ByteString -> ByteString

Convert a Fixed-width format to a CSV