random-access-file: Random file access methods, supporting application-level page cache.

[ bsd3, library, system ] [ Propose Tags ]
Versions [RSS]
Change log ChangeLog.md
Dependencies base (>=4.7 && <5), bytestring, concurrent-extra, containers, directory, lrucaching, stm, unix, unix-bytestring, unix-memory [details]
License BSD-3-Clause
Copyright 2018 Ilya Portnov
Author Ilya V. Portnov
Maintainer portnov84@rambler.ru
Category System
Home page https://github.com/portnov/random-access-file#readme
Bug tracker https://github.com/portnov/random-access-file/issues
Source repo head: git clone https://github.com/portnov/random-access-file
Uploaded by IlyaPortnov at 2019-03-08T13:32:23Z
Downloads 632 total (10 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs available [build log]
Last success reported on 2019-03-08 [all 1 reports]

Readme for random-access-file-

[back to package description]

random-access-file README

This package is aimed to provide some number of different implementations of random file access methods with the same interface.

It can be of use for implementing multithread read-write random access for large files, for example for DB engines.

The following implementations are provided:

  • Simple: trivial wrapper around standard System.IO calls. Created mostly for demonstrative purposes. Specific thing about System.IO calls is that they work with one Handle per file, and that Handle contains a pointer to current position in the file. So it is not possible for several threads to read or write to different positions in the file. This implementation is made thread-safe by adding a global per-file lock: several threads can read and/or write to different positions of the file, but all access will be serialized.
  • Threaded: file access using Posix pread(3), pwrite(3) calls. This implementation is thread-safe. It is using block-level locks; each block can be accessed for write by single thread, or for read by many threads. Size of blocks being locked is adjustable.
  • MMaped: file access using mmap(2) call. This implementation is thread-safe. It is using block-level locks also.
  • Cached: File access using application-level page cache, which can be used over Threaded or MMaped file access. Size of cache pages and capacity of the cache are adjustable. Performance of this implementation depends very seriously on your usage pattern and page size.

Benchmark results

Benchmark random-access-file-benchmark: RUNNING...
benchmarking main/simple
time                 691.1 ms   (-610.8 ms .. 1.689 s)
                     0.639 R²   (0.044 R² .. 1.000 R²)
mean                 1.781 s    (1.088 s .. 2.106 s)
std dev              543.2 ms   (192.5 ms .. 732.0 ms)
variance introduced by outliers: 73% (severely inflated)

benchmarking main/threaded
time                 228.5 ms   (196.1 ms .. 248.2 ms)
                     0.993 R²   (0.978 R² .. 1.000 R²)
mean                 250.5 ms   (239.5 ms .. 257.1 ms)
std dev              11.31 ms   (3.916 ms .. 16.04 ms)
variance introduced by outliers: 16% (moderately inflated)

benchmarking main/mmaped
time                 166.0 ms   (147.6 ms .. 182.9 ms)
                     0.989 R²   (0.963 R² .. 1.000 R²)
mean                 162.5 ms   (157.6 ms .. 168.0 ms)
std dev              7.960 ms   (4.764 ms .. 11.38 ms)
variance introduced by outliers: 12% (moderately inflated)

benchmarking main/cached/threaded
time                 1.272 s    (575.8 ms .. 1.865 s)
                     0.951 R²   (0.925 R² .. 1.000 R²)
mean                 1.517 s    (1.365 s .. 1.626 s)
std dev              150.9 ms   (64.02 ms .. 207.5 ms)
variance introduced by outliers: 23% (moderately inflated)

benchmarking main/cached/mmaped
time                 199.2 ms   (78.98 ms .. 332.2 ms)
                     0.863 R²   (0.708 R² .. 1.000 R²)
mean                 237.7 ms   (190.4 ms .. 285.1 ms)
std dev              62.14 ms   (31.83 ms .. 85.03 ms)
variance introduced by outliers: 58% (severely inflated)

Benchmark random-access-file-benchmark: FINISH