exceptionfree-readfile
exceptionfree-readfile provides a readFile that reads files into memory without throwing any exceptions (internally).
The code for exceptionfree-readfile mirrors base's implementation of readFile wherever possible, but avoids the exception-throwing style of base in favor of recursion with an accumulator.
Why?
exceptionfree-readfile was created after the realization that running a binary with +RTS -xc -RTS in prodution logs an error on every successfully read file, due to base's implementation of readFile's throwing of IOErrors internally.
This particular bad idea was spawned by a reddit comment.
How to use
Stack
Here's how to incorporate exceptionfree-readfile into a stack-powered project:
# Dependency packages to be pulled from upstream that are not in the resolver
# (e.g., acme-missiles-0.3)
extra-deps:
- ... other deps ...
- exceptionfree-readfile
2. Add the library to your project's project.yaml/<project>.cabal
build-depends: base >= 4.7 && < 5
, ... lots of other deps ..
, exceptionfree-readfile
, ... more deps ..
3. Use the library in your project
Here's an example of ExceptionFree used in function that reads some SQL out of a file!
module YourModule where
import qualified System.IO.ExceptionFree as ExceptionFree
-- | Read a given file without logging any exceptions
readFileWithoutXCTraces :: FilePath -> IO (Either IOError String)
readFileWithoutXCTraces = ExceptionFree.readFile
Known Issues / Gotchas
There are a few things to keep in mind when evaluating whether you should use exceptionfree-readfile:
exceptionfree-readfile performs only non-blocking reads, and is thus not a good choice for reading pseudo-files like stdin.
exceptionfree-readfile can be ~1-10x slower than regular readFile depending on the size of the file (see benchmark below).
- While
exceptionfree-readfile makes a concerted effort to be at as robust as System.IO.readFile it is a work in progress, and does not have a test suite as extensive as base.
exceptionfree-readfile does worse than original for almost every case (2x-10x) except very small files. The Oxidized version does better, but not as good as the code in base (and requires dynamic linking of the rust library at runtime).
Benchmark Results
You can find the most recent results in bench/most-recent-results.txt, generated by make bench-gen-results. Here is a set of recent results:
exceptionfree-readfile> benchmarks
Running 1 benchmarks...
Benchmark bench: RUNNING...
benchmarking readFile/Original-KB1
time 25.71 μs (24.04 μs .. 27.52 μs)
0.973 R² (0.956 R² .. 0.988 R²)
mean 27.11 μs (26.19 μs .. 28.13 μs)
std dev 3.471 μs (2.658 μs .. 4.366 μs)
variance introduced by outliers: 90% (severely inflated)
benchmarking readFile/ExceptionFree-KB1
time 38.19 μs (37.00 μs .. 39.64 μs)
0.985 R² (0.977 R² .. 0.991 R²)
mean 39.59 μs (38.39 μs .. 41.02 μs)
std dev 4.608 μs (3.948 μs .. 5.511 μs)
variance introduced by outliers: 88% (severely inflated)
benchmarking readFile/Oxidized-KB1
time 21.65 μs (20.49 μs .. 22.96 μs)
0.983 R² (0.978 R² .. 0.991 R²)
mean 21.11 μs (20.39 μs .. 22.00 μs)
std dev 2.924 μs (2.294 μs .. 3.925 μs)
variance introduced by outliers: 91% (severely inflated)
benchmarking readFile/Original-KB10
time 197.0 μs (185.8 μs .. 210.0 μs)
0.977 R² (0.962 R² .. 0.990 R²)
mean 193.0 μs (185.9 μs .. 206.8 μs)
std dev 31.11 μs (22.64 μs .. 46.27 μs)
variance introduced by outliers: 92% (severely inflated)
benchmarking readFile/ExceptionFree-KB10
time 444.1 μs (429.3 μs .. 463.7 μs)
0.969 R² (0.951 R² .. 0.983 R²)
mean 498.1 μs (477.1 μs .. 525.0 μs)
std dev 90.58 μs (74.15 μs .. 115.0 μs)
variance introduced by outliers: 92% (severely inflated)
benchmarking readFile/Oxidized-KB10
time 200.0 μs (192.7 μs .. 208.2 μs)
0.984 R² (0.976 R² .. 0.990 R²)
mean 217.8 μs (206.3 μs .. 235.5 μs)
std dev 45.13 μs (27.80 μs .. 72.23 μs)
variance introduced by outliers: 95% (severely inflated)
benchmarking readFile/Original-KB100
time 1.600 ms (1.448 ms .. 1.746 ms)
0.955 R² (0.940 R² .. 0.980 R²)
mean 1.641 ms (1.547 ms .. 1.807 ms)
std dev 425.7 μs (309.3 μs .. 595.0 μs)
variance introduced by outliers: 95% (severely inflated)
benchmarking readFile/ExceptionFree-KB100
time 16.16 ms (13.78 ms .. 18.39 ms)
0.945 R² (0.919 R² .. 0.996 R²)
mean 14.23 ms (13.74 ms .. 15.54 ms)
std dev 1.898 ms (1.235 ms .. 2.798 ms)
variance introduced by outliers: 63% (severely inflated)
benchmarking readFile/Oxidized-KB100
time 9.387 ms (8.010 ms .. 11.36 ms)
0.866 R² (0.787 R² .. 0.970 R²)
mean 9.796 ms (9.046 ms .. 10.87 ms)
std dev 2.187 ms (1.214 ms .. 3.476 ms)
variance introduced by outliers: 86% (severely inflated)
benchmarking readFile/Original-MB1
time 18.93 ms (14.96 ms .. 24.41 ms)
0.796 R² (0.693 R² .. 0.974 R²)
mean 16.11 ms (15.08 ms .. 18.72 ms)
std dev 3.853 ms (1.724 ms .. 7.110 ms)
variance introduced by outliers: 86% (severely inflated)
benchmarking readFile/ExceptionFree-MB1
time 173.6 ms (111.2 ms .. 219.5 ms)
0.955 R² (0.926 R² .. 1.000 R²)
mean 220.9 ms (202.2 ms .. 245.1 ms)
std dev 29.51 ms (20.92 ms .. 38.04 ms)
variance introduced by outliers: 32% (moderately inflated)
benchmarking readFile/Oxidized-MB1
time 130.4 ms (116.8 ms .. 143.0 ms)
0.990 R² (0.981 R² .. 1.000 R²)
mean 148.7 ms (138.4 ms .. 174.8 ms)
std dev 23.15 ms (4.719 ms .. 35.59 ms)
variance introduced by outliers: 41% (moderately inflated)
benchmarking readFile/Original-MB10
time 140.3 ms (129.8 ms .. 153.5 ms)
0.990 R² (0.964 R² .. 1.000 R²)
mean 155.6 ms (145.7 ms .. 171.3 ms)
std dev 18.40 ms (9.635 ms .. 27.28 ms)
variance introduced by outliers: 27% (moderately inflated)
benchmarking readFile/ExceptionFree-MB10
time 1.952 s (1.365 s .. 2.302 s)
0.990 R² (NaN R² .. 1.000 R²)
mean 2.012 s (1.894 s .. 2.189 s)
std dev 185.2 ms (81.54 ms .. 258.6 ms)
variance introduced by outliers: 22% (moderately inflated)
benchmarking readFile/Oxidized-MB10
time 1.433 s (1.278 s .. 1.584 s)
0.998 R² (0.994 R² .. 1.000 R²)
mean 1.435 s (1.409 s .. 1.451 s)
std dev 25.96 ms (7.586 ms .. 35.23 ms)
variance introduced by outliers: 19% (moderately inflated)
Benchmark bench: FINISH
Completed 2 action(s).
Running it yourself
The tests, benchmark, and profile runs. The easiest way to run the tests and benchmarks is to use the Makefile:
make test (which runs test-unit and test-e2e targets)
make bench
make profile
Building the Oxidized (Rust-powered) Version
NOTE You must build this library yourself to use the oxidized version.
To build the Rust oxidized version of the library use the cabal flag "oxidized". You can run any of the common targets (build, bench, etc) with the OXIDIZED=true Make variable to build the oxidized version:
$ make rust # build the rust lib
$ make build OXIDIZED=true # build the exceptionfree-readfile lib & binary using rust
Note that you must have the rust toolchain (rustc, cargo, etc) installed before that command is run. To test out that it's working on your machine, build and use the binary like so:
$ make build OXIDIZED=true
$ ./target/exceptionfree-readfile -m oxidized test/fixtures/example-long-file.txt
<file output>
Using the oxidized version in your code
If you'd like to use the oxidized version in your code -- i.e. passing the flag in when you're building your project so cabal knows to enable the "oxidized" flag for exceptionfree-readfile, you'll need to:
- Build the rust code (
make rust)
- Ensure the dynamic shared library files (
rust/target/release/*.so) are available for your project at link & compile time via the usual methods (LD_LIBRARY_PATH, etc)
- Build your project (building
exceptionfree-readfile along the way)
- Ensure when you run your project's binary that the dynamic shared libraries (
*.so) files are available the usual ways (ex. in /lib//usr/lib)
Contributing
Contributions and pull requests are welcome! If you'd like to contribute:
- Fork this repository
- Make changes
make
make test
make profile bench (ensure the benchmark results are indeed better)
- Submit a PR