hw-json-simd: SIMD-based JSON semi-indexer

[ bsd3, data, library, program ] [ Propose Tags ]


Automatic Flags

Enable avx2 instruction set


Enable bmi2 instruction set


Enable SSE 4.2 optimisations.


Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info


Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees


Versions [RSS],,,,,,,
Change log ChangeLog.md
Dependencies base (>=4.11 && <5), bytestring (>=0.10.6 && <0.12), hw-json-simd, hw-prim (>= && <0.7), lens (>=4 && <6), optparse-applicative (>=0.14 && <0.18), vector (>=0.12 && <0.14) [details]
License BSD-3-Clause
Copyright 2018-2021 John Ky
Author John Ky
Maintainer newhoggy@gmail.com
Category Data
Home page https://github.com/haskell-works/hw-json-simd#readme
Bug tracker https://github.com/haskell-works/hw-json-simd/issues
Source repo head: git clone https://github.com/haskell-works/hw-json-simd
Uploaded by haskellworks at 2022-12-19T23:27:24Z
Distributions LTSHaskell:, NixOS:, Stackage:
Reverse Dependencies 3 direct, 6 indirect [details]
Executables hw-json-simd
Downloads 3205 total (29 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs uploaded by user
Build status unknown [no reports yet]

Readme for hw-json-simd-

[back to package description]


This library/tool will generate semi-indexes on JSON files as per the paper: Semi-Indexing Semi-Structured Data in Tiny Space.

The command line tool

For a given JSON file, hw-json-simd will generate two semi-index files, which both together can be loaded into a single in-memory semi-index.

The semi-index files can be generated using two methods, which will be called standard and simple, which correspond to sections 4 and 5 of the Semi-Indexing Semi-Structured Data in Tiny Space paper respectively.

Navigation of the JSON text using the standard index is suppored by the hw-json project for more information. There is currently no support for navigation of the JSON text using the simple index.

Using on the command line

cat test.json | pv -t -e -b -a | time hw-json-simd create-index \
  -i /dev/stdin
  --output-ib-file test.json.ib.idx
  --output-bp-file test.json.bp.idx
  --method standard
cat test.json | pv -t -e -b -a | time hw-json-simd create-index \
  -i /dev/stdin
  --output-ib-file test.json.ib.idx
  --output-bp-file test.json.bp.idx
  --method simple


From hackage

cabal new-install hw-json-simd