hw-json-simd: SIMD-based JSON semi-indexer

This is a package candidate release! Here you can preview how this package release will appear once published to the main package index (which can be accomplished via the 'maintain' link below). Please note that once a package has been published to the main package index it cannot be undone! Please consult the package uploading documentation for more information.

[maintain]

Please see the README on GitHub at https://github.com/haskell-works/hw-json-simd#readme


[Skip to ReadMe]

Properties

Versions0.1.0.0, 0.1.0.0, 0.1.0.1
Change logChangeLog.md
Dependenciesbase (==4.*), bytestring (>=0.10.6 && <0.11), hw-json-simd, hw-prim (>=0.6.2.21 && <0.7), lens (==4.*), optparse-applicative (==0.14.*), semigroups (>=0.8.4 && <0.19), vector (==0.12.*) [details]
LicenseBSD-3-Clause
Copyright2018 - 2019 John Ky
AuthorJohn Ky
Maintainernewhoggy@gmail.com
CategoryData
Home pagehttps://github.com/haskell-works/hw-json-simd#readme
Bug trackerhttps://github.com/haskell-works/hw-json-simd/issues
Source repositoryhead: git clone https://github.com/haskell-works/hw-json-simd
Executableshw-json-simd
UploadedSat Jan 26 11:35:40 UTC 2019 by haskellworks

Modules

[Index]

Flags

NameDescriptionDefaultType
avx2

Enable avx2 instruction set

DisabledAutomatic
bmi2

Enable bmi2 instruction set

DisabledAutomatic
sse42

Enable SSE 4.2 optimisations.

EnabledAutomatic

Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info

Downloads

Maintainers' corner

For package maintainers and hackage trustees


Readme for hw-json-simd-0.1.0.0

[back to package description]

hw-json-simd

This library/tool will generate semi-indexes on JSON files as per the paper: Semi-Indexing Semi-Structured Data in Tiny Space.

The command line tool

For a given JSON file, hw-json-simd will generate two semi-index files, which both together can be loaded into a single in-memory semi-index.

The semi-index files can be generated using two methods, which will be called standard and simple, which correspond to sections 4 and 5 of the Semi-Indexing Semi-Structured Data in Tiny Space paper respectively.

Navigation of the JSON text using the standard index is suppored by the hw-json project for more information. There is currently no support for navigation of the JSON text using the simple index.

Using on the command line

cat test.json | pv -t -e -b -a | time hw-json-simd create-index \
  -i /dev/stdin
  --output-ib-file test.json.ib.idx
  --output-bp-file test.json.bp.idx
  --method standard
cat test.json | pv -t -e -b -a | time hw-json-simd create-index \
  -i /dev/stdin
  --output-ib-file test.json.ib.idx
  --output-bp-file test.json.bp.idx
  --method simple

Installation

From hackage

cabal new-install hw-json-simd