hw-json-simd: SIMD-based JSON semi-indexer

[ bsd3, data, library, program ] [ Propose Tags ]

Please see the README on GitHub at https://github.com/haskell-works/hw-json-simd#readme


[Skip to Readme]
Versions [faq] 0.1.0.0, 0.1.0.1, 0.1.0.2
Change log ChangeLog.md
Dependencies base (==4.*), bytestring (>=0.10.6 && <0.11), hw-json-simd, hw-prim (>=0.6.2.21 && <0.7), lens (==4.*), optparse-applicative (==0.14.*), semigroups (>=0.8.4 && <0.19), vector (==0.12.*) [details]
License BSD-3-Clause
Copyright 2018 - 2019 John Ky
Author John Ky
Maintainer newhoggy@gmail.com
Category Data
Home page https://github.com/haskell-works/hw-json-simd#readme
Bug tracker https://github.com/haskell-works/hw-json-simd/issues
Source repo head: git clone https://github.com/haskell-works/hw-json-simd
Uploaded by haskellworks at Wed Apr 3 10:51:32 UTC 2019
Distributions NixOS:0.1.0.2, Stackage:0.1.0.2
Executables hw-json-simd
Downloads 113 total (40 in the last 30 days)
Rating (no votes yet) [estimated by rule of succession]
Your Rating
  • λ
  • λ
  • λ
Status Hackage Matrix CI
Docs available [build log]
Last success reported on 2019-04-03 [all 1 reports]

Modules

[Index] [Quick Jump]

Flags

NameDescriptionDefaultType
avx2

Enable avx2 instruction set

DisabledAutomatic
bmi2

Enable bmi2 instruction set

DisabledAutomatic
sse42

Enable SSE 4.2 optimisations.

EnabledAutomatic

Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info

Downloads

Maintainer's Corner

For package maintainers and hackage trustees


Readme for hw-json-simd-0.1.0.2

[back to package description]

hw-json-simd

This library/tool will generate semi-indexes on JSON files as per the paper: Semi-Indexing Semi-Structured Data in Tiny Space.

The command line tool

For a given JSON file, hw-json-simd will generate two semi-index files, which both together can be loaded into a single in-memory semi-index.

The semi-index files can be generated using two methods, which will be called standard and simple, which correspond to sections 4 and 5 of the Semi-Indexing Semi-Structured Data in Tiny Space paper respectively.

Navigation of the JSON text using the standard index is suppored by the hw-json project for more information. There is currently no support for navigation of the JSON text using the simple index.

Using on the command line

cat test.json | pv -t -e -b -a | time hw-json-simd create-index \
  -i /dev/stdin
  --output-ib-file test.json.ib.idx
  --output-bp-file test.json.bp.idx
  --method standard
cat test.json | pv -t -e -b -a | time hw-json-simd create-index \
  -i /dev/stdin
  --output-ib-file test.json.ib.idx
  --output-bp-file test.json.bp.idx
  --method simple

Installation

From hackage

cabal new-install hw-json-simd