hw-mquery: Conduits for tokenizing streams.

[ bsd3, data, library, program ] [ Propose Tags ]

Please see README.md


[Skip to Readme]
Versions [faq] 0.0.0.1, 0.1.0.0, 0.1.0.1, 0.1.0.3, 0.2.0.0, 0.2.0.1
Dependencies ansi-wl-pprint, base (>=4.8 && <5), dlist, hw-mquery [details]
License MIT
Copyright 2016 John Ky
Author John Ky
Maintainer newhoggy@gmail.com
Revised Revision 1 made by GeorgeWilson at Mon Jun 3 07:02:10 UTC 2019
Category Data, Conduit
Home page http://github.com/haskell-works/hw-mquery#readme
Source repo head: git clone https://github.com/haskell-works/hw-mquery
Uploaded by haskellworks at Thu Sep 29 22:44:08 UTC 2016
Distributions LTSHaskell:0.1.0.3, NixOS:0.2.0.1, Stackage:0.2.0.1
Executables hw-mquery-example
Downloads 1643 total (70 in the last 30 days)
Rating (no votes yet) [estimated by rule of succession]
Your Rating
  • λ
  • λ
  • λ
Status Hackage Matrix CI
Docs available [build log]
Last success reported on 2016-11-19 [all 1 reports]

Modules

[Index]

Downloads

Note: This package has metadata revisions in the cabal description newer than included in the tarball. To unpack the package including the revisions, use 'cabal get'.

Maintainer's Corner

For package maintainers and hackage trustees


Readme for hw-mquery-0.1.0.1

[back to package description]

hw-mquery

0.0-branch

import           Control.Monad
import qualified Data.DList as DL
import           Data.Function
import           Data.List
import           HaskellWorks.Data.LoadJson
import           HaskellWorks.Data.Micro
import           HaskellWorks.Data.MQuery
import           HaskellWorks.Data.Json.PartialValue
import           HaskellWorks.Data.Row
import           HaskellWorks.Diagnostics
import           Text.PrettyPrint.ANSI.Leijen
!json <- loadJsonPartial "data/78mb.json"
!json <- loadJsonWithIndex "data/78mb.json"
!json <- loadJsonWithPoppy512Index "data/78mb.json"
let q = MQuery (DL.singleton json)
putPretty $ q >>= item & limit 10
putPretty $ q >>= item & page 10 1
putPretty $ q >>= item >>= hasKV "founded_year" (JsonPartialNumber 2005) & limit 10
putPretty $ q >>= item >>= entry
putPretty $ q >>= item >>= entry >>= named "name" & limit 10
putPretty $ q >>= item >>= entry >>= satisfying (\(k, _) -> k == "name") >>= value & limit 10
putPretty $ q >>= item >>= entry >>= satisfying ((== "name") . fst) >>= value & limit 10
putPretty $ q >>= (item >=> entry >=> key) & limit 10
putPretty $ q >>= item >>= entry >>= key & limit 100 & onList (uniq . sort)
putPretty $ (q >>= item >>= entry & limit 1) >>= field "name" & limit 10
putPretty $ do {j <- q; e <- item j; (k, v) <- entry e; return k}
putPretty $ do {j <- q; e <- item j; (k, v) <- entry e; guard (k == "name"); return v}

Decoding

Line separated base 64 encoded gzipped json

while read in; do echo "$in" | base64 --decode | gunzip; echo ""; done < file.lgz > firehose.json

Profiling with stack traces

mafia build -p
cabal repl --ghc-options='-fexternal-interpreter -prof'
import HaskellWorks.Data.Succinct.BalancedParens
import HaskellWorks.Data.Succinct.RankSelect.Binary.Poppy512
import HaskellWorks.Data.Positioning
import qualified Data.Vector.Storable as DVS
import HaskellWorks.Data.IndexedSeq
(jsonBS, jsonIb, jsonBp) <- loadJsonRawWithIndex "firehose.json"
let bp1 = SimpleBalancedParens jsonBp
let bp2 = SimpleBalancedParens (makePoppy512 jsonBp)
let bp3 = makePoppy512 jsonBp