html-parse: A high-performance HTML tokenizer

[ bsd3, library, text ] [ Propose Tags ]

This package provides a fast and reasonably robust HTML5 tokenizer built upon the attoparsec library. The parsing strategy is based upon the HTML5 parsing specification with few deviations.

For instance,

>>> parseTokens "<div><h1 class=widget>Hello World</h1><br/>"
[TagOpen "div" [],
TagOpen "h1" [Attr "class" "widget"],
ContentText "Hello World",
TagClose "h1",
TagSelfClose "br" []]

The package targets similar use-cases to the venerable tagsoup library, but is significantly more efficient, achieving parsing speeds of over 80 megabytes per second on modern hardware and typical web documents. Here are some typical performance numbers taken from parsing a Wikipedia article of moderate length:

benchmarking Forced/tagsoup fast Text
time                 186.1 ms   (175.3 ms .. 194.6 ms)
0.999 R²   (0.995 R² .. 1.000 R²)
mean                 191.7 ms   (188.9 ms .. 198.3 ms)
std dev              5.053 ms   (1.092 ms .. 6.809 ms)
variance introduced by outliers: 14% (moderately inflated)

benchmarking Forced/tagsoup normal Text
time                 189.7 ms   (182.8 ms .. 197.7 ms)
0.999 R²   (0.998 R² .. 1.000 R²)
mean                 196.5 ms   (193.1 ms .. 202.1 ms)
std dev              5.481 ms   (2.141 ms .. 7.383 ms)
variance introduced by outliers: 14% (moderately inflated)

benchmarking Forced/html-parser
time                 15.81 ms   (15.75 ms .. 15.89 ms)
1.000 R²   (1.000 R² .. 1.000 R²)
mean                 15.72 ms   (15.66 ms .. 15.77 ms)
std dev              140.9 μs   (113.6 μs .. 174.5 μs)
Versions 0.1.0.0, 0.2.0.0, 0.2.0.1, 0.2.0.2
Dependencies attoparsec (==0.13.*), base (>=4.7 && <4.13), containers (>=0.5 && <0.7), deepseq (>=1.3 && <1.5), text (==1.2.*) [details]
License BSD-3-Clause
Copyright (c) 2016 Ben Gamari
Author Ben Gamari
Maintainer ben@smart-cactus.org
Category Text
Home page http://github.com/bgamari/html-parse
Source repo head: git clone git://github.com/bgamari/html-parse
Uploaded by BenGamari at Thu Jul 19 16:17:55 UTC 2018
Distributions NixOS:0.2.0.2
Downloads 782 total (18 in the last 30 days)
Rating (no votes yet) [estimated by rule of succession]
Your Rating
  • λ
  • λ
  • λ
Status Docs available [build log]
Last success reported on 2018-07-19 [all 1 reports]
Hackage Matrix CI

Modules

[Index]

Downloads

Maintainer's Corner

For package maintainers and hackage trustees