Streaming.Osm

This library provides the ability to read and process OpenStreetMap data via the streaming ecosystem. Since streaming allows for very little RAM overhead despite file size, we can process very large OSM PBF files just by providing a file path:

import           Streaming
import           Streaming.Osm
import qualified Streaming.Prelude as S

-- | Count all nodes.
count :: IO ()
count = do
  len <- runResourceT . S.length_ . nodes . blocks $ blobs "yourfile.osm.pbf"
  print len

Streams

blobs :: FilePath -> Stream (Of Blob) RIO ()

Given a FilePath to read OSM PBF data from, stream all parsed Blobs out of it. A Blob is a potentially compressed ByteString that further parse into actual OSM Elements.

blocks :: Stream (Of Blob) RIO () -> Stream (Of Block) RIO ()

Every Block of ~8000 Elements.

nodes :: Stream (Of Block) RIO () -> Stream (Of Node) RIO ()

All OSM Nodes.

ways :: Stream (Of Block) RIO () -> Stream (Of Way) RIO ()

All OSM Ways.

relations :: Stream (Of Block) RIO () -> Stream (Of Relation) RIO ()

All OSM Relations.

Util

type RIO = ResourceT IO

A friendly alias. All Streams in this module are constrained to this type for optimal performance. The opposite, say:

nodes :: Monad m => Stream (Of Block) m () -> Stream (Of Node) m ()

will actually run significantly slower.

After evaluating your Stream to some final RIO, you can further escape back to IO via runResourceT.