jsontsv: JSON to TSV transformer

[ mit, program, text ] [ Propose Tags ]

Transforms JSON into tab-separated line-oriented output, for easier processing in Unix-style pipelines.


[Skip to Readme]

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

  • No Candidates
Versions [RSS] 0.1.0.0, 0.1.1.0, 0.1.1.1, 0.1.2.0, 0.1.3.0, 0.1.3.1, 0.1.4.0, 0.1.4.1, 0.1.4.2, 0.1.4.3, 0.1.4.4, 0.1.4.5, 0.1.4.6, 0.1.5.0, 0.1.6.0, 0.1.6.1, 0.1.7.0
Dependencies aeson (>=0.8.0.0), attoparsec (>=0.12.1.0), base (>=4.7 && <4.8), bytestring, containers, csv (>=0.1.2), optparse-applicative, scientific, text (>=1.2.0.0), unordered-containers, vector [details]
License MIT
Copyright (c) 2014 Daniel Choi
Author Daniel Choi
Maintainer dhchoi@gmail.com
Category Text
Home page https://github.com/danchoi/jsontsv
Uploaded by DanielChoi at 2014-12-03T04:55:37Z
Distributions
Reverse Dependencies 1 direct, 0 indirect [details]
Executables jsontsv
Downloads 10204 total (52 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs not available [build log]
Last success reported on 2015-06-02 [all 6 reports]

Readme for jsontsv-0.1.2.0

[back to package description]

jsontsv

Transforms JSON objects into tab-separated line-oriented output, which is more convenient for downstream processing with Unix tools like grep, awk, diff, etc., and for inspecting with spreadsheet programs using a CSV output option.

Synopsis

input.json:

{
  "title": "Terminator 2: Judgement Day",
  "year": 1991,
  "stars": [
    "Arnold Schwarzenegger",
    "Linda Hamilton"
  ],
  "ratings": {
    "imdb": 8.5
  }
}
{
  "title": "Interstellar",
  "year": 2014,
  "stars": [
    "Matthew McConaughey",
    "Anne Hathaway"
  ],
  "ratings": {
    "imdb": 8.9
  }
}
{
  "title": "Terminator 2: Judgement Day",
  "year": 1991,
  "stars": [
    {
      "name": "Arnold Schwarzenegger"
    },
    {
      "name": "Linda Hamilton"
    }
  ],
  "ratings": {
    "imdb": 8.5
  }
}
{
  "title": "Interstellar",
  "year": 2014,
  "stars": [
    {
      "name": "Matthew McConaughey"
    },
    {
      "name": "Anne Hathaway"
    }
  ],
  "ratings": {
    "imdb": 8.9
  }
}

Note that this input is not actually JSON at the top-level. It is a stream of JSON objects. It can be fed into jsontsv:

jsontsv 'title year stars.name ratings.imdb' < input.json

outputs this tab-separated text:

Terminator 2: Judgement Day	1991	Arnold Schwarzenegger,Linda Hamilton	8.5
Interstellar	2014	Matthew McConaughey,Anne Hathaway	8.9

You can pick off array elements using [i] syntax:

jsontsv 'title year stars[0].name' < input.json

outputs

Terminator 2: Judgement Day     1991    Arnold Schwarzenegger
Interstellar    2014    Matthew McConaughey

Installation

Assuming you have a recent version of the Haskell platform on your system,

cabal update
cabal install jsontsv

Make sure the installed executable is on your PATH.

Usage

jsontsv

Usage: jsontsv FIELDS ([-c|--csv] | [-d DELIM])
  Transform JSON objects to TSV

Available options:
  -h,--help                Show this help text
  -c,--csv                 output CSV
  -d DELIM                 output field delimiter. Defaults to tab

Input should be a stream of JSON objects of the same or mostly similar shape, separated by whitespace such as newlines. If the objects are wrapped in a JSON array at the top level or nested inside a top-level object, use the jq tool by Stephan Dolan to extract an object stream, e.g.:

curl -s "https://api.github.com/users/danchoi/repos?type=owner&sort=created&direction=desc" \
    | jq '.[]' | jsontsv 'id name stargazers_count open_issues_count' 

outputs

27397673        jsontsv 0       0
26033118        ngrender        24      1
25832026        rdoc    0       0
24756523        treehtml        0       0
24022588        heistexamples   0       0
24022042        hxtexamples     0       0
24005242        jdiff   0       0
23997156        https-types     0       0
22763562        podcasting      0       0
19294791        vimscript       3       0

JSON leaf values are printed as follows:

  • Strings and numbers are copied to output.
  • Boolean values are output as t or f.
  • null is printed as null
  • If the leaf value is an array, it is concatenated into a single comma-separated string

Performing post-processing on field values

Use awk!

Known alternatives