curl-runnings: A framework for declaratively writing curl based API tests

[ library, mit, program, testing ] [ Propose Tags ]

Please see the README on Github at https://github.com/aviaviavi/curl-runnings#readme


[Skip to Readme]
Versions [faq] 0.1.0, 0.2.0, 0.3.0, 0.6.0, 0.9.2, 0.10.0, 0.11.0, 0.11.1, 0.12.0, 0.14.0, 0.15.0, 0.16.0
Dependencies aeson (>=1.2.4.0), base (>=4.7 && <5), base64-bytestring (>=1.0.0.2), bytestring (>=0.10.8.2), case-insensitive (>=0.2.1), clock (>=0.7.2), cmdargs (>=0.10.20), connection (>=0.2.8), curl-runnings, dhall (>=1.8.2), dhall-json (>=1.0.9), directory (>=1.3.0.2), hashable (>=1.3.0.0), hspec (>=2.4.4), hspec-expectations (>=0.8.2), http-client-tls (>=0.3.5.3), http-conduit (>=2.3.6), http-types (>=0.12.3), megaparsec (>=7.0.4), pretty-simple (>=2.0.2.1), regex-posix (>=0.95.2), tar (>=0.5.0.3), text (>=1.2.2.2), time (>=1.8.0.2), transformers (>=0.5.2.0), unordered-containers (>=0.2.8.0), vector (>=0.12.0), yaml (>=0.8.28), zlib (>=0.6.1.2) [details]
License MIT
Copyright 2020 Avi Press
Author Avi Press
Maintainer mail@avi.press
Category Testing
Home page https://github.com/aviaviavi/curl-runnings#readme
Bug tracker https://github.com/aviaviavi/curl-runnings/issues
Source repo head: git clone https://github.com/aviaviavi/curl-runnings
Uploaded by aviaviavi at 2020-10-03T22:31:29Z
Distributions NixOS:0.16.0
Executables curl-runnings
Downloads 3863 total (9 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Hackage Matrix CI
Docs available [build log]
Last success reported on 2020-10-03 [all 1 reports]

Modules

[Index] [Quick Jump]

Downloads

Maintainer's Corner

For package maintainers and hackage trustees


Readme for curl-runnings-0.16.0

[back to package description]

curl-runnings

Build Status Hackage Scarf

Feel the rhythm! Feel the rhyme! Get on up, it's testing time! curl-runnings!

A common form of black-box API testing boils down to simply making requests to an endpoint and verifying properties of the response. curl-runnings aims to make writing tests like this fast and easy.

curl-runnings is a framework for writing declarative tests for your APIs in a fashion equivalent to performing curls and verifying the responses. Write your tests quickly and correctly with a straight-forward specification in Dhall, yaml, or json that can encode simple but powerful matchers against responses.

Alternatively, you can use the curl-runnings library to write your tests in Haskell (though a Haskell setup is absolutely not required to use this tool).

Installing

The best way to install curl-runnings is with the scarf package manager.

# If you don't have scarf, you can easily install it with:
$ curl -L https://scarf.sh/install | bash
 
$ scarf install curl-runnings

Alternatively, you can compile from source with stack.

Writing a test specification

Curl runnings tests are just data! A test spec is an object containing an array of cases, where each item represents a single curl and set of assertions about the response. Write your tests specs in a Dhall, yaml or json file. Note: the legacy format of a top level array of test cases is still supported, but may not be in future releases.

let JSON = https://prelude.dhall-lang.org/JSON/package.dhall

let CurlRunnings = ./dhall/curl-runnings.dhall

in   CurlRunnings.hydrateCase
        CurlRunnings.Case::{
        , expectData = Some
            ( CurlRunnings.ExpectData.Exactly
                ( JSON.object
                    [ { mapKey = "okay", mapValue = JSON.bool True },
                      { mapKey = "message", mapValue = JSON.string "a message" }]
                )
            )
        , expectStatus = 200
        , name = "test 1"
        , requestMethod = CurlRunnings.HttpMethod.GET
        , url = "http://your-endpoing.com/status"
        }
---
# example-test.yaml
#
# specify all your test cases as an array keys on `cases`
cases:
  - name: A curl runnings test case
    url: http://your-endpoint.com/status
    requestMethod: GET
    # Specify the json payload we expect here
    expectData:
      # The 1 key in this object specifies the matcher we want
      # to use to test the returned payload. In this case, we
      # require the payload is exactly what we specify.
      exactly:
        okay: true
        msg: 'a message'
    # Assertions about the returned status code. Pass in
    # an acceptable code or list of codes
    expectStatus: 200

See /examples for more example curl runnings specifications, which walk through some of the other features that can be encoded in your tests such as:

  • reference data from previous responses of previous test cases
  • reference environment variables
  • various easy-to-use json matchers
  • support for importing data from other yaml files in your spec

Running

Once you've written a spec, simply run it with:

curl-runnings -f path/to/your/spec.yaml

(hint: try using the --verbose flag for more output)

If all your tests pass, curl-runnings will cleanly exit with a 0 code. A code of 1 will be returned if any tests failed.

You can also select specific test cases by filtering via regex by using the --grep flag. Just make sure your case isn't referencing data from previous examples that won't get run!

For more info:

curl-runnings --help

Running With Docker

A dockerfile is included in the root of the project. The Dockerfile will expect the linux based curl-runnings executable in the same directory as the Dockerfile and a tests.yml file. You can download the latest executable from the release page : https://github.com/aviaviavi/curl-runnings/releases .

docker build . -t curl-runnings-tests

docker run curl-runnings-tests

If you use docker-compose, you can add this to docker-compose.yml:

tests:
    build:
      context: .
      dockerfile: ./Dockerfile

Contributing

Contributions in any form are welcome and encouraged. Don't be shy! :D