The eros package

[Tags: bsd3, library]

A Haskell library for censoring text, using DansGuardian phraselists. I converted the phraselists into JSON. You can view the converted phraselists here. There exist compressed versions, for use within your code. You can see the compressed versions here. The library is not very well tested (or really, tested at all), so I wouldn't recommend using it for anything important just quite yet. I'm quickly working towards a stable version, though! I recommend looking at the API documentation for Text.Eros if you want an idea of how to use the library. Hackage seems to be completely unable to build the documentation for Eros. For that reason, I publish the documentation on GitHub.


[Skip to ReadMe]

Properties

Versions0.0.0.0, 0.1.0.0, 0.2.0.0, 0.2.0.1, 0.2.1.0, 0.2.1.1, 0.3.0.0, 0.3.0.1, 0.3.1.0, 0.4.0.0, 0.4.0.1, 0.4.1.0, 0.5.0.0, 0.5.1.0, 0.5.2.0, 0.5.3.0, 0.5.3.1, 0.5.3.2, 0.6.0.0
Change logNone available
Dependenciesaeson, aeson-pretty, base (==4.7.*), bytestring, containers, eros, text [details]
LicenseBSD3
Copyright2014, Peter Harpending.
AuthorPeter Harpending
MaintainerPeter Harpending <pharpend2@gmail.com>
CategoryText
Source repositoryhead: git clone https://github.com/pharpend/eros.git -b master
Executableserosc
UploadedMon Jun 30 06:35:16 UTC 2014 by pharpend
DistributionsNixOS:0.6.0.0
Downloads2078 total (95 in last 30 days)
Votes
0 []
StatusDocs uploaded by user
Build status unknown [no reports yet]

Modules

[Index]

Downloads

Maintainers' corner

For package maintainers and hackage trustees

Readme for eros-0.3.1.0

Eros

A Haskell library for text censorship, using DansGuardian Phraselists.

I converted those Phraselists to JSON. You can see the converted Phraselists here. There are compressed versions for use in your code.

Eros is still in development, and is not ready to be actually used. If you would like to contribute, please do.

At the time of this writing, I'm at version 0.3 (see here) for the current release.

See the API documentation on Hackage if you want to learn how to use the library.

Usage - v.0.3.1.0

This is a usage guide for version 0.3.1.0. There will be more up-to-date usage guides as more versions come, hopefully.

To install, run cabal install eros.

erosc

The only way to interact with the library, for the time being is through the client I built, erosc. It accepts input in Javascript Object Notation (JSON) through stdin.

Here is the schema for the input:

{
  "name": "erosc-input",
  "type": "object",
  "description": "The schema for the input to erosc.",
  "properties": {
    "text": {
      "type": "string",
      "description": "The text you want to be checked against the phraselists.",
      "required": true
    },
    "eros-lists": {
      "type": "array",
      "description": "The phraselists provided by eros you want \"text\" to be checked against.",
      "required": true,
      "items": {
        "type": "string"
      }
    },
  }
}

That is the up-to-date schema, as of version 0.3.1.0 . It is liable to change. If the version you downloaded is greater than 0.3.1.0, make sure to check the schema (it is distributed with the package) to make sure it is up to date. The schema is located in res/schemata/erosc-input.json.

erosc will take that input, and send back output in JSON, in accordance with the schema found in res/schemata/erosc-output.json.

{
  "name": "erosc-output",
  "type": "array",
  "description": "The output of erosc.",
  "items": {
    "type": "object",
    "description": "The phraselist name, along with the score for that phraselist.",
    "properties": {
      "eros-list": {
        "type": "string",
        "description": "The name of the phraselist corresponding to this object."
      },
      "score": {
        "type": "number",
        "description": "The score corresponding to this phraselist."
      }
    }
  }
}

Example

This is the example input from res/erosc-dummy-inputs/input0.json.

{
  "text": "Fuck you, you fucking fuck! Fucking bitch tits milf sex sluts!",
  "eros-lists": [
    "gambling",
    "pornography"
  ]
}

Running erosc < res/erosc-dummy-inputs/input0.json yields

[
  {
    "score": 0,
    "eros-list": "gambling"
  },
  {
    "score": 11394,
    "eros-list": "pornography"
  }
]

Library

You are welcome to build your own client, and use that. To do so, simply import Text.Eros. Hackage seems to be unable to build the API documentation for Eros, but it won't hurt to check eros on Hackage.

If that doesn't work, I publish the documentation here.

Contributing

If you want to contribute, you'll need ghc and cabal-install

  1. Clone the git repo

    git clone https://github.com/pharpend/eros.git
    
  2. Install in a sandbox

    cd eros
    cabal sandbox init
    cabal install --enable-tests
    

Planned features

erosceros-server

As it stands, erosc is tedious and a bit difficult to use. I plan on rewriting erosc to act as an HTTP[s] server, using Happstack, using acid-state as a RAM cloud. This will be the fastest and easiest to maintain solution.

I will still include an offline client, but it won't be the focus of development.

More options

I need to add more filtering options, such as phraselist-specific thresholds, custom phraselists, and, well, just more stuff.

If you have any ideas, please don't hesitate to email me at pharpend2@gmail.com.