pan-os-syslog: Parse syslog traffic from PAN-OS

[ bsd3, library, network ] [ Propose Tags ]

Parse syslog traffic from PAN-OS. The data types in this library are optimized for decoding logs, not for creating them. On consumer-grade hardware, the benchmark suite demonstrates that 500-byte traffic logs are parsed in under one microsecond. Contributions are welcome. This project's goals are:

• Support as many PAN-OS syslog types as possible: traffic, threat, hip-match, etc.

• Support as many versions of PAN-OS as possible: 8.0, 8.1, 9.0, etc.

• High performance. This library strives to avoid unneeded allocations. Some allocations cannot be avoided. For example, it is necessary to allocate space for the results.

• Do a minimum amount of useful work on each field. The reasoning is that users will typically discard most of the fields, so there is no point wasting clock cycles doing unneeded work. Its hard to define what this is precisely. Roughly, the rule this library follows is that integral fields are parsed as Word64, and non-integral fields are Bytes. This library does not attempt to validate hostnames, URIs, etc.

A good way to think about this library is that it is kind of like a tokenizer. It is the first step when parsing PAN-OS logs into some application-specific data type. There almost certainly needs to be a second step to decodes fields that are actually of interest to an application. This second step may involve validating URIs, splitting the user domain and user name, etc.