Another series of [philosopher's] stones in the form of “programming tools” is produced under the banner of “software engineering”, which, as time went by, has sought to replace intellectual discipline by management discipline to the extent that it has now accepted as its charter “How to program if you cannot”.
Edsger W. Dijkstra: The threats to computer science (EWD 898)

Code

This software is made available "as is" - no guarantees, etc, etc. There is no support provided for any of this software, but feel free to Email me if you find a bug in any of it.

Overwatch A scoring system/engine for anomaly detection. Maintains databases of scores tied to frequencies at which things happen. (A paper on Overwatch)

NBS Never Before Seen Anomaly detection driver. This utility creates a fast database of things that have been seen, and includes tools to print and update the database. Includes PDF documentation and walkthroughs.

plog Plog is a promiscuous syslog listener. It sucks UDP syslog packets up off a network, rips the message screaming and kicking out of the packet body, and stuffs it into /dev/log. This program supports a bare minimum of options. Be very careful you do not use plog to inject messages into a syslog server that forwards the messages to a loghost over a network! It will hurt! (the good news is you'll get lots of log messages..)

retail Retail is a stateful version of the 'tail' command. It's used for submitting new additions to a logfile through a batch process. In other words, if you have a logfile that "rolls" every day, but want to get the stuff that has been added since the last time you looked at it, retail is the tool for the job!

logbayes Logbayes is a really kludgy implementation of an experimental bayesian log analysis tool. It uses Eric Raymond's "bogofilter" to do the heavy lifting (actually, it's just a wrapper around bogofilter). I've had some interesting results with this toy; I wish someone would make something like it that was faster and designed specifically with logs in mind!! PDF slides are included with the distribution.

choplog Choplog is a minimal tool for normalizing and parsing system log messages based on a single template. It won't do a good or useful job against free-form messages like syslog produces but is useful for picking fields out of a web log or similar log. Also re-buckets log entries into files by time (very useful!) No documentation. See choplog.cf for vague hints how it works.

Overwatch	A scoring system/engine for anomaly detection. Maintains databases of scores tied to frequencies at which things happen. (A paper on Overwatch)
NBS	Never Before Seen Anomaly detection driver. This utility creates a fast database of things that have been seen, and includes tools to print and update the database. Includes PDF documentation and walkthroughs.
plog	Plog is a promiscuous syslog listener. It sucks UDP syslog packets up off a network, rips the message screaming and kicking out of the packet body, and stuffs it into /dev/log. This program supports a bare minimum of options. Be very careful you do not use plog to inject messages into a syslog server that forwards the messages to a loghost over a network! It will hurt! (the good news is you'll get lots of log messages..)
retail	Retail is a stateful version of the 'tail' command. It's used for submitting new additions to a logfile through a batch process. In other words, if you have a logfile that "rolls" every day, but want to get the stuff that has been added since the last time you looked at it, retail is the tool for the job!
logbayes	Logbayes is a really kludgy implementation of an experimental bayesian log analysis tool. It uses Eric Raymond's "bogofilter" to do the heavy lifting (actually, it's just a wrapper around bogofilter). I've had some interesting results with this toy; I wish someone would make something like it that was faster and designed specifically with logs in mind!! PDF slides are included with the distribution.
choplog	Choplog is a minimal tool for normalizing and parsing system log messages based on a single template. It won't do a good or useful job against free-form messages like syslog produces but is useful for picking fields out of a web log or similar log. Also re-buckets log entries into files by time (very useful!) No documentation. See choplog.cf for vague hints how it works.