Introduction to logpecker
Summary
logpecker is an analysis tool for Unix syslog messages, written for
sysadmins. It will find the interesting messages in your pile, like
that this disk has SCSI failures, some RAID is now using the spare
disk, an NFS server is not responding anymore.
It can notify your tech support people immediately, create
summaries for not so important things or directly mail out reminders
into your ticketing system, and all this highly configurable.
logpecker is heavily tested on SuSE Linux, but should work well
on any POSIX complying Unix-like system that supports Python 2.0 or
higher.
Current Status
There is a first downloadable release (v0.1.1). This release has
only very limited documentation. While the overall processing will stay
similar, usage has shown that the syntax of rule definitions and main
configuration will have to change drastically, so consider this as a
pre-view. Nonetheless, the code has at least beta quality, I actually use it
in our production systems at work without trouble.
This documentation actually describes the next release. I expect
to have a beta by the end of October or earlier. Actually, the changes mostly
affect how the configuration is parsed.
Features
I know, there are other tools on the market, and a lot can be
achieved by grep and awk, but only logpecker has these unique
features:
- logpecker pre-parses syslog messages. This makes your rules easier
to write, read and maintain. Look at the Quick Tour, and you'll see.
- logpecker works incident-driven: From the logs and your rules it
derives incidents which refer to a specific problem. Multiple
reports for the same problem can be suppressed securely. In the
reports, you don't have only the log message but also the name that
includes details from the analysis. Incidents have a life-time and can
be marked as resolved, etc.
- logpecker rules work modular, i.e. there are several rule
collections each refering to one specific service or incident
type. These modules can be shared and distributed safely. One module
cannot interact with another one, and you can safely use other
people's rules. The rules define the transformation of syslog messages
into incidents; your master configuration decides what rule modules
are used and how important the created incidents are.
- Messages yet undefined are handled in a smart way that uses the syslog
priority information to derive the severity of the incident. Unknown
messages can be reported in a special way that makes it easy to
cut-and-paste them into your local rule base.
- batteries included: logpecker contains praxis-derived report
frameworks for message tickers, daily summaries and notification with an
escalation scheme, and it contains a rich rule base.
- Python coded: This might be a matter of taste, but in my opinion
Python code tends to be better readable and maintainable than Perl or
C for things like this. You can take a look at the sources and help programming new
features if you doubt it.
- There is this great documention you are reading, with explicit and detailed references and
typical use samples
- And, last not least, it's open source and free to use under GPL.
What it's not:
- it does not create statistical data yet (well, it could be implemented if you want to ...)
- it does not have a multi-colour clicky-clacky mouse interface (real sysadmins don't use mice)
- it is not a replacement for syslogd or similar daemons, but a companion.
Details
Please continue with the
Also available, but hard food if you haven't read the Quick Tour (in recommended order of reading):
If you want to compare, here's a link page to other logfile analysis tools.
You probably already know: logpecker is hosted by 