Logtool A logfile parsing/manipulation tool for managing syslog/syslog-ng/etc generated logfiles. What: At it's heart, logtool is a command line program, to parse up logfiles into a more palatable format. It will take anything resembling a standard syslog file (this includes syslog-ng, and probably most of the other variants out there), and crunch it into one of the following formats for your viewing pleasure: ANSI (colorized for easy "at a glance" viewing) ASCII (for e-mail'ed reports, and term's that don't support color) CSV (for importing into your favorite spreadsheet/database) HTML (for generating web pages (duh?)) RAW (for no good reason other than an excuse to printf();) New as of the 1.2.x branch, logtool will also parse logfiles created by D.J.Bernstien's multilog and other similarly formatted logfiles, as well as handle unknown formating. It also knows some specizalized logfile formats such as Snort, and IPtables, and will parse those into specialized colors as well as do IP->Hostname resolution and other neat things. It can be configured to parse the data any one of several ways, including stripping the host, and/or program fields, and modifying the time display format of the log entry's. New in the 1.2.x branch, it can also handle syslog-ng style hostname fields, and optionally strip the src@/kern@ for scree-space sake, as well as parse the Hostname/IP format to show only hostname or IP address(es). See logtool.conf for more details. Somewhat new in the 1.2.x branch, is the reliance on config file's as the primary runtime configuration rather than command line switches. The default logtool.conf is well commented, and you should have a look through it to get comfortable with the options at your disposal. Those of you who have written scripts and such which rely on the old command-line syntax will be happy to know that backwards compatibility has been maintained in this regard, though you should update your scripts to use config file's as much as possible. Please check out the doc/* directory for more information on the things you can/can't do with logtool (like how to do them :). Why?: For all my Linux/UNIX life, especially as a SysAdmin, I've been up to my neck in logfiles. I'm also a shell script junkie, and I've been writing scripts to make my life easier (for logfiles, and everything else) for years, but when dealing with big logfiles (and it doesn't take much to become a "big" file to a shell script based parsing routine), it's just too dang slow to do it right. Perl is a pain in the butt in terms of portability (yes, I know you can run perl anywhere... anywhere you've installed a gazillion friggin perl modules). So what I really needed was a nice little C program to handle the grunt work, and some shell scripts to interface to that program. I looked around, but I didn't see any GNU tools to do precisely what I wanted, so I wrote this one, and am now releasing it to the public, because (IMHO), there should be a program like this in the world to let folks who have to muck with logfiles have easier lives. Uses: I expect this program to have primarily two uses (although if it suits other needs, use it for whatever you want). 1. As an engine for use by shell scripts to generate nice little reports to e-mail people, or create webpages, and other similar uses. 2. As a logfile monitoring tool for network operations center like environments, where it's nice to have logfile's scrolling across the screen in easy to read color. I've included a "redbeep" option so that in such cases, "red" events will generate a Ctrl-G to catch the attention of NOC personnel. Notes: This program _should_ compile/run on almost anything resembling UNIX, although I have only tested with Linux. Hopefully some of you will give it a try on some of the other UNIX flavors out there. It was written in plain old C library function call's (no snprintf() or other things that are known to be pesky on some UNIX'es (Solaris anyone?)), and it compiles clean with -W -Wall passed to the compiler, so I assume it is something like reasonable code (I'm a self-taught programmer, so all bets are off if the compiler lied :). Especiall note, the 1.2.x code tree has changed a LOT from the 1.0 tree. It may no longer work on platforms it used to work on! If you find your platform is one of these, please feel free to contact me (al@xjack.org) and let me know the problem, and I'll work on fixing it for you (free of charge even, can you beat that? :) Also, the documentation is not yet complete (nobody likes to write documentation, me included), but what is there should be enough to get you up to speed if you're at all familiar with UNIX or Linux. --A.L.Lambert