logfilterd installation instructions (rough guide) ------------------------------------------------------------------------------ - Make sure you have Perl 5.004 or newer installed. - Ensure that you have the following Perl modules installed on your system: Net::Domain Date::Manip Set::IntSpan Mail::Send I /think/ I'm right in saying you can check which of these are already installed by running the following command: % find `perl -le 'print "@INC"'` -name \*.pm > /tmp/perl.modules and then checking /tmp/perl.modules to see whether they're mentioned. (Make sure that the perl interpreter on your path is the one which will be running the script when you type this command.) If you're missing some of these, install them. The easiest way is probably by doing the following (as root): # perl -MCPAN -e shell This provides an excellent, easy to use, front end interface to the CPAN archive, from which you can easily perform automated installs of the required modules mentioned above. The weak-hearted should type 'man CPAN' for more details; others can just type the command and follow things through. - Put the logfilterd script somewhere sensible (e.g. /usr/local/logfilterd/logfilterd). ** BEGIN EDITING THE SCRIPT AS FOLLOWS: (use emacs with folding mode loaded if at all possible) - Edit the 'my $email_to = ...' line near the top of the script so that it points to an appropriate e-mail address. - Edit the file and directory locations near the top of the script. Get rid of any mentions of output files you don't want. For example, if you don't run dhcpd, remove the my $dhcpd_log = "$output_dir/dhcpd"; line. Obviously, you would then also need to go through the script removing any lines other mentioning dhcpd. Common sense applies here; there should be no hidden catches. The only slightly tricky case may be if you're not running Abacus Sentry or doing anything else which requires contexts, in which case you may want to remove all the context code (although you could probably get away without doing so). - Edit the &filter_line subroutine (this is where all the pattern matching/filtering actually gets done) to suit your own filtering requirements. Knowledge of Perl regexps is a serious advantage at this point! (man perlre if you need to brush up on this.) Note that the /x switch allows multiline regexps, but in that case all whitespace must be quoted. - Once finished editing the script, run perl on it with the -cw switches to check that it's still a working script! If not, go back and debug it. - If on a RedHat box, you'll probably want to use the etc/rc.d/init.d/logfilterd shell script provided. Put it in /etc/rc.d/init.d, create appropriate symlinks from /etc/rc.d/rc[0-6].d pointing to it, and check that it points to the correct location for the main program, and that it will be run with the correct options (see below). - Fire it up -- either via a '/etc/rc.d/init.d/logfilterd start' command, or directly from the command line, depending on how brave/competent you are. Actually it's probably sensible to use -no-follow until you're happy with the filtering algorithm (see below). The options are: -clean: delete all output files and terminate immediately. This is useful if for example a test run has gone wrong and you want to quickly clear up the crap and start again. -no-follow: To fully understand this option, you need to know some background, which actually you should know anyway: Normally, when logfilterd is run, it first looks for a record of whether it has been previously run (stored in a filename corresponding to the $recent variable), and if so, how much of the input file it previously processed on the last run. This is to ensure that the same bit of the input log file never gets processed twice. In other words, it carries on filtering the input file from where it left off last run (or from the beginning if there is no record of the last run). Once it reaches the end, it sits in an infinite loop very similar in fashion to a 'tail -f'; i.e. every second it checks to see if any new lines have been appended to the input file, and if so, it processes them. The -no-follow option suppresses the 'tail -f'-like behaviour, so that once logfilterd has processed to the end of the file, it terminates rather than waiting for new input data. -no-email: Unsurprisingly, this suppresses the sending of any e-mail. Note that at the end of the day, all the program will do is read through /var/log/messages (or whatever you've changed the input file to), filter out unwanted messages, dump the remainder in /var/log/filtered-messages (or whatever you've changed the main output file to), generate new log files for each service mentioned in the program, and generate e-mail. So if something goes wrong, it shouldn't create any disastrous problems, although obviously standard disclaimers apply at this point! During the testing phase, -clean is your friend. - Check the output files to make sure that /var/log/filtered-messages (or whatever) is a version of /var/log/messages (ditto) filtered in a way which suits you, sir, and that the other output files contain the right stuff. If not, go back and change &filter_line until it does and they do respectively. - Once you're happy it's doing the right stuff, shove it in your bootup sequence without the -no-follow option, and let it do its stuff. - One final note of caution: if you plan to rotate any of the log files it generates using a standard log rotating program (e.g. RedHat's logrotate), you will probably need to stop logfilterd before the rotation and restart it afterwards. That's the lot, I think. If anything is unclear, e-mail me at .