Apache Error Log Summarizer

Attached, you will find the Perl source code to my Apache error log summarizer.

This is really something I should have written years ago, but my resolution to make use of my post-SCEA unemployment stint is going to turn up a few useful scripts. In essence, this script is designed to look at only the previous day's worth of Apache log errors. It looks at the pile of errors, removes things like timestamps and client IPs, and then compares only the non-numeric part of the errors. This ensures that 500 lines of "module blah has an error at line X", where X varies due to development/debugging, will be compacted into single lines as much as possible. It then prints a header, spits the top 10 "error forms" to stdout, and exits.


  • /bin/date - used to format the date as egrep will want to see it.
  • /bin/egrep - used to grep for the relevant lines in the error log.

I should note that I've also hardcoded in certain exclusions. There's no good reason for the script to spam you for not having a robots.txt or favicon.ico. Debatably it may be an error or oversight that you don't, but you don't need to see two lines of reports on it every day if you choose not to have one.

Anyway, the script's power comes in once you feed it a list of log files and pipe it to sendmail. An example invocation (as used on my server) is:

ls -1 /home/*/error_log | xargs -n 1 /usr/local/sitetools/reporterrors.pl | /bin/mail -s "Apache complaints: `/bin/date +%D -d 'yesterday'`" you@example.com

You'll then receive a daily emailed summary of the latest things which annoy the various websites on this host. Everything under /home/*/error_log will get summarized and sent to you. My theory is that if I see a few fixable errors every day and fix them, my site's going to run a lot smoother. Now, if you only wanted to do this for certain files, you could always just ls -1 an explicit list of files instead of a wildcard like I used.

Future enhancements to this script will probably involve exclusion lists and pattern matching to identify new trends.

-- SeanNewton - 14 Feb 2011

Topic attachments
I Attachment History Action Size Date Who Comment
Texttxt reporterrors.pl.txt r1 manage 1.1 K 2011-02-14 - 10:00 SeanNewton Perl source to my error_log parser
Topic revision: r1 - 2011-02-14 - SeanNewton
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback