[Logwatch] Feedback wanted: reworked filter for amavisd-new

logwatch at mikecappella.com logwatch at mikecappella.com
Mon Apr 17 13:37:30 MST 2006

[ posted in logwatch and logwatch-devel ]


I've been reworking the amavis reporting filter, and would like to get some
feedback.  Once I've implemented the feedback, I'll submit a patch.  My
goals were to:

  - Facilitate ease of spotting problem senders
  - Provide increasing detail as required, at runtime 
  - Provide more summarized information
  - Provide amavis startup summary when available and requested
  - Reduce the amount of manual scanning/weeding through mail logs

So far, I like the approach, and have already found several additional
spam/virus-sending IPs to block, as the summarization more easily calls out
problematic IPs.

Below are the basic changes:

  - Multiple levels of reporting
    Levels <5, 5, 6, 7, 8, 10 provide increasing more levels of detail.
    Try them all!  Code was refactored to provide multiple levels of

  - Formatting more easily shows total and sub-totals
    Counts have been moved to the left, and are aligned by depth.  Each
level contains
    totals of children sub-levels.  This removes the excessive Time(s), at
the right
    of each line, which I find difficult to scan for obvious offenders.

  - The category name of "virus" was changed to the broader category name of
    since malware includes both viral payloads and phishing email.

  - Formatting more focused on envelope sender's IP
    Malware and spam typically contain bogus sender information.  The
envelope sender IP
    is more useful in determining when IP blocks should be instituted.

  - Additional reporting when amavis' log_level > 1
    Amavis reports more information with higher log levels, which is useful
for various
    reporting scripts.  More details are summarized when available.  For
    counts and types of email or MIME violations are summarized, amavis
startup info
    is summarized (Detail 5, 10), etc.  

  - More counts available in summary

  - Percentages of passed/spam/malware vs. scanned are included in summary

  - Capture additional output lines that were being caught in the Other
Items report.

Known Problems
  - There is some intentional inconsistency in which fields are used as the
first sort
    field between categories of reports. I'm soliciting feedback on what
users find most
    useful.  See Future Consideration, but please reply with feedback.
  - I've intentionally pushed email address far to the right, to reduce
clutter, and 
    provide more focus on IP addresses, etc.  I'd like to get feedback here
Future Consideration
  - Support customization of key sort fields used for reports (eg. sender
IP, sender email,
    recipient email, type of malware, etc.)
  - Provide multiple reports per call to the filter
  - Create amavis+postfix filter that better evaluates and summarizes email

To test out the filter, you can simply place the attached amavis script
(after decompressing) in /etc/logwatch/scripts/services/amavis (your
distro/setup may be different) and run logwatch as usual.  I've used the
filter on 7.2.1 and 7.3, and it will likely work for some older versions,
but I don't know how far back.  I've been testing with, for example:

   logwatch --print --service amavis --range today --detail 5
   logwatch --print --service amavis --range 'between 4/13 and 4/17'
--detail 6
   logwatch --print --service amavis --range yesterday --detail 7
   logwatch --print --service amavis --range today --detail 8
   logwatch --print --service amavis --range yesterday --detail 10

Feedback (onlist preferred) welcome and encouraged.  If you have log lines
that are not captured or processed correctly, please send me a copy of the
line in some form of archive so that whitespace is not altered, and I'll
update the script.  Either alter private information, or leave it as is, and
rest assured your data will remain confidential.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: amavis-7.3-update.gz
Type: application/x-gzip
Size: 4571 bytes
Desc: not available
Url : http://ip70-176-100-107.ph.ph.cox.net/pipermail/logwatch/attachments/20060417/76177093/amavis-7.3-update.gz

More information about the Logwatch mailing list