Re: [Evolution] Built-in spam filtering?

[Brett Johnson <brett hp com>]

On Thu, 2003-01-02 at 16:46, Jim Frost wrote:
> On Thu, 2003-01-02 at 18:13, guenther wrote:
> > If you don't control the server, get your own. ;)
>
> I do have my own for personal use.  Cannot have my own for corporate
> use, don't have the choice.

What, is there a corporate policy preventing you from running your own
(local) email server?  How would they even know?

[..]
> I want statistical, thanks.

Me too.  Bogofilter rocks!

[...]
> What pop3 server?  Everything I use is imap, which is one reason that
> it's hard to use a lot of the existing bayesian tools.

Eh?  What does the protocol have to do with which tools you use?

[...]
> So, running my own server does work with personal mail although, as I
> said, it's not a straightforward drop-in to put in most of the server
> based filters.  Finding the time to figure out what I need to do has
> been problematic.  And, even with that done, I still have to deal with
> the corporate spam residing on servers I do not and cannot control.

FWIW, I'll describe the system I have set up to use bogofilter (which,
after a month or so of training, has achieved pretty close to 99.9%
detection, and I've yet to get a false positive):

      * First, I'm running the UW imap server locally (which sucks, as
        you said, but it's really easy...).
      * I use fetchmail to grab mail from my various email sources (like
        the stupid corporate mandated M$ exchange server).
      * I use procmail to filter mail into various folders (like mailing
        lists, for example), and also to invoke bogofilter for filtering
        spam.
      * I use evolution (mostly) to read my mail, so I've also got some
        filters set up in evolution that I use to train bogofilter.
      * I have also set up a couple of email aliases on the machine in
        question so that, when I'm not using evolution, I still have a
        way to train bogofilter on new spams (by forwarding the target
        email to the given alias).

The only part of this setup that's less than trivially easy is the
procmail setup, so I'll explain that here:

Here are the procmail rules that I use (put in ~/.procmailrc)
        

        # "bogospam" and "bogoham" are email aliases I set up for the sole
        # purpose of training bogofilter:
        
        # Anything sent to the "bogospam" mail alias just goes into the
        # bogofilter database as spam.
        :0HB
        * ^TO.*bogospam
        | bogofilter -S
        
        # Anything sent to the "bogoham" mail alias just goes into the
        # bogofilter database as legit mail.
        :0HB
        * ^TO.*bogoham
        | bogofilter -H
        
        # Here's where we let bogofilter do its work...
        :0HB
        * ? bogofilter
        {
            # If bogofilter thinks this message is spam, reinforce that
            # conclusion by adding it to the spam database.
            :0HBc
            | bogofilter -s
        
            # Then file it away in my spam folder, for later perusal and deletion.
            :0
            $AUTOFILED/Spam
        }
        
        # By default, assume that everything else is not spam (and reinforce
        # the assumption by adding it to the non-spam database).
        :0EHBc
        | bogofilter -n
        
The evolution filters I use are trivial, although perhaps not obvious:

      * First, I defined a couple of labels I can use to label target
        messages ("spam" and "not spam")
      * Then, when I get a spam that wasn't detected (or if I ever get a
        false positive), I label the message in question appropriately,
        and re-run the evolution filters on the folder that contains it.
      * I have two filters defined ("bogofilter spam" and "bogofilter
        not spam".
      * The "spam" filter just has two criteria: [label is "spam"] and
        [pipe message to shell command "bogofilter -S"]
      * The "not spam" filter has: [label is "not spam"] and [pipe
        message to shell command "bogofilter -H"] criteria.

> Probably 99% of Evolution's users don't run their own servers and would
> benefit from this kind of thing even if you personally don't, and a hell
> of a lot of people would prefer not to be screwing around with procmail
> just to get rid of spam.

That's true.  I also think that a lot of people would prefer to not be
screwing around with spam at all (I know that's my preference)!

Cheers!
-- 
Brett Johnson <brett hp com>
   -  i  n  v  e  n  t  -