Spam filtering

Warning: The content of this article may be out of date. It was imported from mozilla.org and last updated in 2003.


Seth Spitzer

The Mozilla implement is based on Paul Graham's A Plan for Spam.

Training data is global across all mail accounts within a profile.

Training data is stored in a binary format, in a file named "training.dat".

Initially, the training.dat file is empty (there was discussion of shipping with a default file).

On spam detection, the user can choose to move spam to a special "Junk" folder.

The user can configure junk mail can be automatically purged from the "Junk" folder.

To analyze a message for spam, we need the entire message, not just the headers.

Spam detection happens after filters are run.

White listing happens after filters, but before spam detection.

The purge code is implemented as a search of the "Junk" folder, looking for "old" message that have the proper junk status.

When does purging happens?

Elaborate on the mime changes that were made for spam.

Currently, spam filtering is does not work for news, but it would be possible to add support for this. (there is a bug on it.).

initial state user action table changes
unknown (user can't see this, looks like "not junk") mark as junk add tokens to bad
unknown (user can't see this, looks like "not junk") mark as not junk add tokens to good
not junk mark as junk remove tokens from good, add tokens to bad
not junk mark as not junk no op
junk mark as junk no op
junk mark as not junk remove tokens from bad, add tokens to good