On Feb 16, 2004, at 12:19 am, Grendel wrote:
For each mail caught as spam, the database automatically updates itself
with any new contents of that mail, making it learn as it's catching
mails.
I regard this as risky business - if you fail to reclassify any
mistakes, then the filter will be more likely to make errors in future.
You do state that bmf allows you to reclassify mistakes, but IMO it's
better only to add spam/ham messages to the token database when the
user specifically requests them.
Actually I thought that a bayesian filter is supposed to do exactly the above, ie it learns and autoupdates its database as as time goes by it gets better and better.
Indeed, very much so. But it is important not to teach the filter incorrectly - if it slings an important email in the trash, and you don't notice & correct it, then having the database automatically update itself is a Bad thing (tm), because it will misclassify ALL the words in the email as spammy, not just the handful that tripped it up.
It's safer to place bogus-looking messages in a "spam/suspected" folder, and only update the database when the user consciously checks through them & consciously moves them to a "spam/definite" folder.
Additionally, the larger one's corpus of messages, upon which the filter has been taught, the less effect additional messages have upon the database - once we have established that "viagra" occurs in 1000 spam, but only in 1 ham, adding it as a spammy word doesn't really teach our spam-filter much new. But if a friend were to forward us the joke about the vicar's marquee (in which the punch-line is "viagra"), then training the database on this joke as ham will *double* the hamminess of this particular word.
In order for this to be useful, to teach our filter that "it's ok for `viagra' to appear sometimes in a message, depending on the context of the (other words in the) message", it's helpful if "viagra" does not have an overwhelming number of spammy instances in the database. One should avoid allowing one's database to become so large that it gains too much inertia against the addition of new words or reuse of learned ones in new contexts. (cf random dictionary words in spam).
If we visualise this as a graph (showing effectiveness of the statistical-filter on the vertical axis against the number of messages added to the corpus on the horizontal) then we both agree that the curve rises sharply at first and slows, eventually reaching a plateau. When the angle of the curve becomes difficult to distinguish from the horizontal, it's redundant to add messages that the filter is already classifying correctly - if one adds only messages that the filter has classified incorrectly, then these corrections will be more effective.
I hope this explanation is somewhat helpful. I'm no mathemetician, so I've found it a little difficult to explain. I should also add that these remarks are not from my own experience, but from comments posted to the Bogofilter mailing list by those who claim to have tested them empirically. Your statistical-filter may already operate differently in order to account for these conditions (but I doubt it).
Stroller.
-- [EMAIL PROTECTED] mailing list
