Improvements to spam filtering after a year of tests
It seems I am still having problems with spam on this site now. Instead of getting tons of spam comments posted, I get false positive, that is legitimate content marked as spam.
First off, I wish to apologize for the people that haven't seen their comments published immediately. Next, I will work on improving this and explaining the current situation.
Here are the statistics that the spam module gives me:
- prevented comment spam: 100 (27% aka positives)
- marked comment as spam: 242 (65% aka positives)
- manually marked comment as spam: 3 (0.8%, aka false negatives)
- marked comment as not spam: 17 (5% negatives)
- manually marked comment as not spam: 8 (2.2% false positives)
- total: 370 comments
Another way to put it:
- spam: 93%, 99% of which is automatically detected
- ham: 7%, 32% of which is marked as spam
In other words, the filter works very well at marking spam, so well in fact that it also marks non-spammy comments as spam... Which is a shame because it breaks the interactivity of the site. Basically, you have one chance out of 3 of not seeing your content posted online, which leads people to try to submit it three times, which actually gets them blocked even more.
And this is only the spam module filters: the CAPTCHA module tells me it's blocked 26535 attempts (26 thousand) - amazing.
So overall, the solutions I put in place a year ago work, somehow, but are too aggressive. I have therefore bumped the threshold from 65% to 80%, hopefully that will help with this.
I have also updated the Honeypot module since they have done lots of releases since 1.5, fixed the issue I reported and it's now in use on Drupal.org after some tweaking by our venerable webmaster killes.
If this fails, I will look again at other solutions, like the blogspam plugin that relies on the communnity-run blogspam.net site which powers, amongst other things, Debian Administration and Ikiwiki sites. Back then, I also was considering the Bad Behavior and Block Anonymous links modules.
(Note to self: I have also removed around 150 comments sitting in the spam module moderation queue, that it didn't seem to remove as it should have done.)
well that was quick... I already have a false negative, ie. a spam that was accepted, in this very post (since then removed). It marked a 58% probability so it would have probably gotten through before the change.
What's bad is that the legitimate post of mvc here did get marked as spam (99%??) even though there were no links in it or seemingly illegitimate content...
I guess I'll have to try blogspam...
So it seems all comments posted here by anonymous (or registered!!!) users are marked as spam by the spam module, which seems to be just totally freaking out. So I have disabled and uninstalled its bayesian filter module (again), which may lead to a metric ton of spam getting through. We'll see.
I have also disabled spam.module for authenticated users (yikes!). Hopefully I won't have to use ban & unpublish to kill authenticated users.
If all this still fails, I guess I'll have to give up on spam module totally, which is, anyways, blocking the upgrade. Then I'll look at:
In comparison, spam.module has 3981 installs, last release nov 2011, although there's a dev release from july 2012. I also note that the issue for installing spam.module on d.o has been postponed... indefinitely? Note that cweagans, who has proposed a few patches to spam.module fro drupal.org, has changed his mind and was now suggesting mollom only. But then drupal.org went with honeypot (and nothing else, apparently). I've asked for more information about this...
It seems spamming has been subdued now. I don't see any false positive or negatives since that comment, which is great!
Note that I had the answers I was looking for: d.o is honeypot only, with a fasttoggle extension to mark certain roles as "not a spammer", which is cool.