Current test results

[jlasman]

Currently my private test copy of SpamBlocker3 is running the following blocklists with the following results:
SPAMHAUS sbl.spamhaus.org 5947 88.23%
NJABL dnsbl.njabl.org 25 0.37%
CBL cbl.abuseat.org 9 0.13%
SORBS safe.dnsbl.sorbs.net 381 5.65%
DSBL list.dsbl.org 14 0.21%
SPAMCOP bl.spamcop.net 209 3.10%
SPAMCANNIBAL bl.spamcannibal.org 155 2.30%
==== ======
TOTAL REJECTED BY BLOCKLISTS 6740 99.99%
TOTAL REJECTED FOR OTHER REASONS 36215 80.41%
TOTAL EMAILS DELIVERED 8821 19.59%
TOTAL EMAILS REJECTED OR DELIVERED 45036 100%
Based on these results I've just removed both SORBS and SPAMCOP (since they're both considered controversial) and will have some new results in about 48 hours for comparison purposes.

We're moving along.

Should be done real soon now.

Jeff

[floyd]

Jeff do you have a script that produces that output? I know I could write one myself grep'ing the log but I thought I would cheat if I could.

[tlchost]

Based on these results I've just removed both SORBS and SPAMCOP (since they're both considered controversial) and will have some new results in about 48 hours for comparison purposes.
Jeff

For those of us that are using spamblocker2 ... can we also stop using SORBS and SPAMCOP be commenting out the lines:

# deny using sorbs smtp list
deny message = Email blocked by SORBS - to unblock see http://www.example.com/
# only for domains that do want to be tested against RBLs
domains = +use_rbl_domains
dnslists = dnsbl.sorbs.net=127.0.0.5

# deny using spamcop
deny message = Email blocked by SPAMCOP - to unblock see http://www.example.com/
hosts = !+relay_hosts
domains = +use_rbl_domains
!authenticated = *
dnslists = bl.spamcop.net

Thanks

Thom

[jlasman]

Jeff do you have a script that produces that output? I know I could write one myself grep'ing the log but I thought I would cheat if I could.
Nope; I just do some greps:
# Local Deliveries
grep "T=virtual_localdelivery" mainlog | grep "2007-05-24" --count

# Local Rejected in rejectlog
[root@da12 exim]# grep "2007-05-24" rejectlog --count

# Local Rejected per blocklist
grep BLOCKLIST rejectlog | grep 2007-05-24 --count
and then a bit of cutting/pasting into my kwrite window (windows users can use notepad :) ).

Jeff

[jlasman]

Sure (be sure to restart exim), but results were mixed; see my post on details for 26-Jan-2007 below.

And Thom, don't you just love how much easier exim.conf is to manage then sendmail.cf ?

Jeff

[floyd]

don't you just love how much easier exim.conf is to manage then sendmail.cf

I know I do. Exim can be tough for me at times but at least its more readable text than special symbols.

[jlasman]

The previous results were for 24-May-2007. Here are the results for 26-May-2007:
SPAMHAUS sbl.spamhaus.org 5749 95.61%
NJABL dnsbl.njabl.org 25 0.42%
CBL cbl.abuseat.org 4 0.07%
SORBS safe.dnsbl.sorbs.net NOT IN USE
DSBL list.dsbl.org 51 0.85
SPAMCOP bl.spamcop.net NOT IN USE
SPAMCANNIBAL bl.spamcannibal.org 184 3.06
==== ======
TOTAL REJECTED BY BLOCKLISTS 6013 100.01%
TOTAL REJECTED FOR OTHER REASONS 36206 86.58%
TOTAL EMAILS DELIVERED 6543 13.42
TOTAL EMAILS REJECTED OR DELIVERED 48762 100%
Please feel free to comment on what you think the results show.

Jeff

[tlchost]

Sure (be sure to restart exim), but results were mixed; see my post on details for 26-Jan-2007 below.

And Thom, don't you just love how much easier exim.conf is to manage then sendmail.cf ?

Jeff

Sure do....Sendmail was "interesting" to say the least.

Thom

[jlasman]

Here are the test results for May 27th. This is a bit more full a listing; I had made a few shortcuts in the above listing which made them a bit unspecific.
blocklist catch for 2007-05-27:
======================================================
SPAMHAUS sbl.spamhaus.org 5875 96.58%
NJABL dnsbl.njabl.org 26 0.43%
CBL cbl.abuseat.org 2 0.03%
SORBS safe.dnsbl.sorbs.net NOT IN USE
DSBL list.dsbl.org 32 0.53%
SPAMCOP bl.spamcop.net NOT IN USE
SPAMCANNIBAL bl.spamcannibal.org 148 2.43%
==== ======
TOTAL REJECTED BY BLOCKLISTS 6083 100.00%
======================================================
TOTAL REJECTED BY BLOCKLISTS 6083 13.42%
TOTAL REJECTED FOR OTHER REASONS 39261 86.58%
==== ======
TOTAL REJECTED 45344 100.00%
======================================================
TOTAL REJECTED 45344 87.14%
TOTAL EMAILS DELIVERED 6692 12.86%
======================================================
TOTAL EMAILS REJECTED OR DELIVERED 52036 100.00%
Note that I'm still waiting for comments.

Jeff

[tlchost]

[QUOTE=jlasman;103693]Here are the test results for May 27th. This is a bit more full a listing; I had made a few shortcuts in the above listing which made them a bit unspecific.
blocklist catch for 2007-05-27:
======================================================
SPAMHAUS sbl.spamhaus.org 5875 96.58%
==== ======
======================================================
TOTAL REJECTED BY BLOCKLISTS 6083 13.42%
TOTAL REJECTED FOR OTHER REASONS 39261 86.58%
==== ======
TOTAL REJECTED 45344 100.00%
======================================================
TOTAL REJECTED 45344 87.14%
TOTAL EMAILS DELIVERED 6692 12.86%
======================================================
TOTAL EMAILS REJECTED OR DELIVERED 52036 100.00%
/[QUOTE]
Looks like SPAMHAUS catches most of the blcoklist stuff

What are the "Other Reasons" ? Open Relay? Malformed Headers?

I'm sure it's not proactical, but it would be interesting to know how many of the 6692 delivered messages might be classified as spam by a human.

Amazing, 87% garbage!

Thom

[byteman]

My results show that CBL is the major source for blocking email at about 40%. I have about 15% to 18% email being delivered. I just added SPAMCANNIBAL today to see what that does to the stats. We've been using nolisting for the last month and have noticed about 25% less log volume.

[jlasman]

byteman, did you change the order of the blocklists? Which version of SpamBlocker are you using? I'm using a private beta version of SpamBlocker3. And yes, I also use nolisting on this server. My savings were a lot more than 25%.

Thom, other reasons are as you surmised.

I can't tell you how many of these emails might be classified as spam by a human because this server manages a lot of domains besides my own, but I find approximately 40 or so spams in my inboxes every day; about 10% of my personal incoming email is still spam.

Jeff

[byteman]

No I didn't change the order of the black lists... Spamhaus comes before CBL.

The Spamblocker is version 3.0beta which I obtained from you.

Best,
Tom

[jlasman]

Here are the results for 28-May-2007. They're for Monday, which was a holiday in the U.S. The first test I ran was for 24-May-2007. The results for Tuesday, 29-May-2007 will be a better comparison because they're both non-holiday weekdays.

However, the results for 31-May-2007, which will be the same weekday as well, will be the best comparison; I'll post it on 1-June-2007.

Here are the results for 28-May-2007:
blocklist catch for 2007-05-28:
======================================================
SPAMHAUS sbl.spamhaus.org 6019 96.24%
NJABL dnsbl.njabl.org 22 0.35%
CBL cbl.abuseat.org 2 0.03%
SORBS safe.dnsbl.sorbs.net NOT IN USE
DSBL list.dsbl.org 43 0.69%
SPAMCOP bl.spamcop.net NOT IN USE
SPAMCANNIBAL bl.spamcannibal.org 168 2.69%
==== ======
TOTAL REJECTED BY BLOCKLISTS 6254 100%
======================================================
TOTAL REJECTED BY BLOCKLISTS 6254 12.14%
TOTAL REJECTED FOR OTHER REASONS 45255 87.84%
==== ======
TOTAL REJECTED 51519 100%
======================================================
TOTAL REJECTED 51519 87.48%
TOTAL EMAILS DELIVERED 7376 12.52%
======================================================
TOTAL EMAILS REJECTED OR DELIVERED 58895 100%

While I'll wait until the first to make a decision, I think we'll be putting safe.dnsbl.sorbs.net back into the mix.

Jeff

[ozcorp]

Can I ask a silly question, since zen.spamhaus.org is a combination of the follow, couldn't that be used instead. ( re: http://www.spamhaus.org/zen/ )

ie: sbl.spamhaus.org + dnsbl.njabl.org + cbl.abuseat.org = zen.spamhaus.org

which should result in less queries ?

Here are the results for 28-May-2007. They're for Monday, which was a holiday in the U.S. The first test I ran was for 24-May-2007. The results for Tuesday, 29-May-2007 will be a better comparison because they're both non-holiday weekdays.

However, the results for 31-May-2007, which will be the same weekday as well, will be the best comparison; I'll post it on 1-June-2007.

Here are the results for 28-May-2007:
blocklist catch for 2007-05-28:
======================================================
SPAMHAUS sbl.spamhaus.org 6019 96.24%
NJABL dnsbl.njabl.org 22 0.35%
CBL cbl.abuseat.org 2 0.03%
SORBS safe.dnsbl.sorbs.net NOT IN USE
DSBL list.dsbl.org 43 0.69%
SPAMCOP bl.spamcop.net NOT IN USE
SPAMCANNIBAL bl.spamcannibal.org 168 2.69%
==== ======
TOTAL REJECTED BY BLOCKLISTS 6254 100%
======================================================
TOTAL REJECTED BY BLOCKLISTS 6254 12.14%
TOTAL REJECTED FOR OTHER REASONS 45255 87.84%
==== ======
TOTAL REJECTED 51519 100%
======================================================
TOTAL REJECTED 51519 87.48%
TOTAL EMAILS DELIVERED 7376 12.52%
======================================================
TOTAL EMAILS REJECTED OR DELIVERED 58895 100%

While I'll wait until the first to make a decision, I think we'll be putting safe.dnsbl.sorbs.net back into the mix.

Jeff

[jlasman]

Using zen results in less queries.

Using consolidated lists always results in making less queries.

Remember, this is all DNS-based, so anything already in the local DNS server (because of previous queries) doesn't go any further. If you get 100 hits from the same server, your local DNS resolver will get 100 hits, but the upstream blocklist DNS server only the first.

Jeff

[eger]

I would like to make a suggestion that any lists you plan on removing to just comment out so those who do want to continue using them may simply uncomment them.

[jlasman]

It's an interesting suggestion. I'm not going to take it because it's just too easy to use lists that shouldn't be used in combination for one reason or another. Of course you can make any changes you want.

And you can even convince the folk at DA to not follow my lead but to write their own set, if that's what they want to do; that's the beauty of open-source code.

Jeff

[jlasman]

Here's my results for May 29th:
blocklist catch for 2007-05-29:
======================================================
SPAMHAUS sbl.spamhaus.org 6453 97.26%
NJABL dnsbl.njabl.org 20 0.30%
CBL cbl.abuseat.org 3 0.05%
SORBS safe.dnsbl.sorbs.net NOT IN USE
DSBL list.dsbl.org 26 0.39%
SPAMCOP bl.spamcop.net NOT IN USE
SPAMCANNIBAL bl.spamcannibal.org 133 2.00%
==== ======
TOTAL REJECTED BY BLOCKLISTS 6635 100%
======================================================
TOTAL REJECTED BY BLOCKLISTS 6635 15.26%
TOTAL REJECTED FOR OTHER REASONS 36855 84.74%
==== ======
TOTAL REJECTED 43490 100%
======================================================
TOTAL REJECTED 43490 83.07%
TOTAL EMAILS DELIVERED 8864 16.93
======================================================
TOTAL EMAILS REJECTED OR DELIVERED 52354 100%
Jeff

[eger]

The majority of my spam is blocked by spamcop. What is controversial about spamcop? I actually find their algorithm and the fact that it is based on user reports to be pretty sophisticated and accurate.

[jlasman]

I love SpamCop. But because it's user-report based, it has been known to have some false positives. The good news is that you get OFF the SpamCop last as fast as you get on. We will put it back in before we release SpamBlocker3.

Jeff

[jlasman]

Here are my results for May 30th:
blocklist catch for 2007-05-30:
======================================================
SPAMHAUS sbl.spamhaus.org 7610 96.79%
NJABL dnsbl.njabl.org 17 0.22%
CBL cbl.abuseat.org 11 0.14%
SORBS safe.dnsbl.sorbs.net NOT IN USE
DSBL list.dsbl.org 61 0.78%
SPAMCOP bl.spamcop.net NOT IN USE
SPAMCANNIBAL bl.spamcannibal.org 163 2.07%
==== ======
TOTAL REJECTED BY BLOCKLISTS 7862 100%
======================================================
TOTAL REJECTED BY BLOCKLISTS 7862 18.79%
TOTAL REJECTED FOR OTHER REASONS 33981 81.21%
==== ======
TOTAL REJECTED 41843 100%
======================================================
TOTAL REJECTED 41843 82.08%
TOTAL EMAILS DELIVERED 9136 17.92
======================================================
TOTAL EMAILS REJECTED OR DELIVERED 50979 100%
Let's compare this just past Wednesday with Wednesday of a week ago:
SPAMHAUS sbl.spamhaus.org 5749 95.61%
NJABL dnsbl.njabl.org 25 0.42%
CBL cbl.abuseat.org 4 0.07%
SORBS safe.dnsbl.sorbs.net NOT IN USE
DSBL list.dsbl.org 51 0.85
SPAMCOP bl.spamcop.net NOT IN USE
SPAMCANNIBAL bl.spamcannibal.org 184 3.06
==== ======
TOTAL REJECTED BY BLOCKLISTS 6013 100.01%
TOTAL REJECTED FOR OTHER REASONS 36206 86.58%
TOTAL EMAILS DELIVERED 6543 13.42
TOTAL EMAILS REJECTED OR DELIVERED 48762 100%
Since a week ago we've dropped SORBS and SPAMCOP. Did the other lists pick up the slack? No.

Time to put them back.

I've also just moved SPAMCANNIBAL to the top to see if it makes a huge difference in catch rates.

The earliest we'll be able to see an entire day will be Friday's results, which I can pick up and post some time Saturday.

An interesting experience.

Jeff

[jlasman]

Here are the results for June 1st, 2007. Note that I've re-added both SORBS and SPAMCOP, and moved SPAMCANNIBAL to the top so we can determine how many spams it blocks by itself.

I won't be posting these daily anymore; I think we have a good understanding now of what's going on.

blocklist catch for 2007-06-01
======================================================
SPAMCANNIBAL bl.spamcannibal.org 228 2.64%
SPAMHAUS sbl.spamhaus.org 7653 88.57%
NJABL dnsbl.njabl.org 38 0.44%
CBL cbl.abuseat.org 7 0.08%
SORBS safe.dnsbl.sorbs.net 414 4.79%
DSBL list.dsbl.org 26 0.30%
SPAMCOP bl.spamcop.net 275 3.18%
==== ======
TOTAL REJECTED BY BLOCKLISTS 8641 100%
======================================================
TOTAL REJECTED BY BLOCKLISTS 8641 20.48%
TOTAL REJECTED FOR OTHER REASONS 33559 79.52%
==== ======
TOTAL REJECTED 42200 100%
======================================================
TOTAL REJECTED 42200 82.72%
TOTAL EMAILS DELIVERED 8818 17.28
======================================================
TOTAL EMAILS REJECTED OR DELIVERED 51018 100%

[h2d]

when do you expect the the release date to be ?

[jlasman]

I've just released SpamBlocker2.1.1 which contains some of the features. Now I'm working on SpamBlocker3.

But I never give out a release date anymore. I've learned the hard way ;) .

When I do release it I'll also be releasing a Why SpamBlocker3 document and also a How-To document.

Remember that releasing SpamBlocker requires cooperation between me and DirectAdmin staff, as it may include distribution or other changes by them.

Jeff