SpamBlocker2.1.1 released

[jlasman]

SpamBlocker version 2.1.1 has been released. It offers a completely reworked and optimized set of blocklists, and a fix (which you may or may not already have on your server) to help with plaintext authorization when using certain email clients.

While SpamBlocker version 2.1.1 is not mandatory, it's strongly suggested, since it removes a nonworking blocklist and will fix authentication issues for some clients.

SpamBlocker version 2.1.1 requires the latest version of exim.pl.

SpamBlocker version 2.1.1 is currently only available for mbox-based systems. The exim.conf.dovecot.patch file available dated 15-December-2005 will NOT convert it to work with Dovecot/Maildir, so if you're running Dovecot/Maildir you should either wait until a new patch file is available, manually patch your new exim.conf file, or update to the SpamBlocker3 file specifically available for your Maildir configuration (either with or without ClamAV).

Remember that the SpamBlocker version 2.1.1 file you down load will not include your changes to point senders of emails detected as false positives to your whitelist page; be sure to search and replace for all instances of example.com before installing the file.

SpamBlocker version 2.1.1 may be found here:
http://files.directadmin.com/services/exim.conf
and also at:
http://www.nobaloney.net/downloads/spamblocker/DirectAdminSpamBlocker2/SpamBlocker.exim.conf.2.1.1-release
The latest exim.pl file may be found here:
http://files.directadmin.com/services/exim.pl
and also at:
http://www.nobaloney.net/downloads/spamblocker/DirectAdminSpamBlocker2/exim.pl

Jeff

[keefe007]

Any word on a new dovecot patch?

[jlasman]

I posted somewhere (obviously not here :( ) that the patch published by DA should work. At least that's what John told me.

Have you tried it?

Jeff

[tristan]

I posted somewhere (obviously not here :( ) that the patch published by DA should work. At least that's what John told me.

Have you tried it?

Jeff

Yes I did and it works perfect, thanks!

[jjma]

The new version of spamblocker works with dovecot?

regards

Jon

[tristan]

The new version of spamblocker works with dovecot?

regards

Jon

Yes, if you use the DirectAdmin patch.

[jjma]

Is this the patch:

http://files.directadmin.com/services/custombuild/exim.conf.dovecot.patch

regards

Jon

[tristan]

Yes just do a:

cd /usr/local/directadmin/customapache
patch -p0 < exim.conf.dovecot.patch

after you updated exim.conf, also see:

http://help.directadmin.com/item.php?id=51

[Duboux]

Took me some time to do this..

Had to update exim first, which I thought I already did earlier, but apparently http://help.directadmin.com/item.php?id=51 doesn't work for me.
But http://help.directadmin.com/item.php?id=126 plus this (http://directadmin.com/forum/showpost.php?p=101551&postcount=16) did the trick.

I downloaded the http://www.nobaloney.net/downloads/spamblocker/DirectAdmin/exim.conf.spamblocker file, edited it into notepad, got it as exim.conf using VI. (just noticed that I could have used DA to edit the exim.conf file -_-; )

Anyway, got it to work, with a lovely help page for non-spammers.
That's a form that sends an email from me to me, and from me to the client that the non-spammer whishes to contact (from me so spamblocker, me and the client knows it's good).
It checks on all the bot-infiltrate-nastyness stuff, like headers, bad email addresses, and even checks with a captcha and the php function: gethostbyname() if the domain matches the ip-range we have.

If my client replies to me with OK, then then I'll add that address to the whitelist and send an email. :)
(I suppose I could automate this as well, but let's see how often it will be used)

[jlasman]

Yes, Duboux; please keep us posted. I never automated because on average I get less than one whitelist request a week.

Jeff

[Duboux]

Okay, since the update, I've seen Exim log lines like these:

2007-08-01 21:52:42 H=([89.240.167.159]) [89.240.167.159] incomplete transaction (connection lost) from <lincomputerfotomet@computerfoto.de>
2007-08-01 21:52:42 unexpected disconnection while reading SMTP command from ([89.240.167.159]) [89.240.167.159]
2007-08-01 21:52:47 H=([89.240.167.159]) [89.240.167.159] incomplete transaction (connection lost) from <linallendorfmet@allendorf.de>
2007-08-01 21:52:47 unexpected disconnection while reading SMTP command from ([89.240.167.159]) [89.240.167.159]
2007-08-01 21:52:51 H=([89.240.167.159]) [89.240.167.159] incomplete transaction (connection lost) from <lincollinmet@collin.de>
2007-08-01 21:52:51 unexpected disconnection while reading SMTP command from ([89.240.167.159]) [89.240.167.159]
2007-08-01 21:52:56 H=([89.240.167.159]) [89.240.167.159] incomplete transaction (connection lost) from <linaiesecmet@aiesec.de>
2007-08-01 21:52:56 unexpected disconnection while reading SMTP command from ([89.240.167.159]) [89.240.167.159]
2007-08-01 22:17:52 H=(yahoo.com) [12.32.39.254] F=<aazlan_cauaqzj@yahoo.com> rejected RCPT <****@****>: Email blocked by SPAMHAUS - to unblock see http://****
2007-08-01 22:17:52 H=(yahoo.com) [12.32.39.254] incomplete transaction (connection lost) from <aazlan_cauaqzj@yahoo.com>
2007-08-01 22:17:52 unexpected disconnection while reading SMTP command from (yahoo.com) [12.32.39.254]

2007-08-01 02:14:14 1IG0i0-0006l6-JD User 0 set for local_delivery transport is on the never_users list
2007-08-01 02:14:14 1IG1qw-0007rw-N8 User 0 set for local_delivery transport is on the never_users list
2007-08-01 04:02:44 1IG3Xw-0000XJ-IH User 0 set for local_delivery transport is on the never_users list
2007-08-01 04:02:45 1IG3Xx-0000Xd-Da User 0 set for local_delivery transport is on the never_users list

2007-08-01 21:49:26 H=cpc3-stkn8-0-0-cust696.midd.cable.ntl.com (GRAHAM-EFNU14F3) [81.96.158.185] F=<palmerq0a8@hotmail.com> rejected RCPT <****@****.com>: Email blocked by SPAMHAUS - to unblock see http://*****
2007-08-01 21:49:28 H=cpc3-stkn8-0-0-cust696.midd.cable.ntl.com (GRAHAM-EFNU14F3.2euu91.org) [81.96.158.185] F=<trickeroge0@hotmail.com> rejected RCPT <****@****.com>: Email blocked by SPAMHAUS - to unblock see http://*****
2007-08-01 21:49:31 H=cpc3-stkn8-0-0-cust696.midd.cable.ntl.com (rr1a4.e9aai.ameritech.net) [81.96.158.185] F=<hirdz4012@hotmail.com> rejected RCPT <****@****.com>: Email blocked by SPAMHAUS - to unblock see http://*****
2007-08-01 22:17:52 H=(yahoo.com) [12.32.39.254] F=<aazlan_cauaqzj@yahoo.com> rejected RCPT <****@****>: Email blocked by SPAMHAUS - to unblock see http://****@****

Seems all spammers, but are the "unexpected disconnection while reading SMTP command" lines in the Main log errors or rejection lines from spamblocker ?

And I see double lines in both EximMain and EximReject on the same actions. Are the rejections supposed to show in the main log as well ? or can they only show in the rejectlog ?

Also I used to receive emails from US NMA, who use different email addresses and domains with every message. But I don't know if they are blocked yet. (hard to see in the logs as the email address is varies constantly). Is there a way to filter on contents too ?

[jlasman]

Seems all spammers, but are the "unexpected disconnection while reading SMTP command" lines in the Main log errors or rejection lines from spamblocker ?
The sender is closing the connection.
And I see double lines in both EximMain and EximReject on the same actions. Are the rejections supposed to show in the main log as well ? or can they only show in the rejectlog ?
I never heard of EximMain or EximReject; do you mean the exim mainlog and the rejectlog? Yes, they'll both show the same information; the purpose of the mainlog is to give you one log where you see everything; the purpose of the rejectlog is to help you focus on just rejected email, for example if you get a whitelist request and you want to look instead of just whitelist.
Also I used to receive emails from US NMA, who use different email addresses and domains with every message. But I don't know if they are blocked yet. (hard to see in the logs as the email address is varies constantly). Is there a way to filter on contents too ?
Yes, but not in SpamBlocker. You can use the mail filter settings from the control panel. I hate those emails too :0 .

Jeff

[Duboux]

Bugger, Spam Cannibal blocks smtp servers o_0

And who's ip are in the mail logs... indeed the smtp servers'

Spam Cannibal blocked an ip that wasn't blocked on it's own, but 2 ip's that looked alike were blocked, so this one got blocked as well:
http://spamcannibal.org/cannibal.cgi search on: 213.75.38.85
hpsmtp-eml20.kpnxchange.com
spam source
see
213.75.38.115
213.75.38.116


Another thing.
A client has 2 client-accounts on that block.
He sends an email from one account to the other.
But get's rejected by SpamCannibal, because his ISP's smtp server (he obviously doesn't use the mail.hisdomain.com for smtp) is marked as spam.
ANd with SpanCannibal the whole ip get's blocked after someone used it to send a spam message :eek:



Some global data:
# grep -c 2007-08-02.*"Email blocked by SPAMCANNIBAL" /var/log/exim/mainlog
31
# grep -c 2007-08-02.*"Email blocked by SPAMHAUS" /var/log/exim/mainlog
2072
# grep -c 2007-08-02.*"Email blocked by LBL" /var/log/exim/mainlog
0
# grep -c 2007-08-02.*"Email blocked by BSHL" /var/log/exim/mainlog
0
# grep -c 2007-08-02.*"Email blocked by BSAL" /var/log/exim/mainlog
0
# grep -c 2007-08-02.*"Email blocked by NJABL" /var/log/exim/mainlog
0
# grep -c 2007-08-02.*"Email blocked by CBL" /var/log/exim/mainlog
0
# grep -c 2007-08-02.*"Email blocked by DSBL" /var/log/exim/mainlog
3
# grep -c 2007-08-02.*"Email blocked by SORBS" /var/log/exim/mainlog
0

[jlasman]

I've removed the SpamCannibal blocklist on my own system and will block it on final releases and next updates.

Jeff

[Duboux]

Is it possible to use more than 1 line in the reply message when an email is blocked ?

Like that line "blocked by SPAMHAUS, see http... for details"
Could it be multiple lines ?

[Duboux]

oi.. when installing this on another box, I got this line in the Exim paniclog:
2007-08-16 03:01:29 non-existent configuration file(s): /config/file.new

What does this mean ?

[jlasman]

Is it possible to use more than 1 line in the reply message when an email is blocked ?

Like that line "blocked by SPAMHAUS, see http... for details"
Could it be multiple lines ?
It's been many years since I visited this issue.

I think you can do it (my guess is you'd add something which would be understood by mail programs as a newline character; you can find that on the 'net). However my understanding is that most error handling systems will only return the first line.

Jeff

[Duboux]

oi.. when installing this on another box, I got this line in the Exim paniclog:


What does this mean ?
I actually get this too on the first box I installed it on..

# exim -C /config/file.new -bV
Exim version 4.67 #1 built 31-Jul-2007 22:10:38
Copyright (c) University of Cambridge 2006
Berkeley DB: Sleepycat Software: Berkeley DB 4.2.52: (September 21, 2004)
Support for: crypteq iconv() Perl OpenSSL move_frozen_messages Content_Scanning Old_Demime
Lookups: lsearch wildlsearch nwildlsearch iplsearch cdb dbm dbmnz
Authenticators: cram_md5 plaintext spa
Routers: accept dnslookup ipliteral manualroute queryprogram redirect
Transports: appendfile/maildir/mailstore/mbx autoreply lmtp pipe smtp
Size of off_t: 8
2007-08-17 00:18:05 non-existent configuration file(s): /config/file.new


:(

[jlasman]

What were you installing when you got this? Exim? DirectAdmin?

On what OS Distribution?

Were you installing from a DirectAdmin supplied RPM, or some other kind of package? Or from source?

Why were you running this line:
# exim -C /config/filenew -bV
Where did you get the instructions to run that?

Jeff

[Duboux]

What were you installing when you got this? Exim? DirectAdmin?

On what OS Distribution?

Were you installing from a DirectAdmin supplied RPM, or some other kind of package? Or from source?

Why were you running this line:
# exim -C /config/filenew -bV
Where did you get the instructions to run that?

Jeff
I was updating exim and installing SpamBlocker.
OS = FC3

I started from the rpm:
# wget http://files.directadmin.com/services/da_exim-4.67-2.src.rpm

That # exim -C /config/filenew -bV line, I ran to check, as was advised by your SpamBlocker txt file.

[Duboux]

Another question:
SpamBlocker doesn't seem to check emails that go to a Catch-All E-Mail.
Is this correct ?

[jlasman]

That # exim -C /config/filenew -bV line, I ran to check, as was advised by your SpamBlocker txt file.
You need to replace /config/filenew with the path to the new exim.conf file you're installing, to test it for syntax, before you install it.

For example, if you uploaded the spamblocker file to /home/admin, first cd to the /home/admin directory, then edit the file and change it according to instructions and rename it to exim.conf.

Then run:
# exim -C /home/admin/exim.conf -bV
Then, when it passes, copy it over the working copy:
cp /home/admin/exim.conf /etc/exim.conf
and restart exim.

Remember the # mark at the beginning of each line is simply to remind you that you run the command as root; you do NOT type it in.

Jeff

[Duboux]

Aaaaah :)

# whereis exim.conf
exim: /usr/sbin/exim /etc/exim.conf /etc/exim.cert /etc/exim.key /etc/exim.pl /usr/share/man/man8/exim.8.gz
[root@da ~]# exim -C /etc/exim.conf -bV
Exim version 4.67 #1 built 31-Jul-2007 22:10:38
Copyright (c) University of Cambridge 2006
Berkeley DB: Sleepycat Software: Berkeley DB 4.2.52: (September 21, 2004)
Support for: crypteq iconv() Perl OpenSSL move_frozen_messages Content_Scanning Old_Demime
Lookups: lsearch wildlsearch nwildlsearch iplsearch cdb dbm dbmnz
Authenticators: cram_md5 plaintext spa
Routers: accept dnslookup ipliteral manualroute queryprogram redirect
Transports: appendfile/maildir/mailstore/mbx autoreply lmtp pipe smtp
Size of off_t: 8
Configuration file is /etc/exim.conf

Looks okay (as in: no errors), thanx ;)
Perhaps it would be handy to enrichen the SpamBlocker help/instruct files with this info :)


Could you also look into my post #21 please ?

[jlasman]

Perhaps it would be handy to enrichen the SpamBlocker help/instruct files with this info :)
What help/instruction files ;) ?

The README files are designed for experienced system administrators. Unfortunately. Generally my responses are as well :( .

After SpamBlocker3 is released I'll try to take some time to write better instructions; perhaps a manpage.
Could you also look into my post #21 please ?
I will, and I'll respond.

Jeff

[jlasman]

SpamBlocker doesn't seem to check emails that go to a Catch-All E-Mail.
First of all, let me say that catchall email just doesn't work anymore on today's Internet; just too many spammers sending too much spam to nonexistent address; too many so-called dictionary attacks against domains. Getting rid of catchall accounts will probably lower the mail traffic on your server to less than 10% of what it is now. Really.
Is this correct ?
SpamBlocker works based on sender reputation; it checks servers to see if they're in any of several blocklists. It doesn't even consider the recipient...

except ...

that it checks first to see if the recipient is whitelisted in one of several whitelists, and if it is, the SpamBlocker checking is bypassed.

So you should check to make sure the target of your catchall email isn't in any of the whitelists:
# grep USERNAME /etc/virtual/whitelist_*
where you should replace USERNAME with the username you're looking for.

Jeff

[Duboux]

That is strange...

I hear your argument on the catchall. I already turned it off..

But what I don't understand is that if I turn it on, I receive loads of spam, which would most likely be blocked if it was sent directly to my existing email box =/
It gave me the idea that somehow this catch-all option bypasses Spamblocker..

[mattb]

I turned off catchall on all domains over the last 6 months.

It means less work for your SMTP server as well.. as the spammer gets a reject immediately and isn't sending tonnes of stuff down your line.

I don't think it bypasses Spamblocker, it just loads up the server... also SpamBlocker whilst good isn't a 100% solution... and as such you'll still see email sit in the catch-all accounts.

Just due to it's nature it gets hit much harder (review ya logs) and as such you're likely to see it grow in size rapidly.

[Duboux]

k, so say a spam message that didn't get caught, would go through..
If such a message would be sent to all the non-existing email addresses, with catch-all on, there would be like 50 of those spam mails in ones box..
k, I get that.


New thing though.. why would an email with the word "viagra" get through SpamBlocker..
Return-path: <efvhyr@bodyclockfrance.com>
Envelope-to: me@myemail.com
Delivery-date: Wed, 22 Aug 2007 14:09:37 +0200
Received: from fw1.sanomabp.hu ([81.0.89.154])
by my.box.host with esmtp (Exim 4.67)
(envelope-from <efvhyr@bodyclockfrance.com>)
id 1INp1l-0008Ll-Op
for me@myemail.com; Wed, 22 Aug 2007 14:09:37 +0200
Received: from [81.0.89.154] by corpspool.clara.net; Wed, 22 Aug 2007 14:12:04 +0000
Message-ID: <01c7e4c6$6ab569c0$9a590051@efvhyr>
From: "Roger Adams"
To: <me@myemail.com>
Subject: RE: Ever counted how much you spend for meds?
Date: Wed, 22 Aug 2007 14:12:04 +0000
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_NextPart_000_0007_01C7E4C6.6AB569C0"
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 4.72.2106.4
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.2106.4
X-Antivirus-Scanner: Clean mail though you should still use an Antivirus



Viagra Pro (SALE 50%)



- Increase S*e*x Drive
- Boost Sexual Performance
- Fuller &amp; Harder Erections
- Increase Stamina & Endurance
- Quicker Recharges
- Improved sildenafil citrate formula
- Works in less than 15 minutes
- Best p*r*i**ce on the internet


BUY NOW

[floyd]

why would an email with the word "viagra" get through SpamBlocker

Because Spamblocker does not check content. See Jeff's previous post.

SpamBlocker works based on sender reputation; it checks servers to see if they're in any of several blocklists.

[Duboux]

Ah right.. sorry abt that.

So I'd need to have SpamAssassin behind it to have it more solid ?

(shame, lol)



And is there a way to report our spam-mails to all those used rbl's in one go ?

I could block the senders email address or the used smtp-servers ip, but I don't know what to do, since all of them could have been abused/missused by the actual bad-guy

[floyd]

There is a difference between spam filters and spam blockers. A spam blocker will not accept the email from the sender server and therefore bounces it back to the server that sent it. A spam filter accepts the email and then examines it and then does something with it. But because it has already accepted the email it cannot be bounced back to the sending server.

Now here is my opinion. Spam filters should never be used because many of them catch legitimate email and put it in a spam box along with the spam. Many people never look at there spam box and therefore they never know about mail they should have received. At the same time the sender never knows that the person never got it and so they think the recipient is just ignoring them.

I suspend people weekly because of overdue invoices. The customer never got the invoice notice because of a spam filter. Suddenly their business is shut down because of a spam filter. Had they been using a spam blocker instead we would at least know that they never got the invoice notice and therefore tried other means of contacting them before shutting them down. Legitimate business email often looks like spam because of the nature of the email.

Spam filters also increase the load on the server.

All email should either be accepted of rejected but never filtered.

[Duboux]

Good point floyd :)

So then users should be able to mark emails as spam (I don't bother asking them abt hosts, ip's senders, etc.. They just see it as an email and it's not their job to go techie (in their opinion ;))

So do u think users should be able to forward or mark their spam and we hosters should add that spam (ip/sender/host/w/e) to the blocklists ?

[floyd]

I think you are still missing the point of Spamblocker.

and we hosters should add that spam (ip/sender/host/w/e) to the blocklists

You don't add them to a blocklist unless you decided to maintain your own blocklist. Spamblocker uses publicly available blocklists that are already out there.

If a user gets spam they themselves can report it to blocklists and then the blocklists admins can decide if they want to add them.

If a user decides to use Spamassassin then they cannot hold you responsible for not getting important email and they should be informed of such.

[Duboux]

Ow, I thought I would add them to one of these files:
/etc/virtual/bad_sender_hosts
/etc/virtual/blacklist_domains
/etc/virtual/blacklist_senders

[jlasman]

why would an email with the word "viagra" get through SpamBlocker..
Because SpamBlocker doesn't care about content. Spam is NEVER about content; it's about consent.

So SpamBlocker blocks on reptuation of the sending server.

True, we block based on content, but we have to be careful when we do; while one user might consider every message with the word viagra in it to be spam, another may think it very important to his marital happiness ;) .

If you want to block on content you can do that with DirectAdmin Spam Filters and with SpamAssassin.

Jeff

[tom3000]

Is there a way to stop spamblocker to send emails back to spammers. It means avoid to answer to the sender the deny message. (email block by spmahaus, to unblock see http://...). What I am looking for is that spamhaus blocks and delete the spam emails but does not answer back, it means much less work for exim (half of work) even if this means that some emails could be "lost without notice". I´ve haven´t seen a thread about this. Sory if this is a naif question.

[jlasman]

No, it's actually more work for exim to discard the message; to do that it has to accept it and then delete it. It's much more efficient for it to just send the reason and then drop the connection.

You could of course leave the message section blank, but if you did the only thing you'd save is one packet, and there'd still be a deny message; the deny message actually comes from the sending server after your exim server shuts the connection.

Jeff

[chasjs]

Is spamblocker rejecting the message before or after the Data command?

[jlasman]

Before, as it should :).

Jeff

[enginaar]

I'm using spamblocker 2.1.1 and my spam messages don't go to spam folder. they are mailed to me with ***SPAM*** on their subject although i checked redirect to spam folder from da panel.

[jlasman]

This is a SpamAssassin issue, not a SpamBlocker issue. SpamBlocker does send the email through SpamAssassin.

There may be a problem in how SpamAssassin is implemented, as this is being reported a bit recently.

John, any ideas?

Jeff

[DirectAdmin Support]

Hello,

Yeah, I'm putting my money on the From tags being set to <> so that the filters are ignored on the line:

if error_message then finish endif

in /etc/virtual/domain.com/filter

There is a thread on it here:
http://www.directadmin.com/forum/showthread.php?t=26264&highlight=error_message

The solution at the moment is to remove that line from the filter file.

If you want to remove it globally, edit:
/usr/local/directadmin/data/templates/filter_base

and remove the line.

John

[enginaar]

Thanks. I will search deeper before I post next time.