Spamassassin & Email Aliases

[Vibe]

Hi everyone,

Just a basic question regarding Spamassassin with user accounts vs. alias email addresses.

While logged in as root (ssh) under /home/username/ I see that each domain configured with spamassassin has a user_spam directory that holds each email accounts' spam file (e.g. "spam box"). I also notice that alias accounts are treated the same in that there is also a spam file for each alias.

If 3 users share an alias will spam assassin filter all spam to the alias spam file alone, or will the spam be filtered into each users email file as well (mbox format here)?

*OR* does the filtering stop at the alias file alone?

The problem that I can see occurring is that if users share a common alias, any spam filtered into the alias file alone may give rise to non-spam emails not being delivered, which will not be available to the user (can't login with an alias :-).

Does anyone encounter issues with this?

Thanks for sharing any information!

Michael

[Vibe]

Just wanted to post an update. From what I have read, Spamassassin is supposed to overlook email aliases on the server, and only act upon actual email accounts. However, on my system Spamassassin is creating spam files for email accounts AND aliases in /home/username/.spamassassin/user_spam/.

Does anyone know what interaction takes place with DirectAdmin/alias configuration and Spamassassin?

This is a problem in that there is no way to check whatever is tagged as spam at the alias level. Email/spam sent to aliases is not making it to user email accounts.

Any help would be greatly appreciated - thanks in advance!

Michael

[Dualdot]

I have the very same problem. I have a domain example.com, and a domainalias (or pointer) domain2.com. Now /home/admin/.spamassassin/user_spam is getting filled with files like info@domain2.com, while these spammessages SHOULD be sent to info@example.com.

Not only can I not check if there are any false positives, but there is the bigger problem of these spamfolders getting bigger and bigger, and there is no easy way of cleaning it up from time to time (yes it can be done from within DA, but I prefer to do it from my mailclient of course).

[Vibe]

Hi Dualdot -

As an update John (DA) suggested the following:

*******************
edit your /etc/exim.conf, and add 1 extra condition line:

spamcheck_director:
driver = accept
condition = "${if and { \
{!def:h_X-Spam-Flag:} \
{!eq {$received_protocol}{spam-scanned}} \
{!eq {$received_protocol}{local}} \
{exists{/home/${lookup{$domain}lsearch{/etc/virtual/domainowners}{$value}}/.spamassassin/user_prefs}} \
{if eq {}{${if exists{/etc/virtual/${domain}/passwd}{${lookup{$local_part}lsearch{/etc/virtual/${domain}/passwd}}}}}{0}{1}}\
} {1}{0}}"
retry_use_local_part
transport = spamcheck
no_verify

This will only run Spamassassin for the pop users, thus won't add spamboxes for non accounts.

*******************

I tried adding the above (bold line - no carriage return after "if eq{}{${if") to exim.conf - however, all email regardless of recipient (alias/forwarder or actual account) was no longer being scanned by spamassassin.

This can be a big issue in that email directed to an alias/forworder will not be scanned *AND* will not be available for clients/users to review for false positives (can't login to an alias :( ).

[Dualdot]

Uhm okay so you're saying that adding that line to exim.conf only makes it worse? :)

[Vibe]

Exactly...on our system it doesn't work. I am not certain if it is a syntax issue or ??? I just posted the info to see if anyone else has an idea. Hopefully someone can shed some light on this.

Ahhh...Dualdot...I misread your original post - my mistake.

The issue I am having revolves around email aliases and not domain. For example info@domain.com being an email alias that is directed to user1@domain.com and user2@domain.com.

On our system, Spamassassin treats info@domain.com as an actual email account by creating a spambox in /home/user/.spamassassin/user_spam. This won't work as there is no way to check the spam box for an alias other than through SSH. Fun stuff for sure :).

[nobaloney]

You may have copied something incorrectly; Since John gave you the line I'm sure he won't mind if you ask him to double-check it for you :) .

Jeff

[Dualdot]

Vibe: the aliasses may be different, but the problem is the same :) I also get a spambox for an emailaddress that doesn't exist (an alias). Have you asked John yet?

[Vibe]

Hi Jeff,

I was thinking I copied something incorrectly too :). I tried playing with the code a bit without success. What is strange is that I have literally spent hours looking for some type of documentation regarding SpamAssassin and the issue of alias email addresses and have not found anything on newsgroups, google, the SpamAssassin blog or the installation/maintenance documentation - very weird!

Because of this I am assuming that EVERYONE has the same issue but doesn't realize it.

Dualdot - I did send John a second email yesterday to let him know that the code he sent doesn't appear to be working. Hopefully he will be able to get back to me with some ideas. I don't want to be a "pain in the rear" as I am sure he is very busy. I will definitely post more information as I learn/hear more.

Thank you both for your input!

Michael

[Dualdot]

Yeah but we ARE paying customers with a problem :)

I'm looking forward for new info.

[different]

Anyone find the solution?

how to make alises work?

and how to make email not exist won't be scaned..?

I mean if I use the :blackhole: for domain, all the email account doesn't exist will be scaned by spamassassin, and create many spam file.

[DirectAdmin Support]

Hello,

I think I've found a cleaner version of it.. that actually works ;)
spamcheck_director:
driver = accept
condition = "${if and { \
{!def:h_X-Spam-Flag:} \
{!eq {$received_protocol}{spam-scanned}} \
{!eq {$received_protocol}{local}} \
{exists{/home/${lookup{$domain}lsearch{/etc/virtual/domainowners}{$value}}/.spamassassin/user_prefs}} \
{exists{/etc/virtual/${domain}/passwd}} \
{!eq {}{${lookup{$local_part}lsearch{/etc/virtual/${domain}/passwd}}}} \
} {1}{0}}"
retry_use_local_part
transport = spamcheck
no_verifyBut in testing I quickly noticed that this meant the system account doesn't get checked, only the pop account will be scanned. An even messier "or" would need to be inserted in that last "eq" statment. pseudo code:

or {eq user is system from /etc/virtual/domainowners}{!eq {}lookup user from passwd}

So if you only want to scan you pop accounts, the above code is perfect. It doesn't scan the system user account.. more needs to be added, but... it would be very ugly to try and do.

John

[Vibe]

JOHN! You are THE MAN!!!

This works PERFECTLY. I just tested on our server and can verify email accounts are being scanned properly AND alias email addresses are being scanned. Better yet, any email alias/forwarder addressed to a "local" user is scanned and if tagged, placed in their email account's spam box. Also, alias addresses that forward to remote accounts are NOT scanned - so no load on the server!

Beautiful!

Thank you for all of your help!

Michael

[nobaloney]

John, should I put this into the exim.conf masters I maintain?

Thanks.

Jeff

[DirectAdmin Support]

well.. it's not "complete"... it's very good at what it does. But what it doesn't do:
- does not scan the catch-all
- does not scan the main system pop account.
- does not scan forwarders/aliases

So it will be great for your pop users, and you won't get any mystery inboxes being created, but your catch-all and system account will receive unfiltered spam. So It's still not the best solution, but works great for pop accounts. I'd probably hold off for the time being (unless you dont think the above are that big of a deal)

I guess another spamcheck_director could be added as "spamcheck_director_2".. to handle the extra cases (system username and aliases), so that we don't end up with too much of a mess in the above.. might solve those issues.

The catch-all would be the only one that can't really be done, because it's not the exim.conf, it's the filter. The filter itself doesn't know that it's a non-existant mailbox, hence the original problem. And the filter syntax isn't flexible enough to figure it out.

John

[nobaloney]

Thanks for your points. Well taken.

I'll wait for you to add the spamcheck_director_2.

Except everyone on the exim lists tells me it's not a director. It's really a router-transport set. Or so they're telling me.

I understand the catchall problem, and can probably fix it even in the filter, though I don't have time to study it right now.

I'm also being told that we're doing this inefficiently; that we should do it at SMTP DATA time.

I'm looking for a way to do it then, and add headers; so then we could do everything later with filters or more complex router-transport sets.

Unfortunately it takes a while to get a handle on it and everyone wants it now :) .

Jeff

[Vibe]

Hmmm...this is very interesting. On our system we have been performing tests using the changes John shared.

For our configuration Spamassassin *is* scanning aliases/forwarders, and if tagged as spam, depositing the email to the corresponding user - hence my extreme enthusiasm.

I will perform some additional tests to see if the system accounts are being scanned and let everyone know. Thanks again John!

Michael

[DirectAdmin Support]

Hello,

Is the forwarder forwarding to a regular pop account?
If so, it's the pop account that's being scanned after the email is forwarded (forwarders themselves don't have inboxes). Or else, there is some magic/fluke in our favor, which is totally fine by me ;)

John

[different]

It still have problem with setting domain_pointer, it will create two spam file in user_spam folder..

[Vibe]

Hi John,

You exactly right. The forwarder/alias is directed to a *local* pop email account. I believe, like you mentioned, the email is then being scanned when it arrives to the user account. This is *exactly* what I was looking for. Before, a spam box was being created in the name of the forwarder address which prevented us from viewing possible "false positives" through SquirrelMail.

Now any email directed to a local user (be it their local pop address or through a forwarder to a local pop account) is being scanned and all spam is being placed in their "user_spam" directory for each domain.

The "side-effect" that also takes place is that forwarders to external email addresses are *not* being scanned - which is what I wanted also.

I will post again to see if the admin accounts are being scanned properly when I have the opportunity.

Thanks again!

[diradmin]

It seems I found solution at last.
On our servers works fine.
Please, check it on your servers.... may be I something dismissed


# Spam Assassin
spamcheck_director:
driver = accept
condition = "${if and { \
{!def:h_X-Spam-Flag:} \
{!eq {$received_protocol}{spam-scanned}} \
{!eq {$received_protocol}{local}} \
{exists{/home/${lookup{$domain}lsearch{/etc/virtual/domainowners}{$value}}/.spamassassin/user_prefs}} \
{exists{/etc/virtual/${domain}/passwd}} \
{ \
or \
{ \
{!eq {}{${lookup{$local_part}lsearch{/etc/virtual/${domain}/passwd}}}} \
{eq {$local_part} {${lookup{$domain}lsearch{/etc/virtual/domainowners}}}} \
} \
} \
} {1}{0}}"
retry_use_local_part
transport = spamcheck
no_verify


Its scann spam of main system account too.

[Dualdot]

Has anyone tested above code? Sorry for my late reply, but I never got email notifications after my previous post... :(

[harro]

Has this 'fix' been implemented in the current exim.conf? OR should we keep an eye out for phantom mailboxes and if they get out of hand add the modifications to the config file ourselves?

[diradmin]

It is not implemented in the current exim.conf, because it is not scan catch-all for default account:

username: username
here /etc/virtual/username/aliases

Suggest to user not use username: username
Say them to use something like
username: virtual_pop_account

and all be fine.

We tested it on our several servers, and no problem with it..

[nobaloney]

I'm not sure what you mean because a linux username is a real pop account.

Or do you mean the forward should be to a virtual email account?

Thanks.

Jeff

[DirectAdmin Support]

It's because it won't scan the catch-all address.

But.. if the catch-all is forwarding TO a real address, then the email probably would get scanned in the end, so there might not be any problems at all with the above code ;)

Once we get a few "yes it works" results, then I'll be adding it to the exim.conf ;)

John

[nobaloney]

Just tell me when, John, so I can add it to mine as well.

I've cancelled an out-of-town trip so I can finish some installs and then work on the next version of exim.conf (the one which will block viruses).

:)

Jeff

[diradmin]

Originally posted by jlasman

Or do you mean the forward should be to a virtual email account?

Jeff

Yes, as John explained

[different]

I had test the setting ...

the user_pref doesn't work on forwarding setting, if I want set the spam for forwarding mail, I had to set the local.cf, but that will effect all the incoming mail.

[Dualdot]

Well this seems to send spam for aliasses to the forwarded-to address, like I needed it to. User preferences are not a real big concern for me.

I did notice something else however, in combination with the latest spamblocker exim.conf: non-existent addresses (like lalala@example.com) still get checked by all dns-lists and so on (spamhaus, orbs, etc). Why isn't it rejected right away because these is no such address?

But that is not a real problem. The following however is: let's say I have the domain example1.com, and example2.com is a domain pointer for it. Spam sent to info@example2.com now gets redirected to the info@example1.com spam folder, like I wanted it to. So this works. However, when spam gets sent to non-existent@example2.com it ALSO gets delivered to the spam folder on info@example1.com... why? It is a non-existent address and should be rejected by exim...

[DirectAdmin Support]

Hello,

I've just spent the day trying to come up with another solution. My reasoning was that everything should be able to be scanned.. but the problem is with the filter in that it doesn't know what type of emails it's dealing with, which is why it's blindly saving things.

The idea is simple: tell the filter what type of email it's dealing with, so it can more intelligently figure out where it's supposed to go. The only way I was able to manage this was by adding extra headers to the email before spamd time, so that the filters could see them (headers added at domain filter time are not seen by the filter).

This is just another possibility, and is likely still very rough, but provides another option for admin's should they need it.

1) Make your spamcheck director look like this
# Spam Assassin
spamcheck_director:
driver = accept
condition = "${if and { \
{!def:h_X-Spam-Flag:} \
{!eq {$received_protocol}{spam-scanned}} \
{!eq {$received_protocol}{local}} \
{exists{/home/${lookup{$domain}lsearch{/etc/virtual/domainowners}{$value}}/.spamassassin/user_prefs}} \
} {1}{0}}"
headers_add = "${if !eq {} {${lookup{$local_part}lsearch{/etc/virtual/${domain}/aliases}}} {X-is-alias: yes}{X-is-alias: no}}\n\
${if !eq {} {${lookup{$local_part}lsearch{/etc/virtual/${domain}/passwd}}} {X-is-pop: yes}{X-is-pop: no}}"
retry_use_local_part
transport = spamcheck
no_verify2)You'll have to have your user_prefs file add 2 extra lines. To make them globally added, add these lines to your /usr/local/directadmin/data/templates/user_prefs file:
report_safe_copy_headers X-is-pop
report_safe_copy_headers X-is-aliasor just to your own local /home/username/.spamassassin/user_prefs file for testing with 1 user.

3) Change your filter to use the new data. Your /usr/local/directadmin/data/templates/filter_userspamfolder will have to look like this
if
$h_X-Spam-Status: contains "Yes,"
then
|*if DOVECOT="yes"|
if "$h_X-is-pop:" is "yes"
then
save |HOME|/imap/$domain/$local_part/Maildir/.INBOX.spam/new/ 660
elif $local_part is "|USERNAME|" or "$h_X-is-alias:" is "no"
then
save |HOME|/Maildir/.INBOX.spam/new/ 660
endif
|*else|
if "$h_X-is-pop:" is "yes"
then
save |HOME|/.spamassassin/user_spam/$local_part@$domain 660
elif $local_part is "|USERNAME|" or "$h_X-is-alias:" is "no"
then
save |HOME|/.spamassassin/spam 660
endif
|*endif|
finish
endifOf course, to make any template permanent, be sure to copy it to the custom folder first, and work with it from there.

This may not be the final answer to the problem, but will hopefully spur some new ways of trying to deal with it.

And to point out, the drawbacks with this method are that you'll have the 2 extra headers in all of your emails. If anyone knows of an easier way to get the info from the directors in the exim.conf over to the filter without having to use headers, please let me know ;)

John

[xemaps]

Seems that it runs so on my server
it spam scan
- forwarded internal aliases & external domains
- local mailboxes
it DOESN'T forward outside spam-scanned POSITIVES mail for main user and for mailboxes => but just leave them in spambox - can be good for blackberry's ;-) . I don't use catchall

Could be an alternative



Originally posted by diradmin
It seems I found solution at last.
On our servers works fine.
Please, check it on your servers.... may be I something dismissed


# Spam Assassin
spamcheck_director:
driver = accept
condition = "${if and { \
{!def:h_X-Spam-Flag:} \
{!eq {$received_protocol}{spam-scanned}} \
{!eq {$received_protocol}{local}} \
{exists{/home/${lookup{$domain}lsearch{/etc/virtual/domainowners}{$value}}/.spamassassin/user_prefs}} \
{exists{/etc/virtual/${domain}/passwd}} \
{ \
or \
{ \
{!eq {}{${lookup{$local_part}lsearch{/etc/virtual/${domain}/passwd}}}} \
{eq {$local_part} {${lookup{$domain}lsearch{/etc/virtual/domainowners}}}} \
} \
} \
} {1}{0}}"
retry_use_local_part
transport = spamcheck
no_verify


Its scann spam of main system account too.

[nobaloney]

Originally posted by Dualdot
I did notice something else however, in combination with the latest spamblocker exim.conf: non-existent addresses (like lalala@example.com) still get checked by all dns-lists and so on (spamhaus, orbs, etc). Why isn't it rejected right away because these is no such address?
Exim accepts email in multiple steps:

1) mail from
2) receipt to
3) data

Blocklists are checked during mail from and recipients are checked for validity in receipt to.

Jeff

[stan]

Originally posted by DirectAdmin Support
I've just spent the day trying to come up with another solution. My reasoning was that everything should be able to be scanned.. but the problem is with the filter in that it doesn't know what type of emails it's dealing with, which is why it's blindly saving things...


this solution works great for pop-boxes, but not for aliasses. the filter doesn't specify what to do when X-is-alias = "yes".

what happens now is that spam for aliasses is delivered in the inbox of the corresponding pop-box, not in the spam folder (spamassassin is setup in DA to deliver to the spam folder).

Summarized:
- spam to: pop@domain.com = delivered to spam folder

forwarder: alias@ => pop@
- spam to alias@ is delivered to the inbox of pop@


How can this be solved?

[xemaps]

this is solved as forwarded spam => delivered to aliased spam box (not to inbox which would stupid)

need change exim code as shown in this topic.

[stan]

Originally posted by xemaps
this is solved as forwarded spam => delivered to aliased spam box (not to inbox which would stupid)

need change exim code as shown in this topic.
yes we changed the exim code as suggested by directadmin support above, but alias-spam is still delivered in the inbox of the corresponding pop-box, and not in the spam folder, which is rather strange.

hopefully DA-support could help with this one?

[xemaps]

be sure to delete box and follow new exim sa code
i hope you redirected spam to user folder spam box.

Even don't use catchall spambox /or account.

[stan]

Originally posted by xemaps
be sure to delete box
what do you mean with 'delete box'?


follow new exim sa code
could you post the exact code which is working for you? there are several solutions in this thread, we have tried the latest solution in this thread from directadmin support.


Even don't use catchall spambox /or account.
we need catchall to be enabled, but it's no problem if the catchall-spam doesn't get forwarded properly


i hope you redirected spam to user folder spam box.
yes of course :) (set in DA cpanel)

[xemaps]

Code that work on my DA/FC3/Exim 4.63/SA 3.1.4 server
replace in exim.conf the spamcheck director :

#sa2 replace spamcheck_director
# Spam Assassin modified for alias spam
spamcheck_director:
driver = accept
condition = "${if and { \
{!def:h_X-Spam-Flag:} \
{!eq {$received_protocol}{spam-scanned}} \
{!eq {$received_protocol}{local}} \
{exists{/home/${lookup{$domain}lsearch{/etc/virtual/domainowners}{$value}}/.spamassassin/user_prefs}} \
{exists{/etc/virtual/${domain}/passwd}} \
{ \
or \
{ \
{!eq{}{${lookup{$local_part}lsearch{/etc/virtual/${domain}/passwd}}}} \
{eq {$local_part}{${lookup{$domain}lsearch{/etc/virtual/domainowners}}}} \
} \
} \
} {1}{0}}"
retry_use_local_part
transport = spamcheck
no_verify
#sa2

Delete any redirected mail box from alias if exist in /var/spool/virtual/domain.tld/
=> you must have ONLY existing true mailbox (no alias) in that folder

and ... restart exim

One good thing is that if you forward mail, only spamless will go through ;-) spam stay in original box.
It is not perfect for all situation but for me it's very good.

[OliverScott]

Code that work on my DA/FC3/Exim 4.63/SA 3.1.4 server
replace in exim.conf the spamcheck director :

#sa2 replace spamcheck_director
# Spam Assassin modified for alias spam
spamcheck_director:
driver = accept
condition = "${if and { \
{!def:h_X-Spam-Flag:} \
{!eq {$received_protocol}{spam-scanned}} \
{!eq {$received_protocol}{local}} \
{exists{/home/${lookup{$domain}lsearch{/etc/virtual/domainowners}{$value}}/.spamassassin/user_prefs}} \
{exists{/etc/virtual/${domain}/passwd}} \
{ \
or \
{ \
{!eq{}{${lookup{$local_part}lsearch{/etc/virtual/${domain}/passwd}}}} \
{eq {$local_part}{${lookup{$domain}lsearch{/etc/virtual/domainowners}}}} \
} \
} \
} {1}{0}}"
retry_use_local_part
transport = spamcheck
no_verify
#sa2

Delete any redirected mail box from alias if exist in /var/spool/virtual/domain.tld/
=> you must have ONLY existing true mailbox (no alias) in that folder

and ... restart exim

One good thing is that if you forward mail, only spamless will go through ;-) spam stay in original box.
It is not perfect for all situation but for me it's very good.

I have set this up and it seems to be working well!

I take it that the only known limitation is that a catchall account when set to the main domain account won't get scanned? And that this can be fixed by having catchall emails delivered to a regular user pop mail box?

[OliverScott]

I had test the setting ...

the user_pref doesn't work on forwarding setting, if I want set the spam for forwarding mail, I had to set the local.cf, but that will effect all the incoming mail.

"user_pref doesn't work on forwarding setting" - what does this mean exactly? I tried changing my user_prefs and they seemed to also be applied to emails forwarded to external addresses OK.

Does this mean that it is the user_prefs of the original domain which are used rather then the user_prefs of the receivers domain, when both domains are on the same DA server?