Email purge feature

[Seniferol]

Hello,

I'm posting this after I've already searched the forums and the web for it, so unless my search queries weren't good enough, I'm pretty sure this feature doesn't yet exist in DirectAdmin, namely: purging emails older than X days. Some of our clients would like such a feature (especially if they're forwarding their mail from their relatively small inboxes tied to their domain to something like Gmail with gigs of emailspace and thusly the emails in the "old" box are mere copies which can be deleted), but as the feature appears to be absent, I can't help them in a simple way.

Now I'd like to know if there is such a feature and if I just overlooked it, or if it's not, if there are plans for adding such a feature anytime soon.

Thanks!

[spectrumproduct]

i would also like this .. guys?

[floyd]

Some of our clients would like such a feature (especially if they're forwarding their mail from their relatively small inboxes

If they are forwarding mail then nothing is in the inbox on your server.

[nobaloney]

Floyd, if the user is set up both as a mailbox and a forward, then it will be delivered to both places.

Seniferol and spectrumproduct:

There's no such feature in DirectAdmin; there is a way to delete all the email.

Jeff

[floyd]

if the user is set up both as a mailbox and a forward

Why would somebody set up both and then not check the mailbox? That seems a little unintelligent.

I just tested a system user which of course has a mail account that cannot be deleted. I set up a forward through DA using one of their domains. I forwarded username@domainname.com to my personal address. I then sent a test email. The test email was forwarded to my personal address and NOT delivered to the local user's mailbox. If this were done for each domain then the only mail the system user would possibly get is mail sent to the users from that server or username@hostname.com.

With virtual addresses Jeff you are correct. But the question still remains why would somebody set up a virtual email account and then not check it? If the virtual email box is not going to be checked then delete it and problem solved.

If they ARE checking the mailbox and leaving mail on the server then they can have the mail client delete the mail after x number of days.


if they're forwarding their mail from their relatively small inboxes

Mail does not get delivered to the mailbox and the mail in the mailbox forwarded. Mail is forwarded from the smtp process. So if you are not going to check the mail in the user's mailbox then just delete the mailbox account. Then the mail will only get forwarded and not delivered to the local mailbox.

[nobaloney]

With virtual addresses Jeff you are correct. But the question still remains why would somebody set up a virtual email account and then not check it?
Because on many platforms (including straight linux/unix you have to set up a user before you can set up a forward.

Old habits die hard.

Mail is forwarded from the smtp process.
That's a bit of a simplification. Exim includes a delivery mechanism which has nothing to do with SMTP.

Jeff

[floyd]

If Seniferol and spectrumproduct are talking about virtual mail accounts and not system users then all they really need to do is delete the virtual mail account. The forward still works.

[floyd]

Because on many platforms (including straight linux/unix you have to set up a user before you can set up a forward.

I know that is true for system users but not for virtual email accounts which is why I said


why would somebody set up a virtual email account and then not check it?

[nobaloney]

I know that is true for system users but not for virtual email accounts
And that's why I said they'd do it because old habits die hard.

Jeff

[IT_Architect]

Please add this feature. I've had the problem for years, and I thought it time to say something about it. I found someone already has, and I'm adding my support to this request.

Typically, clients often have multiple devices for accessing e-mail. Moreover, some clients have shared e-mail accounts. E.G. Office, CAD, etc. where several people access the same mail box. They all set their POP3 clients to "Leave mail on server" You can tell them to set the client to delete them after X days, but they don't. If they use webmail, they just sit out there forever. They don't delete them until access gets slow. Then they have about 1500 to delete per account, one page at a time, to get back to 30 days worth. In short order our web hosting server becomes their company archive.

This is not a little problem that affects a few. It is a VERY widespread problem. It's also a pain for them to go into webmail and delete 1500 e-mails, one page at a time. When the customer starts to get close to his limit, HE CALLS US! He doesn't know that it's his e-mails causing the problem. When he finds out, it becomes our fault for not having something to maintain it for them.

Adding this feature would be of huge benefit. Any limit would be better than none. If it isn't too much work, I'd like to see:
1. Reseller max setting.
2. User setting that can only go up to the reseller max setting.

But again, even a global setting would be fine with me.

[floyd]

This is not a little problem that affects a few. It is a VERY widespread problem.

What evidence do you have that its a widespread problem?
What is the total number of people using DirectAdmin?
What is the total number of people having the problem?

(total number having a problem * 100) / total number using DA = percentage of people having a problem.

What percentage do consider to be widespread? And how have you arrived at the above numbers?


When he finds out, it becomes our fault for not having something to maintain it for them.

Its up to the customer to maintain his own email, not the host.

I also do not think the customer wants the host to arbitrarily decide what email should be deleted from the server.



.

[getUP]

Its up to the customer to maintain his own email, not the host.

You are absolutely right.


I also do not think the customer wants the host to arbitrarily decide what email should be deleted from the server.

I really have to agree with Floyd on this. The provider will always take the blame for any situation. If e-mail is deleted after 30 days and your client is not aware of this happening, he will be even more upset than having a full mailbox. That's up to the provider of course, but in my opinion if you really need a feature like this; create it yourself.

[IT_Architect]

Floyd, this isn't about what you want. It's what the first two guys on this thread want and myself. This is a request for the means to reign in e-mail just like we do disk space, bandwidth, and log files. If we want to spell out to our customers that e-mail will be deleted off the servers after 30 or 60 days, that's our prerogative. Just don't turn that feature on for your sites.

[redesb]

Googling for 'dovecot expire' you get 'Expire plugin - Dovecot wiki (http://wiki.dovecot.org/Plugins/Expire)' link. And in the 'Dovecot configuration file - Plugin settings section (http://wiki.dovecot.org/MainConfig?highlight=%28expire%5C-tool%29#Plugin_settings)' you can get some more info about.

Hope this help,
Ramon

[getUP]

Floyd, this isn't about what you want. It's what the first two guys on this thread want and myself. This is a request for the means to reign in e-mail just like we do disk space, bandwidth, and log files. If we want to spell out to our customers that e-mail will be deleted off the servers after 30 or 60 days, that's our prerogative. Just don't turn that feature on for your sites.

The vast majority will likely never use this feature. Therefor the added value is close to zero. Perhaps some feel different, including yourself, though I'd recommend using custom tools to achieve this. The plugin Ramon pointed out will probably work. I don't see why DirectAdmin should be responsible for such features.

[floyd]

Floyd, this isn't about what you want. It's what the first two guys on this thread want and myself.

I don't want DA working on a feature that 99% of the people do not want. So I am making it known that I think this is a bad feature and why. I have just as much right to request that DA not waste their time on a feature as you do to request the feature.

This is not as simple as deciding not to turn a feature on or off. Its about DA wasting their time on something when that time could be used for something that 99% of the people DO want.

Further even if you and the others DO implement such a feature it will likely cause you to lose customers and perhaps even cause you legal trouble since you will be vandalizing customers' property without justification.

The OP had a problem because an unused mailbox was getting filled up. He thought in order to have a forward he also had to have a local mailbox. The customer did not use the local mailbox. So deleting mail that should never have been there is not a problem. The customer had the mail set up wrong.

You are suggesting deleting customers email in a mailbox that is currently being used by the customer. I hope your customers read this forum so they are warned about you are going to do to their personal mailboxes.




.

[IT_Architect]

I don't want DA working on a feature that 99% of the people do not want.
What evidence do you have that 99% of the people do not want it?
What is the total number of people using DirectAdmin?
What is the total number of people that don't want it?

(total number that don't want it * 100) / total number using DA = percentage of people that don't want it.

The guys at DA a big boys. They know how to prioritize their time.

[floyd]

What evidence do you have that 99% of the people do not want it?
What is the total number of people using DirectAdmin?
What is the total number of people that don't want it?


I can appreciate you trying to use my words against me but as you can see I never said that 99% don't want this specific feature whereas you did say that this specific feature request would address "a VERY widespread problem." How do you define "widespread"? I see 3 so far and 2 of them are over a year old.

I made a general statement. I don't want DA working on any feature that 99% of the people will not use. They will know what the numbers are.

But if DA decides that there are enough requests to make it a feature then that will be great for the ones that want it. I am just trying to let people know that they could be in trouble by making use of such a feature. Deleting customers' email is a very serious matter and should not be taken lightly.


.

[IT_Architect]

I can appreciate you trying to use my words against meNo trying about it. You're not fooling anybody. There is no difference in this context between wouldn't use and don't want.


...they could be in trouble by making use of such a feature. Deleting customers' email is a very serious matter and should not be taken lightly.As having been CTO of two large automotive suppliers before starting my own business to consult them, I'm well qualified to speak to that. The only way you get into trouble is if you don't have a policy, you don't enforce your policy, or your policy is in violation of governmental or your industry mandates. Without a policy, you can get in trouble for erasing a former customer's web site. Without a policy, you could be in trouble when a customer doesn't do his backups. With a hosting policy that states e-mails will automatically deleted from the server after XX days, there is zero risk. For businesses with formally mandated e-mail retention requirements, the e-mail handling capabilities included with DA do not meet the requirements.

[DirectAdmin Support]

Hello,

Regarding this request, for system using Maildir (dovecot) we do already have code in place to do this check on the spam and trash folders.
http://www.directadmin.com/search_versions.php?query=purge

This code could be easily (without much time) adopted as a another button next to the "Empty" button that's aready there. It's very much along the same lines. I would not make this an automated feature, since people are bound to get themselves into trouble with it, however a option for Users to click it as they want would be fine, and not too much work.

Ultimately, disputes for should we have it, or should we not, almost always come to "give them the option". Since the code already exists internall in DA for us to call, and it would be much work at all, I'll throw it in for the next release.

If it were to have required more work (if the code was not already present), then more weight on it's priority of implementation would have been required for the decicion.

John

[floyd]

however a option for Users to click it

This I agree with since users already have the option to delete their own email. The users should always have the option to delete their email whether its individually or in bulk. The hosts should not take it upon themselves to do this.

Keep in mind you have to switch to dovecot. If you were already using dovecot then the host already had to ability to delete emails older than x days using cronjobs.


The only way you get into trouble is if you don't have a policy

Trouble is a general term to cover many different aspects, legal only being one of them and where a policy has any effect. A policy will not keep a customer from canceling because a host deleted his email.


There is no difference in this context between wouldn't use and don't want.

I agree but you missed the point entirely and you will never understand it so I will stop wasting my time trying to help you. I volunteer my time here to help reasonable people, not to argue.

[IT_Architect]

I'll throw it in for the next release. That's Awesome! Thank you! Thank you! Thank you!


...so I will stop wasting my time trying to help you. That's hilarious. Help? What help? All I asked for a something a lot of people need and you came trolling and got caught in your own trap.

[floyd]

All I asked for a something a lot of people need and you came trolling and got caught in your own trap.

All I asked for is what evidence do you have that a lot of people need it. Sometimes people make big claims that are not true just to get their personal agenda pushed through.

Its ok if its just 1 person or a small number of persons that want a feature. That is why the forum is here. People don't have to puff up the numbers to get DA to consider a feature especially if its easy to implement.

[capono]

Today I used the email purge feature for the first time. And it seems it don't work well.

First at squirrelmail I found a lot of spam in the spamfolder, from today going back to 2007. I then logged in DA, and at the email accounts page I selected the checkbox for the right email account, and at the table footer I choose 'Purge from' > 'Spambox' > 'Older than 356 days'. (Why is that 356 and not 365?)

Going back to squirrelmail, I saw the ALL the spam was deleted. I expected to have left about a year of spam.

Is this a bug or by design?

[DirectAdmin Support]

Hello,

I just went through the entire code based for this tool and didn't have any errors with it. My test was as follows
[root@server new]# pwd
/home/admin/imap/test.com/char/Maildir/.INBOX.spam/new
[root@server new]# ll
total 8
-rwxr-xr-x 1 admin mail 178 Jul 7 00:28 pack.sh
-rw-r--r-- 1 admin mail 5 Nov 25 01:26 test
[root@server new]# ll
total 4
-rw-r--r-- 1 admin mail 5 Nov 25 01:26 test
[root@server new]#Where I just grabbed a random file with an old timestamp and put it into the folder. I created a new file, and put it in the folder. I ran DA in debug mode. Level 600 will give you info on the purging tool. This is the debug output I got from that
empty_file_for_user(char, test.com, spambox, 60): days: 60
Emptying /home/admin/imap/test.com/char/Maildir/.INBOX.spam: days=60
email user /home/admin/imap/test.com/char/Maildir/.INBOX.spam
checking pack.sh : no date in name
** Deleting: pack.sh
checking test : no date in nameSo likely, the only difference between your files and mine would be the "date in name". My random files didn't have timestamps in them, so DA defaulted to using the system timestamp via the stat() function. If there is a timestamp in the filename itself, DA will search for "%d.", meaning the first number it finds that is followed by a dot. That number would be assumed to be the timestamp of the file. If it was an incorrect number, that could explain your missing files.

A sample filename is:
1257540811.H855755P781.server.hostname.com:2,SThe timestamp in the filename is "1257540811". If you turn that into human time, it would read "Fri, 06 Nov 2009 13:53:31 -0700", more or less, depending on your timezone, etc...

So any "wrong number" in that time in the filename could be the cause of the file being deleted. If you have backups of your Maildir/.INBOX.spam directory, you could keep reloading the files in there to duplicate the issue repeatedly to get a better picture, then let me know.

Anyway, if you want to run DA in debug mode to see what it's deciding, this feature has a lot of handy debug code for it's decisin making, so that might help figure out what's going on in your case.
cd /usr/local/directadmin
killall -9 directadmin
./directadmin b600then issue the purge on some other account to see what it does.

If anyone else is also noticing weird behavior with this tool, mention it here.

And yes, 356 was a typo, I've changed it to 365 for the next release. It would have resulted in DA doing the code for emails older than 356 days instead of 365.

John

[DirectAdmin Support]

For completeness, I've tested again with a more realworld setup using dates in the filenames. This is the before and after shot of the directory:
[root@server new]# ll
total 12
-rw-r--r-- 1 root root 10 Nov 25 01:44 1057540811.H855755P781.server.hostname.com
-rw-r--r-- 1 root root 6 Nov 25 01:44 1257540811.H855755P781.server.hostname.com
-rw-r--r-- 1 admin mail 5 Nov 25 01:26 test
[root@server new]# ll
total 8
-rw-r--r-- 1 root root 6 Nov 25 01:44 1257540811.H855755P781.server.hostname.com
-rw-r--r-- 1 admin mail 5 Nov 25 01:26 test
[root@server new]#where 1257540811 is Nov 6, 2009, and 1057540811 is July 6, 2003. This is the debug output from my test:
empty_file_for_user(char, test.com, spambox, 365): days: 365
Emptying /home/admin/imap/test.com/char/Maildir/.INBOX.spam: days=365
email user /home/admin/imap/test.com/char/Maildir/.INBOX.spam
** Deleting: 1057540811.H855755P781.server.hostname.com

[capono]

Hello John,

Do I need root access? I don't have root access on the server. Our company is acting as reseller. We do almost all administration through DA.

If we have collected some spam again, I will look for the message date in the source before using the purge tool. If I keep getting problems, I'll contact our provider first. They have root access.

Thanks for now.

[DirectAdmin Support]

Hello,

Yes, root is required to run in debug mode.

You can check the timestamp of the file (last write), and the timestamp in the filename via your FileManager.
/imap/domain.com/user/Maildir/.INBOX.spam/new
/imap/domain.com/user/Maildir/.INBOX.spam/cur

John