monthly logs

[Richard]

I have a problem with the rotate of website logs. On 1st the server had to rotate, and compress, the last month logs, but that didn't happend, and the system is using to set the access of this month in the same log file.

Maybe is a matter of time.. but in past months the rotate was on 1st of each.

This is happening in all DA servers i check (3 for now). Maybe a update bug?

[DirectAdmin Support]

Hello,

I'm not too sure. Check your /etc/cron.d/directadmin_cron. The command that does the monthly reset is:

echo "action=reset&value=all" >> /usr/local/directadmin/data/task.queue

You can try running it now to see if it works.

John

[Richard]

I did it but nothing happend.
This is happening in all of our servers that we upgraded to the last Direct Admin version 1.22.2

[DirectAdmin Support]

Hello,

What do you mean nothign happened? You won't see any output because the "echo 'text' >> file" command just adds text to a file. Make sure that your cron daemon is running. Also check /usr/local/directadmin/data/task.queue. The file should be non-existant most of the time. The only time it should exist is when you run a command, but the dataskq should find it within one minute and get rid of it. The crons are run from /etc/cron.d/directadmin_cron. Give it about 10 minutes after running the reset command, then check your user stats. (actually view one user.. the list of users has a cache and might not be updated instantly)

John

[RTKS]

I think I am also seeing this problem but want to confirm it before doing anything.

I received notice that one of my accounts had used 75% of their bandwidth for the month. When I looked at the webalizer stats things were right. When I look at DA's internal stats it seems to have combined June and July for it's total. Is this the same problem?

If so, I did check the cron mentioned above and the command is there. Any idea why this might not have worked?

If I run the job now will it properly split June and July or will it begin July from time of run forward?

[LyricTung]

I'm also seeing this problem. Logs did not rotate on 7/1/04 as they should have. Anything new on this issue?

Several servers, DA upgraded on all within days of new version releases.

[LyricTung]

Okay, so, server apache logs did not rotate on 7/1/04 or 8/1/04. User backup logs were not created on 7/1/04 or 8/1/04.

I'm thinking DA uses an internal log rotate solution.
Although, I could be off base here. I'm assuming this to be so because I have never had logrotate on my FreeBSD server. I use newsyslog. And even so, DA had been rotating logs. The last user backup I see occurred on 6/1/04.

Cron runs fine, everything else is fine. When I run the DA reset for all by hand, there are no errors in the DA logs. All tallies run fine, etc. Bandwidth stats are fine. But logs are not rotated.

[LyricTung]

Is noone else having this problem anymore? Is it specific to FreeBSD? Just looking for some input here. Have other people checked to see if their servers are having this problem?

Does DA have an internal log rotation system? Am I missing another thread that actually discusses this?

Ah well, I guess I could just set-up logrotate or something to take care of it.

[jmstacey]

I'm running FreeBSD 4.10 and my logs rotated sucessfully for July/August, so its not a FreeBSD problem.

Have you tried updating directadmin incase something was corrupt? Also check administrator settings for the logrotate configuration for apache.

[outpernet]

RH9, RHE3, Fedora cr1. Tthe problem continues this month

[LyricTung]

I've been updating DA as the releases come out. I've had this problem since upgrading to 1.22.2 on 6/25/04. Server I'm testing and diagnosing on is currently at 1.22.4.

I did go to the admin settings and change the default 100M log rotate size to 20M. I then ran:

echo "action=reset&value=all" >> /usr/local/directadmin/data/task.queue

And logs successfully rotated based on size. I have now changed it back to the default value. It would seem that the monthly reset command is no longer, for whatever reason, rotating and making user backups of logs based on the 1st of the month. I also find no directadmin errors on the 1st. The monthly reset/tallies ran fine according to DA.

It does seem odd that some of us have this problem and some of us don't. So now I'm wondering what other things were upgraded or installed on individual servers in June that could possibly effect DA's inability to rotate logs by date. arrrgh...

[David]

I have the same problem - suddenly a bunch of sites have all been suspended because since the upgrade, the logs were not rotated 08/01/2004, and bandwith tallies are adding up.

This is a really big problem when you have 1000 clients per server. How can I fix this, and have the log rotation run today?

[Dixiesys]

YES I am having this problem too!! Redhat 9 and I checked 4 servers and ALL 4 have not had logs updated in /home/user/domains/domain/logs/ since June 1, every server I've checked has this problem.

[LyricTung]

Has anyone figured this out yet? We are fast approaching 9/1/04.

Current temporary solution is to change size value to something minimal on 8/31/04. But it would be nice to find out what the problem is here.

[rldev]

Has anyone sent a support ticket to DA about this?

[da_host]

Did everybody got it work?
What was the problem?

[Dixiesys]

Looks like pretty much all my servers aren't rotating logs anymore other than when one reaches the proper size. Monthly rotation seems broken.

[jmstacey]

Originally posted by Dixiesys
Looks like pretty much all my servers aren't rotating logs anymore other than when one reaches the proper size. Monthly rotation seems broken.

Well find out in another day I guess.

[jmstacey]

1.224 :: FreeBSD 4.10 and logs just rotated from August to September succesfully so it doesn't appear to be a DA problem (at least the binary for freebsd).

[Dixiesys]

I just checked and one of my newer lesser filled servers shows no log rotation yet for the 1st of the month. Is this supposed to happen at midnight or some other time?

DA latest + RH9 are all my servers.

[Dixiesys]

I have 1 server that was still running 1.22.1 and it also quit rotating logs on a monthly basis as of around May. I can't find a server out of about 30 that is rotating logs monthly in June/July or now August.

[da_host]

Its very odd that support havent published any patch or updates on this prob..

[hostpc.com]

I've been reminded (spanked) on several occasions from another forum user that Support does not monitor these forums for complaints, bugs, etc. If there's a tech issue we need to email them now, supposedly.

Has anyone emailed john yet? If not, I'll fire off another email to him concerning this.

[jmstacey]

Are all of your servers RedHat? Its looking like thats the general trend.

[Dixiesys]

All of mine are Redhat 9, I think hostpc uses Fedora on some of his, or most? all? Anyway, I did email John and first he had me do:


echo "action=reset&value=all" >> /usr/local/directadmin/data/task.queue

which is what I've done again to test after month started just now and still no rotating of logs.

When that didn't work he recommended setting rotate size lower:


The log rotate size is currently set to 100 meg (Admin Panel -> Admin Settings). If you lower the value, the files will rotate more often.

[hostpc.com]

Fedora is on a few servers, the rest are RH9 (honestly like RH9 better).

I haven't emailed DA directly yet, but it looks like I'd probably get the same reply as Gary anyway. We need a clear cut fix for this - it's gonna start getting out of hand, soon!

[nobaloney]

We run servers on both RHL7.3 and on WBEL/RHEL 3. We also operate servers for clients on RHL9.

Please indicate exactly which log we should look at and where, and I'll check one each and see if they rolled over the beginning of September.

Thanks.

Jeff

[Dixiesys]

try this:
ll /home/*/domains/*/logs/

look for files older than May or June I see VERY few, those are all for sites who rotate at 100 megs and get a lot of traffic.

[nobaloney]

Hmmmm.....

RHL 7.3:
Jul-2004.tar.gz
Jun-2004.tar.gz
May-2004.tar.gz
Nov-2003.tar.gz
Nov-2003.test.tar.gz

RHEL 3.0 and RHL9 show similar results.

This is indeed a real problem. And has nothing to do with setting a lower rotate size.

I'll bring the thread to John's attention and see if we can have him look into it.

Jeff

[DirectAdmin Sales]

Hello,

Thanks for bringing this to my attention. I have a hunch what it might be. To confirm this, all I really need is for someone to delete 1 (or more) log files from the domains logs directory (so that there are 4 or less tar.gz files). If the next log is rotated and saved, then that will confirm the problem.

I'll add it to the versions system and it should be all fixed up for the next release.

I pretty sure that the cause is the new log is being deleted instead of the old one because of the way the index for the array of files is stored. It's any easy fix, but I just need a confirmation.

Thanks!

John

[nobaloney]

Okay, John...

What do I have to do?

Are you saying delete one and wait a month, or is there something else I can do?

Do I have to delete it from every domain or just from one?

Jeff

[Dixiesys]

John,

If it were simply a case of 4 files in /home/user/domains/domain.com/logs why would sites with only 1 or 2 files in there not rotate? I have a server that is only a couple weeks old with no files in there for some sites, on one server in particular, it was built in July, every domains/domain.com/logs/ folder is empty except 1 on this server. So I can safely say there were less than 4 .tar.gz files in all 103 sites on this particular server when August rolled into September earlier this week and logs did not rotate.

[outpernet]

any news on this? we hce less thant 4.. thhe problem is that the log not rotate, so morer than 2 months laters we are starting to have problems with the space in a few domains ...

[DirectAdmin Support]

Hmm. If you'd like me to take a closer look, send your IP and root password. The code appears to be fine (as far as I can tell), but it does make assumptions about the datestamp on the file. I'll need to take a closer look at the affected systems to better understand why it's not working on some systems.

John

[outpernet]

thankyou, im contacting you by email

[LyricTung]

Still no monthly log rotate here. I did notice while it's listed in 1.22.5 as a bugfix, closer inspection yields it wasn't finished?

Version Number 1.225
Finished No
Type bugfix
Overview log rotation files are not being kept in the logs directory

And, actually, the problem is not that the backed up log isn't being kept. The problem seems to be that the log doesn't rotate at all based on date. It just keeps growing. The only way I can get the log to rotate and back-up is to drop the size limit.

If I drop the size limit, and issue the reset command everything works fine including the bacl-up being properly stored.

[eSupport.org.ua]

Originally posted by LyricTung
Still no monthly log rotate here. I did notice while it's listed in 1.22.5 as a bugfix, closer inspection yields it wasn't finished?

Version Number 1.225
Finished No
Type bugfix
Overview log rotation files are not being kept in the logs directory

And, actually, the problem is not that the backed up log isn't being kept. The problem seems to be that the log doesn't rotate at all based on date. It just keeps growing. The only way I can get the log to rotate and back-up is to drop the size limit.

If I drop the size limit, and issue the reset command everything works fine including the bacl-up being properly stored.


Ok, What same I need do for fix issue?
Handle run webalizer and flush log to zero?

[UltimeWWW]

Seems like it's doing it on another server too.

[Chrysalis]

I looked on my production server and the logs dir is empty.

/home/user/domains/domain.com/logs

is where I looked and not a single file there for all my users.

[Dixiesys]

I can't find a server with properly rotating logs, granted I ain't looked at all 40 something DA servers but the ones I looked at didn't have rotating logs EXCEPT for those sites busting the 100 meg limit that rotates at least.

[Chrysalis]

am I the only one to notice the file /usr/local/directadmin/data/task.queue doesnt exist making 3 of the crontab entries worthless.

[Richard]

Originally posted by Dixiesys
I can't find a server with properly rotating logs, granted I ain't looked at all 40 something DA servers but the ones I looked at didn't have rotating logs EXCEPT for those sites busting the 100 meg limit that rotates at least.

u can change that in /usr/local/directadmin/conf/directadmin.conf

and change log_rotate_size=10 so logs are rotated when the hit 10 meg

[eSupport.org.ua]

Originally posted by Richard
u can change that in /usr/local/directadmin/conf/directadmin.conf

and change log_rotate_size=10 so logs are rotated when the hit 10 meg

Ok, if I cahnge - what be doing with logs over 100meg?

[Chrysalis]

from what I understand isnt rotating at small size not a fix, would it reset the bandwidth counter?

And what about the non existant task.queue file?

[Richard]

the log's rotation compress (tar.gz) and put in /home/user/domains/logs/file.tar.gz

[Chrysalis]

Originally posted by Richard
the log's rotation compress (tar.gz) and put in /home/user/domains/logs/file.tar.gz

I know that but when it rotates bw counter is reset, and the rotate by time interval does NOT work. So this leaves 1 of 3 situations.

1 - Rotate manually by deliberatly setting the rotate size low at the end of the month to ensure user's domt get unfairly charged for bandwidth.

2 - Set the rotate size low permanatly but this would leave no active bandwidth measurement in place.

3 - Leave how it is and give all users unlimited traffic to compensate since the traffic measurement is effectively broken (not ideal).

I see this as a major problem that needs looking at, maybe it is linked to the exim logs not been rotated problem, but I am going to look at the directadmin method, further analysis I see how task.queue works, the file doesnt exist and when it is made it is processed the following minute after creation. There is the following crontab entries

* * * * * root /usr/local/directadmin/dataskq
2 0-23/6 * * * root echo 'action=vacation&value=all' >> /usr/local/directadmin/data/task.queue;
5 0 * * * root /usr/sbin/quotaoff -a >/dev/null 2>&1; /sbin/quotacheck -aug >/dev/null 2>&1; /usr/sbin/quotaon -a >/dev/null 2>&1;
30 0 * * * root echo 'action=tally&value=all' >> /usr/local/directadmin/data/task.queue
40 1 1 * * root echo 'action=reset&value=all' >> /usr/local/directadmin/data/task.queue
0 4 * * * root echo 'action=check&value=license' >> /usr/local/directadmin/data/task.queue

apperently 'echo "action=reset&value=all" >> /usr/local/directadmin/data/task.queue' resets the logs but when I run this command it fails.

since /usr/local/directadmin/dataskq is what processes task.queue then that file I must presume is broken. Am I on the right track here?

[Chrysalis]

from errortaskq log when I did the echo command

2004:10:30-18:26:00: Reset All has started
2004:10:30-18:26:00: Reset All has finished

but no rotation took place.

[Chrysalis]

tried using mount to mount /usr/home to /home instead of symlink and this made no difference the echo 'action=reset&value=all' >> /usr/local/directadmin/data/task.queue still does nothing for log rotation.

Also got issue of no exim log rotation, friend of mine who also runs directadmin had a full /var earlier, turns out exim_mainlog was 180 meg and the reject file was another 24 meg on top.

This log rotation issue I think should be atop priority for DA it defenitly seems an issue on FreeBSD boxes.

[Jing]

I think that someone should definitely start a list of problems on FreeBSD machines.

Logs are not rotating, Exim logs aren't showing... what else?

[Chrysalis]

John from DA told me the crontab reset entry is only supposed to reset the tally's and not rotate logs, there is no monthly rotation for apache logs they only rotate by size. He also said exim log rotation bugfix is coming.

[LyricTung]

Originally posted by Chrysalis
John from DA told me the crontab reset entry is only supposed to reset the tally's and not rotate logs, there is no monthly rotation for apache logs they only rotate by size. He also said exim log rotation bugfix is coming.

Well, That's nice to say but was not the case back in May. The logs used to rotate monthly. In fact, there was a bugfix included in a recent version of DA (even though it was marked as not completed and has not been addressed since.)

I believe DA doesn't have any idea what they did to stop the monthly logs from rotating or how to fix it. Thus, the issue is being dropped and they are redefining how things are "supposed to" work.

[outpernet]

Originally posted by LyricTung
Well, That's nice to say but was not the case back in May. The logs used to rotate monthly. In fact, there was a bugfix included in a recent version of DA (even though it was marked as not completed and has not been addressed since.)

I believe DA doesn't have any idea what they did to stop the monthly logs from rotating or how to fix it. Thus, the issue is being dropped and they are redefining how things are "supposed to" work.


I think exactly the same

[Chrysalis]

I have only been using directadmin since august so I think I have never seen it working how you guys say, but if its true then yeah its dissapointing.