Sudden surges in bandwidth

[bashy]

I have an issues with bandwidth and cannot find any information in the logs of what might have caused this,

A customer has broguht to my attention that his bandwidth is way over what is should normally be, He has not added and scripts or and extra traffic.

you can see from the image that since the 28th its gone sky high

Please see the image..

1. How can i find out what has caused this bandwidth hike?
2. How can i reset his bandwidth?

I am on freebsd and do not understand much about the info in the logs...
I am pretty new to this game.

Look forward to some help on this one please?

[nobaloney]

It looks like http traffic. Which is actually rare. Check the site carefully to make sure it's not (perhaps inadvertently) hosting warez.

Also check the log at /var/log/httpd/domains/example.com.log where of course you'd replace example.com with the domain name.

Jeff

[bashy]

It looks like http traffic. Which is actually rare. Check the site carefully to make sure it's not (perhaps inadvertently) hosting warez.

Also check the log at /var/log/httpd/domains/example.com.log where of course you'd replace example.com with the domain name.

Jeff
Hi Jeff

Thanks for the reply...

Can i ask what i would be looking for in the log please?
Also is there a way of accessing them through ssh, is so what would be the command please?
Its just that i am on a semi managed dedicated server and the support is pants to say the least!

[nobaloney]

Attack signatures are like art; I don't know what they are, but I know 'em when I see 'em.

Look for a lot of hits from the same domain or from different domains, look for fille requests that shouldn't be there, etc.

Jeff

[aiko]

I seem to have a similar problem like Bashy.
Normally I don't exceed my 2 Gig of bandwidth. From March 28 on bandwidth has been 5 times as high as normal.
From April 1th until now (so in two days) I seem to have used 1 Gig ....

Logfiles show no unusual activity, neither does the mail.
Webalizer only shows about 65 MB of bandwidth in April, Awstats shows about 69 MB, so I wonder what happened to the rest of the 1 Gig :confused:

My host is "working" on this but can't find a good explanation until now.

So, I'm not an expert on this but I wonder if it could have anything to do with the recent DA upgrade to version 1.29.3 ??

[nobaloney]

Usually but not always this comes from spammers using your server.

If they upload their own cgi or php script to send mail rather than use your mailserver it won't appear in your mail logs, but the connection should appear in the site-specific httpd log.

Jeff

[aiko]

Thanks Jeff,

I went throught the access log (and the error log) over and over again and there are no strange php or cgi scripts running...
Went through all the files on the server last night and there are no scrips on the server that don't belong there.

Set up a "catch all" mail address a couple of days ago to see if there are any bounces (as I would expect mail bouncing if the server is used for spamming) but this has no result as well.

Still the problem is my bandwidth is 10x higher as normal.

Would there be any way the counter in DirectAdmin not working properly? Could this be in any way a "config problem" of my host?
(as said before: I'm not an expert.... :confused: )

Bad thing is the "detail" button for the "Bandwidth (meg)" doesn't work (never did) so I can only compare the Webalizer (counting about 95 MB for April - which is the http traffic, right?) and the Account summary on DA (counting about 1.5 Gig)

Got mod_security running for quite some time to prevent bcc and multipart exploits and to "catch" the common spam words. Mod_security is using the error log, but it shows only a single entry now and then.

Even checked if my IP starts showing up in the main blacklists, but it does not (yet -:)

This is driving me nuts !:eek:

Aiko

[aiko]

My host fixed the not-working detail button, so I have some more information.
I "think" I found out what the problem is, but I don't know what causes it and I even don't know if my theory is a possible one.

Here are the bandwidth details for April:

April 1: 330.4 MB
April 2: 362.7 MB
April 3: 391.8 MB
April 4: 423.5 MB
Total : 1,47 Gig

You'll notice the bandwidth is increasing every day.

Now:
April 2 minus April 1 (362.7MB - 330.4MB) = 32.3 MB = my KB use for April 1
April 3 minus April 2 (391.8MB - 362.7MB) = 29.1 MB = my KB use for April 2
April 4 minus April 3 (423.5MB - 391.8MB) = 31.7 MB = my KB use for April 3

To me it looks like: daily total = (daily total + previous daily total)

Looks like the daily total is not reset to zero which causes a cumulative count per day, which of course results in a huge -virtual- increase in bandwidth use.

OK, I'm Dutch, so I hope my explanation in English makes sense .... :rolleyes:

Could anyone please tell me if it's possible to make this kind of mistake in any configuration file (maybe in the daily cronjob) that would cause this kind of behaviour? (so I could tell my host he made a mistake :p )

On the other hand: If I'm talking totally nonsens here, please tell me as well ;)

cheers,
Aiko

[SefAllen]

Aiko I've been having the exact same problem since the end of march aswell. My ISP told me today that DirectAdmin 1.29.3 has a problem counting the correct bandwidth and he upgraded to 1.29.4 (released today) trying to fix it. As it hasn't been 0:00 overhere yet (I'm also from the Netherlands) I cannot confirm if it has been fixed now. I will let you know at 0:00 when directadmin closes the day.

** Edit ** I just calculated the daily differences, like you did, and it came out approximately the same as webalizer indicated. So I think your explanation is very plausible

[aiko]

Thanks SefAllen!

I checked my theory on a friends domain which is at he same host and it's the same there.
Just curious: does my calculation apply to your statistics too, so far?

Noticed my host upgraded to 1.29.4 as well (we might be on the same host :D )

Cheers,
Aiko

[aiko]

Count seems to be normal after the last cronjob: 35MB for last day

So, my guess, as SefAllen and I are on different hosts, it was a version problem of DA that was solved in the last update?

Sysadmin must be one happy guy now :D

Thanks everybody for your input !!

Cheers,
Aiko

[dhutten]

hi folks

The problem still exists i think.

Whe updated yesterday to 1.29.4 on CentOS 4.4 around 16.00

On 0.00 the jobs about quota's are running.

But this are the results of Apache

2007 04 01 1.60 GB
2007 04 02 1.90 GB
2007 04 03 2.25 GB
2007 04 04 2.61 GB
2007 04 06 3.16 GB
2007 04 07 3.50 GB
2007 04 08 3.75 GB
2007 04 09 4.08 GB
total 22.85 GB

You see this can't be true.

Webalizer tells that this customer has used 2,5GB this few days. But not 22,85GB

Please advice.

[dhutten]

Today for the first time i see at some users normal MB levels instead of GB levels.

Now the other problem how can i delete the GB values. Because a lot of customers are over the bandwidth limits for these month.

[canreo]

I'm having the exact same problem:

2007 04 01 1.29 GB
2007 04 02 1.43 GB
2007 04 03 1.70 GB
2007 04 04 2.12 GB
2007 04 05 2.42 GB
2007 04 06 2.68 GB
2007 04 07 3.01 GB
2007 04 08 3.30 GB
2007 04 09 3.59 GB
2007 04 10 3.66 GB

I just upgraded to 1.29.4, so we'll see if this helps. Is there a way to reset these counts back to normal levels?

[ITDevil]

I'm having the exact same problem:

2007 04 01 1.29 GB
2007 04 02 1.43 GB
2007 04 03 1.70 GB
2007 04 04 2.12 GB
2007 04 05 2.42 GB
2007 04 06 2.68 GB
2007 04 07 3.01 GB
2007 04 08 3.30 GB
2007 04 09 3.59 GB
2007 04 10 3.66 GB

I just upgraded to 1.29.4, so we'll see if this helps. Is there a way to reset these counts back to normal levels?

I am having same problem with 1 of my account it use to have ~500MB usage per month now it has used over 7GB in 5 days



2007 04 01 919.2 MB
2007 04 02 1.01 GB
2007 04 03 1.12 GB
2007 04 04 1.31 GB
2007 04 05 2.71 GB

any idea or is it a bug ?

[hostpc.com]

Has anyone had their customers complaining about a sudden, unexplained surge in bandwidth usage this month (already)?

We've got a number of customers, that have fairly mild sites, previously using MAYBE 2gb of traffic - now in the first 10 days of April surging to 20GB+.

I'm evaluating logs, but webalizer and awstats both show significant surges in the tally - almost incrementally every day:


Date Apache Email Ftp DirectAdmin Other Total
2007 04 01 1.09 GB 109 KB 0.00 KB 0.00 KB 0.00 KB 1.09 GB
2007 04 02 1.18 GB 123 KB 0.00 KB 0.00 KB 0.00 KB 1.18 GB
2007 04 03 1.29 GB 135 KB 8.79 KB 0.00 KB 0.00 KB 1.29 GB
2007 04 04 1.44 GB 57.8 KB 0.00 KB 0.00 KB 0.00 KB 1.44 GB
2007 04 05 1.61 GB 68.4 KB 0.00 KB 0.00 KB 0.00 KB 1.61 GB
2007 04 06 1.76 GB 56.5 KB 0.00 KB 0.00 KB 0.00 KB 1.76 GB
2007 04 07 1.87 GB 135 KB 0.00 KB 0.00 KB 0.00 KB 1.87 GB
2007 04 08 2.02 GB 77.0 KB 0.00 KB 0.00 KB 0.00 KB 2.02 GB
2007 04 09 2.15 GB 84.1 KB 0.00 KB 0.00 KB 0.00 KB 2.15 GB
2007 04 10 2.30 GB 0.00 KB 0.00 KB 0.00 KB 0.00 KB 2.30 GB
total 16.71 GB 0.827 MB 8.79 KB 0.00 KB 0.00 KB 16.71 GB

Notice, each day is just a little more than the previous. It's similar to this for each site I check. I'm wondering if perhaps there isn't a bandwidth calculation error in the algorithym?

[ITDevil]

May be this is a bug or something, some other people including me have reported the same surges

http://directadmin.com/forum/showthread.php?p=98877#post98877

[hostpc.com]

Thanks, I didnt see the other thread,

Jeff, this one can be closed/merged when you have time please.

[interfasys]

I had the same problem last week and updating to .4 solved it.
Cheers,

[dhutten]

I had the same problem last week and updating to .4 solved it.
Cheers,

it seems to be solved, but still the other question is it possible to reset al stats?

Or we have to calculate everything from webalizer, even suspend at limit gives a problem.

[Tim]

In order to correct the values (depending on how long you were running 1.29.3) would have to be manually fixed by editing:
/usr/local/directadmin/data/users/*/bandwidth.tally
and setting the httpd= values as well as the total on the far left of th tally entires.
The webalizer stats will show the correct values so you can get your data from there.

So we need to edit ALL the bandwidth.tally files of all the users? Can you send a system-engineer? :p

[nobaloney]

Thanks, I didnt see the other thread,

Jeff, this one can be closed/merged when you have time please.
I merged the other thread into this one; I like the location of this one better :) .

Jeff

[martynr]

We are running Freebsd 6.2 and since yesterday DA 1.29.4. Our server still shows the wrong cumulative figures. Does anyone have any suggestions please?

[DirectAdmin Support]

Hello,

This was the bug in 1.29.3, fixed for 1.29.4 (released shortly after 1.29.3 once found)
http://www.directadmin.com/features.php?id=761

The fix in 1.29.4 will only correct the logging from the time it's upgraded. The previous inaccurate bandwidth will remain unless reset.

To reset the whole server to 0, use:
echo "action=reset&value=all" >> /usr/local/directadmin/data/task.queueNote that this will lose even the valid bandwidth data.

There isn't a real easy way to fix it if you've been using 1.29.3 for a long time. The correct way is to edit each users bandwidth.tally file. From the bottom, take the total number (from the last 1.29.3 entry), and subtract the previous days total. Keep doing that going up the file (ignore the non-tally lines). Then you'd also want to adjust the apache usage on the numbers to the right of the first = character, which is only cosmtetic to show the users the breakdown. Resetting to 0 is definately easier.

John