CPU Limit

[interfasys]

I'd love to be able to set a CPU limit on some accounts, so that those don't take a server down.

[loopforever]

This isn't possible, since CPU load changes based on a number of factors. For example, you may have 100 users on your machine, each using MySQL - but MySQL runs as a single daemon, queuing up queries and executing them as a single user. We can say "Hey, MySQL is driving the load up." but we can't say "Hey, demouser is driving the load up." You can view MySQL processes, but this won't assist you in determining which user is increasing the load.

PHP and Apache is also another great example. You can see that Apache is driving the load up, but you can't see what script is doing it. The process just appears as Apache, doing something, which was commanded my someone, whom we cannot determine.

So really, it's not possible to limit CPU load on a per user basis.

[ProWebUK]

What it would be possible to do is install a process resouce monitor which monitors resource usage of applications then using variables set by you recieve email alerts etc.

If you wanted something like that i would be happy to put up a how-to guide

Just as a note - that would be server specific not user specific due to the reasons listed above.

Chris

[interfasys]

I think I saw some scripts that could could alert me in case Apache was overloading, but that script would let me know which domain was causing it, so I thought it would be possible to throttle it.

[S2S-Robert]

One of my users actually crashed my apache somehow with 150 apache processes and rising, so if you could tell me which scripts you found that would be great.

[interfasys]

Head over to WHT. In the service offer forum, you'll find script and services for sale that will help you (sorry for not having more info right now).

[ProWebUK]

Well some news that you may be suprised by

open httpd.conf with your fav text editor search for the line:

#ExtendedStatus On

once you find it uncomment the line and add the following:

<Location /serverstatus>
allow from YOURIP
deny from all
SetHandler server-status
</Location>

you should now have

ExtendedStatus On
<Location /serverstatus>
allow from YOURIP
deny from all
SetHandler server-status
</Location>

In your httpd.conf

Ensure you replace YOURIP with your actual IP address (this is so only you have access to the page - all other IP addresses will be disallowed access)

once your done run:

# service httpd restart

now go to:

yourdomain.com/serverstatus
SERVERIP/serverstatus

you will see a page showing each vhost whats happening etc.... aswell as CPU usage for each vhost.. so in reality... this feature is possible, although if you have followed this you will have noticed its off as default for performance issues - basically you have this but it will slow apache down and affect its performance.

This should come in handy when your servers resources seem to have shot up though

Chris

[l0rdphi1]

Very, very interesting. If only the 'CPU' column could be monitored, a script could VERY EASILY keep a running total per domain. Then you could "limit X CPU per minute" or something.

Also, if you could do this on a per-domain basis, you'd only set it up for people you know are high CPU eaters.

Hmm...

[ProWebUK]

Originally posted by l0rdphi1
Very, very interesting. If only the 'CPU' column could be monitored, a script could VERY EASILY keep a running total per domain. Then you could "limit X CPU per minute" or something.

Since your the php man...... would it be possible get that area out using php... dont forget you can use php in shell scripting..... so if its possible to split that part of the code up with any lang and basically leave with a domain variable with the CPU usage of that domain (could even interact with a database here)

The only thing i dont like about the idea is the performance issues... but then i have no idea what kind of performace effects this causes.

Chris

[l0rdphi1]

Yeah, that is all very feasible.

Also found this - http://httpd.apache.org/docs/mod/mod_status.html

"if you have lynx installed, you can also get a server statics report from the command line by running the command apachectl status, or, for the extended status report, apachectl fullstatus"

I need to isntall lynx and check that out


As for performance, I'm going to leave http://your.server.name/server-status?refresh=1 on for a bit and see what happens

[l0rdphi1]

Well, I have been invoking this every second for the past hour and I can say I'm seeing absolutely no increase in load. I think I may go a head and start playing with a program to monitor CPU on a per-domain basis.

[ProWebUK]

Performance issues wont neccessarily mean server load.... it could be page loading times (since each time a page is accessed its on that file etc.. if you have it running every 1 second for the application you are planning on developing this could cause problems there.... may want to do a bit of comparison between load times with pages etc before assuring yourself its not needed........

Chris

[l0rdphi1]

Hmm. Good idea.

Hopefully I can get apache to generate this stuff directly in a log file, but we'll see.

Thanks.

[l0rdphi1]

I decided to give a go at parsing the httpd data, and here's what I came up with:

http://www.liquenox.net/cpu.monitor..../server-status

Replace 'http://your.server.name/server-status' with the URL to your stats page. Remember, this is only showing the totals for the data that would normally show on the page. Running totals over a set period of time is the next step, I guess.

[l0rdphi1]

Yes, and below is the source I'm running ...removed.. VB screws it up too bad...

http://www.l0rdphi1.com/junk/cpu.monitor.phps

[S2S-Robert]

Well this is looking tremendously good! The page is however still not working when I C&P, it does work using your php script.

And if this script is not a resource hog, you could just let it run every second and print out the results 'realtime'. Or run a graph next to it.

[l0rdphi1]

I wish you could get Apache itself to print to a log with the data found on the server-status page. It's impossible to get *everything* the way I'm doing now, even running the script every second. Seems silly that Apache wouldn't let you do that.

[S2S-Robert]

Perhaps there is some apache documentation that allows you to do this?

But even if you're not getting everything, it still provides a good way of seeing which sites take too much cpu time etc.

[l0rdphi1]

Yeah, even setting it to run every 5 minutes would give you a good idea. Haven't seen any docs on this (besides the one I linked), so I'll just go a head with the current method.

[ProWebUK]

Originally posted by S2S-Robert
And if this script is not a resource hog, you could just let it run every second and print out the results 'realtime'. Or run a graph next to it.

The graph could be done with mrtg, you just need the number (X) where X would be X% CPU usage...... thats not really wanted or needed for this though and just something that could be a nifty script for later on

lordphi: would it be possible to intergarte with mysql.... leaving the results to be used more easily

======================
Domain || <CPU USAGE>
======================
test.com || 0.01
your.com || 0.04
======================

etc

from there a shell script could simply be put in place with a 'max_cpu' variable.... a quick run through that table for a load higher than the var is specified as sends an email out to the admin (could also be a variable for the user to specify) possible admin@<DOMAIN_OVER_USAGE> and even suspend the account in directadmin for 2 minutes or a specified short period of time.. for the last idea we would need a command to suspend / unsuspend manually from command line

It may not be the best of things but I think its the best we could be getting without having individual daemons running for each domain.

Would make a brilliant 3rd party script if you ask me.... and it should work for all panels (at least everything but the domain suspension / unsuspension if thats possible )

Chris