[rel] AWStats - For all you domains (availiable now)

[FarCry]

Well, a few months ago i said i would toy with awstats to get it going with DA, this i have done, and have been pretty much successfull. about 10% of domains dont generate pages, i have no idea why... but anyways, its very easy to install, very easy to get running.

I've tried to take all the pain out of installing awstats, it automaticaly creates config files, runs updates stats, and creates html files! there are 12 steps to install it, it does require use of SSH to install but is dead easy. All commands are provided if you dont know them, you should have no problems!

Requires:
- Domains/Logs
- DirectAdmin (well duh)

Get it now!
http://sourceforge.net/project/showfiles.php?group_id=87578&release_id=198816

Demo:
http://awstats.spinnahost.com/wa-league.com.html
http://awstats.spinnahost.com/clanmayhem.org.html
http://awstats.spinnahost.com/spinnahost.com.html
http://awstats.spinnahost.com/wieisdemol.com.html ( moved to another server )

Warnings:
Can take along time to parse logs depending on the amount of domains/log file size. Initial update can take quite a while to do.

URGENT: IF YOU HAVE THIS RUNNING, DELETE YOUR CRONTAB, RECREATE AS FOLLOWS. IF YOU DONT, YOU MAY TAKE YOU SERVER DOWN.
minute: 0
hour: 0
rest: *
cmd: php /etc/awstats/domainlist.php


Changelog:
0.11 -
Subdomain's stats are now parsed correctly (no more errors)!

[FarCry]

Feel free to download, now fully functional

[ProWebUK]

the very basic commands to get you moving :)

# wget http://unc.dl.sourceforge.net/sourceforge/datools/awstats59-md.tar.gz

(That download is slow... getting 7kb/s from it.. if you want i will mirror it..)

# tar -zxvf awstats59-md.tar.gz

from there use the readme.txt and install.txt :)

Chris

[FarCry]

actualy if you just goto the link i provided you can get a mirror that is close and fast :)

I would not just untar it anywhere as its not extracted to its own self contained directory.

Hence, here is my idea of how you should download (is pretty much a changed about version of the first steps in the install.txt)

# mkdir /etc/awstats/
# mkdir /etc/awstats/data/
# chmod 777 /etc/awstats/ -R
(recursive so you get the data dir too)
# cd /etc/awstats/
# wget [link address you get at the top of the source forge download page when you choose a mirror]
# tar xzvf awstats59-md.tar.gz

open a new shell window, to use when following the install file, then
# vi install.txt
(or)
# pico install.txt
and then follow instructions from there.

[ProWebUK]

Originally posted by SpinnaHost-Mark
actualy if you just goto the link i provided you can get a mirror that is close and fast :)

No UK mirrors... and the others were extremely slow, think i might have chose a busy American time :D

[gmilazzo]

ok i have my cron job set
min *
hour 0
the rest *

command php /etc/awstats/domainlist.php

and i get

/bin/sh: php: command not found
what did i do wrong

[FarCry]

Login to shell and try
# php /etc/awstats/domainlist.php


If this gives the same response:
try the full php path... (to find see below)
# which php
/usr/local/bin/php

if this fails, make sure php actualy exists there by running it:
# /usr/local/bin/php

then just type in:
<?php echo "hello\n\n"; ?>

then press ctrl-d to execute it...

If you get nothing or a command not found you have a few problems....


if it does exist:

edit domainlist.php and add the following line as the VERY first line (before <?php even)
#!/usr/local/bin/php
(or where php was found with which)

make it executable:
# chmod 755 /etc/awstats/domainlist.php


try it again with the following command:
# /etc/awstats/domainlist.php

If this works, then update your cron to be:
command: /etc/awstats/domainlist.php


If you get any more problems, create a bug report on the DA Tools sf.net project page: http://www.sf.net/projects/datools/

[gmilazzo]

well i think i found the problem i set awstats up ad root now when i do i from admin it say permission not allow do i have to start all over again.

[FarCry]

you dont have to start again, just login as root and:

# chown admin.admin /etc/awstats/ -R
# chown admin.admin /home/admin/public_html/awstats/ -R
# chmod 777 /home/admin/public_html/awstats/
# chmod 777 /etc/awstats/data/

[teedee]

Hi,

I do not have the path /home/admin/public_html/ on my server.
I have tried setting this up using /home/admin/domains/mydomain.com/public_html/awstats (where mydomain.com is my main admin account) but without success.

Am I supposed to have /home/admin/public_html/ by default ?

[FarCry]

/home/admin/public_html/ should be a syslink to your /home/admin/domains/domain.com/public_html.

To create it:
# cd /home/admin
# ln -s ./domains/<yourdomain>/public_html public_html


ln info/syntax/usage can be gained by ln --help

First section here:
--
Usage: ln [OPTION]... TARGET [LINK_NAME]
or: ln [OPTION]... TARGET... DIRECTORY
or: ln [OPTION]... --target-directory=DIRECTORY TARGET...
Create a link to the specified TARGET with optional LINK_NAME.
If LINK_NAME is omitted, a link with the same basename as the TARGET is
created in the current directory. When using the second form with more
than one TARGET, the last argument must be a directory; create links
in DIRECTORY to each TARGET. Create hard links by default, symbolic
links with --symbolic. When creating hard links, each TARGET must exist.

[gmilazzo]

[admin@215 admin]$ chown admin.admin /etc/awstats/ -R
[admin@215 admin]$ chown admin.admin /home/admin/public_html/awstats/ -R
[admin@215 admin]$ chmod 777 /home/admin/public_html/awstats/
[admin@215 admin]$ chmod 777 /etc/awstats/data/
[admin@215 admin]$ /etc/awstats/domainlist.php
bash: /etc/awstats/domainlist.php: Permission denied

[FarCry]

Try this:

[root@localhost /]# chown admin.admin /home/admin/public_html/awstats/ -R
[root@localhost /]# chown admin.admin /etc/awstats/ -R
[root@localhost /]# chmod 777 /etc/awstats/data/ -R
[root@localhost /]# chmod 777 /home/admin/public_html/awstats/
[root@localhost /]# chmod +w /home/admin/public_html/awstats/*
[root@localhost /]# exit
[admin@localhost /]$ cd /etc/awstats/
[admin@localhost awstats]$ chmod +x domainlist.php
[admin@localhost awstats]$ php domainlist.php
... and away it goes ...


And keep using "php /etc/awstats/domainlist.php" as the cron command, if all works well.

[teedee]

OK, sysmlink created but......

php domainlist.php runs OK for me but does not output anything to /home/admin/public_html/awstats (chmod 777) All I have there are the original files untarred from html.tar.gz

My config file looks like

<?php
$loglocation = '/var/log/httpd';
$outputlocation = '/home/admin/public_html/awstats';
?>

[FarCry]

are you getting any output from the program when you run it via command line?

[teedee]

it runs for about 2 seconds, (about 15 domains on this server) and does not give any output on command line but does create files in /etc/awstats/ for each domain

awstas.domain.com.conf

[FarCry]

strange,

try
# chmod +w /home/admin/public_html/awstats -R

OR

just change your config file to be the "full" path to the domain.
eg: /home/admin/domains/spinnahost.com/public_html/awstats

[teedee]

tried both, still the same result.... :(

[FarCry]

hmm, not good.

just try the chmod again... this time as root?

# chmod 777 /home/admin/public_html/awstats

[teedee]

nope, still not working.....

[FarCry]

URGENT: IF YOU HAVE THIS RUNNING, DELETE YOUR CRONTAB, RECREATE AS FOLLOWS. IF YOU DONT, YOU MAY TAKE YOU SERVER DOWN.


minute: 0
hour: 0
rest: *
cmd: php /etc/awstats/domainlist.php

[gmilazzo]

when i shell in as admin and put php /etc/awstats/domainlist.php
it runs normal but when it cron's i get this error
/bin/sh: php: command not found

[nebukazar]

Hi,

just wanted to know if there's a way to make the some of this script parameter working.

Like : http://awstats.mydomain.com/cgi-bin/awstats.pl?config=sub.domain.com&output=allhosts

Thanks :)

Regards,

[FarCry]

arrg, i didnt distribute the version that has that?!

ok, i'll make a release on sf.net before i goto bed, and post here once ive done it.

[koos]

Why do i have to delete my crontab...
I did use mrtg with cronjobs.

[nebukazar]

any news about the other functions not included ? Did you put the version that has those functions on sf ?

[FarCry]

Things have been a bit hectic, I've just got home so now its time to code DA stuff!

[FarCry]

I'm coding in quite a few new features, and making this MUCH faster, so it might take 1 or 2 days longer, but it will be done by monday (your sunday night) as i start a new job then :)

[IsaackRasmussen]

Originally posted by teedee
nope, still not working.....

Hi Teedee,

I experienced the same problems as you. In the config.php for the log location variable. You should add "Domains" to the path, apparently that's where the real logs are.
After that, the script will run and output screen text.

The only problem is, that I can not go back in the months. It tries to run awstats.pl which is not accessible, according to the setup.

[FarCry]

im trying to get time to finish this script, so that everything works 100%, please bear with me :)

[IsaackRasmussen]

Great! Looking forward to it!

[FarCry]

i might put out a release before this weekend that will contain the code to generate all static pages for all domains (slow) that there are logs for. I have already written this, but wanted to code a script that would get all the user domains for the reseller/admin installing the script, as this would use alot less cpu when there are deleted domains.

[netmotiv8]

any news on the release?


Neil.
:D

[FarCry]

i've got a release sitting here waiting to go which i want to just run for 1 more day to make sure its not going to kill your server!

It includes a few new features, including the ability to have all the domains you want to parse in just one txt file (domains.txt) or just use every single one on the server :)

If there is interest i can move this txt file to mysql?

[FarCry]

Seems to be working without any troubles!

I'm going to head home in about 20 mins, i'll write up the documentation (much better this time i hope) and then make a new thread when its done. It will take me about 1hr to get home, and then maybe 1-2 hours to make the release!

Sorry for the long delay. Thx to Two-A-T who got my arse moving on this :P

[FarCry]

Changelog for the next release:

- Feature: Added a domains.txt file which contains domains to be parsed if you dont want to run over the whole servers logs.
- Feature: Can use the log files on the entire server or just the specified domains.
- Feature: Installer. Install script, downloads and installs the latest version of awstats-md.
- Bugfix: Static linking of all pages, now all linked pages function 100%.


I am also thinking of making the graph sizes bigger (4-5x higher).

[IsaackRasmussen]

Sounds great Mark!

[FarCry]

Still working on the docs :)

[FarCry]

Well, i didnt get all the documentation or the update/install script tommorow. Will be done first thing :)

Here is the release without any docs, just the files here :) just follow the install instructions at the top of this thread.
https://sourceforge.net/project/showfiles.php?group_id=87578&package_id=101610&release_id=205726

[pilpelet]

Hi ,

Also i think i found a glich in the installation ,
I dont have the file awstats.pl in the relatrive cgi-bin or elsewhere .

[itman21]

Hi

I have installed Advanced Web Statistics 6.0 (build 1.704) with DA and it seems it works fine except for time thing..

The server time is:

[root@server root]# date
Fri Feb 13 09:03:35 MYT 2004


However, when I click "update now" button and refreshed time shows :

12 Feb 2004 - 17:04 Update now

Doesn't awstats take time from the server ? The refreshed browser time is LA time vs. Malaysian time.

Any clue to it ??

Sun Joo

[IPaddress]

Why not dynamic htmls, I like dynamic statics,

Dario

[xavipolo]

I've searched about install AWStats at those forums:
Some information about scripts to execute after or before domain is created (needed to install awstats for new domains)
some information to install awstats (with a lot of changes in every post)

But I did'nt find a clear How-To. No one change Webalizer for AwStats ??

Why DirectAdmin don't support (by default) a great and free software ?

Thanks

[bonnmac]

Was looking to install this because many of my clients have asked for it, but when going to awstats.mydomain.com I get a forbidden message.

I followed the instructions word for word, any ideas as to what I may have missed?

[sbchasin]

Originally posted by bonnmac
Was looking to install this because many of my clients have asked for it, but when going to awstats.mydomain.com I get a forbidden message.

I followed the instructions word for word, any ideas as to what I may have missed?

Try adding the following to a .htaccess file where you have installed awstats:

Options +FollowSymlinks +ExecCGI
AddHandler cgi-script cgi pl
DirectoryIndex awstats.pl

That should do it

[RealNames]

Just read this entire thread as looking for something like this instead of weblizer. The problems most have getting it to work seem like they would be simple to solve with some easy step-by-step instructions, written from the viewpoint of a non-programmer to a non-geek.

It would seem like all the needed steps could be said concisely on perhaps a page of instuctions. If that was done I would definately get this myself but with all the issues (much more experienced) users have I am extremely reluctant to try it as I am sure it would not work right after reading this thread. Just my 2 cents.

[twhiting9275]

hrrrm
changed everything to /home/wolf/public_html/awstats and /var/log/httpd/ as supposedly I was supposed to (??) (don't use the admin user).
untarred the stuff properly (nothing shows up in /home/wolf/public_html/awstats, but a bit shows up in the /home/wolf/public_html/awstats/* directories)
ran the php code
nada
even data isn't being shown
nothing's changing at all.

[Seth]

I downloaded this but I can't seem to find the install.txt file?

[Seth]

does anyone have the install.txt file?

[a-arse]

I'm new to DA so i apologise for my ignorance.
How do i create a subdomain ?

1) Create a sub domain on the servers primary domain called
awstats.


I tried creating the domain 'awstats.domain.com' ( from the admin panel ) but that doesn't seem to make any form of directory.

Thanks and Sorry :)

[Seth]

In your admin panel goto your USER DOMAIN and create the subdomain there.

Do you have the install.txt? Can you mail it to me?

[a-arse]

thanks for the quick reply,
is that through 'DNS Administration' ?
that doesn't seem to be the way..

Sure i'll email you install.txt but you can download it from the SourceForge link in the first post :)
What's your email ?

[Seth]

sales@sv1.us Thanks, it wasn't in what I downloaded.

Subdomain:
webadmin » Home » PRIMARYDOMAIN » Sub Domains

Hope that helps also checkout:
http://www.site-helper.com/

[a-arse]

I have emailed you,

Which panel are you finding this from?

Admin Pane
Reseller Panel
User Panel

?

Thanks

[Seth]

User Panel

[a-arse]

a-hah!
Thanks!
i was getting confused :)

[Seth]

:D

[Seth]

Has anyone gotten AWStats working? I cannot get it installed correctly.

[netswitch]

runs smooth here, altough, i had to taéke a bit of time to get it up and running.

I had problems with the latest release on sourceforge, I downloaded the previous one and it went fine.

[FeBox]

Ok, here it goes.

I re-worked some things, made it more simple, and fixed a big bug, the domainlist.php was looking for a aws.pl, which doesn't exist. The aws.pl is AKA awstats.pl.

Run the following commands as root:

# cd /home/USERNAME/public_html
# mkdir awstats
# cd awstats
# wget http://www.febox.com/DA-awstats.tar.gz
# tar xzvf DA-awstats.tar.gz
# cd ..
# chown -R USERNAME.USERNAME awstats
# cd awstats

# pico common.conf
// If you dont have pico, then use vi
// Edit USERNAME to the username that you will be using
// Save and Exit the file.

# pico config.php
// Edit USERNAME to the username that you will be using
// For the parsetype, set it to whichever you want, all or domainlist
// Save and Exit the file.

// IF YOU CHOSE DOMAINLIST THEN EDIT THE domains.txt

# su - USERNAME
# cd public_html/awstats
# php domainlist.php


Access it via http://www.domain.com/awstats/domain.com.html

This can get a LOT more fancy, but I whipped this thing up quickly. This would also be very very easy to slap into a bash script. I'll see if I have time tommorow to put it into one.

Let me know if you have any questions.

[twhiting9275]

Here's a very simple script based on what was just given:


USER="insertuser"
EDITOR="pico"
cd /home/$USER/public_html
mkdir awstats
cd awstats
wget http://www.febox.com/DA-awstats.tar.gz
tar xzvf DA-awstats.tar.gz
cd ..
chown -R $USER.$USER awstats
cd awstats
$EDITOR common.conf
$EDITOR config.php
$EDITOR domains.txt
su - $USER
cd public_html/awstats
php domainlist.php


A couple of things wrong, which have yet to be corrected in any version of this:
Firstly, when using "domainlist", you get a bunch (and I do mean a bunch) of Malformed UTF errors. There's a good 1-200 of those errors so I'm not going to paste those.

Secondly, "all" does not work, at least for RHEL3. I couldn't get it to work under 9 either, so its probably NOT the OS. Error follows:


Warning: Syntax error line 5 in file './aws.wolfstream.net.conf'. Config line is ignored.

Error: LogFormat parameter is not defined in config/domain file

Setup ('./aws.wolfstream.net.conf' file, web server or permissions) may be wrong.
See AWStats documentation in 'docs' directory for informations on how to setup aws.


It's generating the html stuff, but it's not generating them properly, obviously.

[FeBox]

You need to change domainlist.txt to domains.txt.

Also, yes I know it spits out the Malformed UTF-8 errors, however it generates everything fine as far as I've seen.

I never tested it with All. I'm testing it now actually.

[twhiting9275]

Changed, thanks for pointing that out.
I had that in there because as "all" it doesn't work oddly enough. I'd bet that once I generated the configs with "domains", it'd read from "all" just fine, or the conf's that've been generated.
I'm not sure why it does it, but I've had that problem with "all" since I started pokin around.

[FeBox]

I fixed it. :)

Cleaning up and repackaging. I'll let you know when its done.

[FeBox]

Ok, its updated.

I have also prepared this automated script.

The reason why I didn't include an su - at the end is because it doesn't execute any of the other commands. So I worked around it.

We should also find out whats causing the Malformed UTF errors...they are quite annoying... :P


#!/bin/bash
############################################################
# AWStats installer for DirectAdmin #
# Prepared by Anthony Drimones of FeBox, LLC. #
# Contact: anthony [@] febox.com #
############################################################
EDITOR="pico"
echo "Enter what username you would like Awstats installed on:"
read USER

file1="/home/$USER/public_html/awstats/common.conf"
file2="/home/$USER/public_html/awstats/config.php"

cd /home/$USER/public_html
mkdir awstats
cd awstats
wget http://www.febox.com/DA-awstats.tar.gz
tar xzvf DA-awstats.tar.gz
cd ..
chown -R $USER.$USER awstats
cd awstats
perl -pi -e "s/USERNAME/$USER/g" $file1
perl -pi -e "s/USERNAME/$USER/g" $file2
echo "Would you like to use all the domains or a list of your choosing? Type 'all' for All and 'list' for list"
read CHOICE

if [ "$CHOICE" = "all" ]; then
php domainlist.php
chown -R $USER.$USER *
echo "Awstats complete!"
echo "For future updates, please su - to $USER and run the command 'php domainlist.php'"
else
perl -pi -e "s/= 'all'/= 'domainlist'/g" /home/$USER/public_html/awstats/config.php
$EDITOR /home/$USER/public_html/awstats/domains.txt
php domainlist.php
chown -R $USER.$USER *
echo "Awstats complete!"
echo "For future updates, please su - to $USER and run the command 'php domainlist.php'"
fi

[twhiting9275]

For this to effectively work through cron, you need to make a couple of adjustments to domainlist.php
We'll take my setup for example

$username="wolf";
chdir ( '/home/'.$username.'/public_html/awstats/' );
include ( 'config.php' );

You've got to have the $username defined before the chdir, and the config after that. Otherwise, the script will bail when trying to run it.
Looks like everything works as it should though, great job.

[FeBox]

Thanks and nice job to you too. :-)

Maybe this will speed up the process of getting it officially released with DA. :D

[thephantom]

why can i only run /etc/awstats/domainlist.php if i logd in as root user?? i like to have an cronjob that update the stats before logrotation at the end of the month. can some help me please how to do that (when do an cronjob as admin i get all error "permission dinied") as root user everithing works fine.

[jmstacey]

I think the awstats directory belongs to root so any lesser user will not be able to do anything that involves it unless permissions / ownership is changed.

[itman21]

I installed awstats for all the websites (automatica installation for each site created) and heard that we could only do 50 sites per server (P4, 512Ram, 80G IDE HDD, 100M shared with 5M dedicated).

Anybody running more than 50 or 100 or 200 sites ?? How is the servers stress level ??

[intelliot]

Is there a simple guide to doing this with the latest versions of DA and awstats? Also does the server load look for those of you who have installed this?

[jmstacey]

Server load is normal. It only spikes a little when a user updates his statistics.

This is a DA Plugin. At the admin level install the plugin in the plugin page, then in the user level choose the domain you want to install it on, then select the link to install it, view stats, or password protect the directory.

No instructions are really needed.

If your not the admin, then you must contact you admin and see if you can convince them to install the plugin on the server.

[intelliot]

without reading through 4 pages of posts, can someone tell me if there is still any problem with "about 10% of domains dont generate pages, i have no idea why"

also, the link on the first post goes to AWStats Multi Domain 0.59, while the current version of awstats is 0.61, is it not?

[Avenueduweb]

Thank you very much for this script ! It's run very good in my server, but i've got the problem of "Malformed UTF-8 errors" and i don't resolve it, have you got a solution ? Thanks. Bye.

[plugin]

its rather easy, UTF errors have something to do with languages, so there are errors in the text.

Open the search_engines.pm and robots.pm and get the errorline and look for things like é á ä ë etc, and make just a and e from it. Then its solved :)

Dont delete whole lines, then it wont work anymore :)

[pinooo]

Hi,

I am a newbi to DA...

How can I add this as a plugin?

Simply copy and paste the script above in a file and upload it to the plugin page. And run it ?

Or am I wrong on this one....

Maybe an Idea to build a plugin link collection page on the forum...


Cheers

[inetworx]

Originally posted by pinooo
Hi,

I am a newbi to DA...

How can I add this as a plugin?

Simply copy and paste the script above in a file and upload it to the plugin page. And run it ?

Or am I wrong on this one....

Maybe an Idea to build a plugin link collection page on the forum...


Cheers

That's my big question to!

I tried everything to get it working but there's still noting.

It's impossible to install Awstats on every domain on my servers. 2049 accounts, that's a lot of work.

Is it so difficult for DirectAdmin to ad it to DA as an options/plugin so it can be accessed true DA

DA is a great panel but there's a lot to do in offering options. A whole lot!

[drmike]

Install and setup instructions here:

http://www.directadmin.com/forum/showthread.php?s=&threadid=3901&highlight=awstats

A simple search would have found it for you.