webstats - serverwide logs with analog and report magic

home

[documentation]

to get help for report magic please ask in the mailing list. i hang arround there usually....

quick installation:

this version is tested with analog 5.01 and report magic 2.10.

1)

unzip, untar the webstats.tar.gz into a temporary directory

edit the Install.PL

i assume u used the default paths...

create a directory

/etc/webstats

and from the directory /usr/local/webstats-0.x/configfiles/serverwide put

default.cfg
default.ini
webstats.ini

into it.

edit these files. follow the instructions in the comments of these files.

2)

from the directory /usr/local/webstats-0.x/configfiles/vhosts put

analog.cfg
rmagic.ini

into the webstats directories of you domains (you specified this directory in point 1) )

edit the files. follow the instructions in the comments of these files.

3)

from the directory /usr/local/webstats-0.xl put webstats

into /usr/local/bin or a smilar location

from the directory ./scripts/php put

all files

into a directory accessible by your webserver.

4a)

run webstats -a

4b)

correct and report all the mistakes i made in
this document and which let analog and report magic spit out a lot of
error messages when you did 4a). run 4a) again.

5)

point your browser to the webstats directory of one of your virtual
domains to see the statistics of these domain.

point your browser to the webstats.php file to see a list of all of your
virtual domains.

6)

if everything is fine add "/usr/local/webstats -a" to your cron jobs...

7)

after successful installation of webstats, please have a look at the code of
the "report_success" script and if you are convinced that it doesn't do
anything malicious run it with your name as argument. this will give me an
overview of the usage of webstats... thanks!

WHAT IT IS

webstats is a front/backend for report magic.

it helps managing the statistics created by analog and report magic for
multiple virtual domains hosted on one server.

set up properly it creates for every virtual domain you add to your site
automagically analog and report magic reports.

additionally is included a php page which shows all existing virtual domains
on your server with links to the appropriate site, web statistics and netcraft
surveys.

this version is tested with analog 5.01 and report magic 2.10

HOW IT WORKS

webstats consists primarily of two parts. the perl script "webstats" and the php page "webstats.php".

these two scripts read their configuration variables from /etc/webstats/webstats.ini (here are mainly configuration options like paths and filenames located) and from /virtualhost_directory/wwwdirectory/webstatsdirectory/analog.cfg and rmagic.ini. these two files provide the virtual domain specific configuration and refer for all server wide options to /etc/webstats/default.cfg and default.ini.

when you run webstats it

  • looks into the directory where your virtual domains have their home dirctories
  • dives into each directory- reads the domain specific configuration
  • loads the serverwide configuration- runs analog for every virtual domain
  • runs report magic for every domain

(Note: i included an option to combine/sort/clean your analog logfiles and put
the result into a file "access_log_CURRENTDATE", because it's the way i like
it. look into the webstats.ini for more details)

when you call webstats.php it

  • looks into directory where your virtual domains have their home dirctories
  • dives into each directory
  • reads the domain specific configuration
  • extracts informations like name of the domain and base url
  • creates a webpage with a list of all your virtual domains, and for each domain a link to the domain itself, to the webstats of the domain, to the netcraft survey for this domain
HOW TO USE IT?

i think, the best way to explain this is to show you my setup:

here are the relevant parts of my directory structure:

         /
         +--/etc
         | | 
         | +--/webstats (serverwide configuration)
         |
         +--/home
         | |
         | +--/vhosts (home directories for virtual domains)
         | |
         | +--/virtual_domain_1
         | | |
         | | +--/logs (logfiles for virt_domain_1)
         | | |
         | | +--/www
         | |    |
         | |    +---/webstats (webstats and configuration for virt_domain_1)
         | |
         | +--/virtual_domain_2
         | |
         | + ...
         |
         +--/var
            |
            +--/www
               |
               +--/serverstats (webstats.php with list of all virtual domains)
         

 

here's the setup:

i run apache with name based virtual hosts. my virtual domains arelocated in /home/vhosts. the naming convention is, that the directory has thename of the second level domain, i.e. "patsplanet.com" is located in directory/home/vhosts/patsplanet. when there are more then one top level domain i hostfor a particular second level domain (i.e. patsplanet.com and patsplanet.net)then they will be served from the same directory.

below the home directory for the domain is a directory "logs" where thelogfiles for just this domain are going to. because i use the advancedextranet server which comes with my distribution(mandrake 8) i end up with a lot of small logfiles in here. but later more about this..

below the home directory of the virtual domains is a directory "www" which isthe www root of the domain. below that directory is a directory called "webstats".

in this directory are placed initially two files - analog.cfg and report magic. in its virgin state they have placeholders as you can see in the files provided in this package. these placeholders are replaced with the actual values by a script which runs after creating the virtual domain users..

because in /home/vhosts are only virtual domains, the job for the webstats script is quite easy. traverse the directories, read out the rmagic.ini to get the belonging domainname and run analog and rmagic.

before this happens i let the script combine, sort and filter the log scripts. all this has no influence on the work of analog or report magic but helps eyeball searching of the logs by having a log for each day and excluding (in
my case) the http checks from netsaint, which i use to monitor my server. you
can read how this works and how to adjust this part of the script to your likening in the comments of the subroutine "combinelogs" in the perlscript...

the big advantage of this setup is, that whenever i create a unix user and place its home directory in /home/vhosts apache is automatically responding to http requests to the domain "unixusername.whatevertopleveldomain". additionally creates webstats the webstats ;) , because it traverses all directories below /home/vhosts, so the newly created one is included...

the small php script "webstats.php" is doing the same what webstats perlscript is doing in respect of the traversing, but it does this to display the available domains and to create links to the according statistics (and as a bonus to the site itself and to the netcraft report of the site). so, after creating the unixuser the virtual domain automatically appears in the list...

addiionally i created some directory aliases which are inside the virtual host container of my apache config file (have a look at the apache config snippet in the folder "stuff" in this package. as a result every virtual host as automatically two folders in its www root - sitecss and siteimages. in this folder i place my logo and my css files and refer to them in the default.ini - the serverwide configuration file for report magic. this way, all reports of virtual domains of hosted by me have the same look and a little advertising for me at the bottom.

to see the relevant parts of my http.conf for mass dynamic virtual hosting look nto the "stuff" subdir in the package...

and thats it... to see it life go here to.

WHAT REMAINS TO DO?

although i tried to make the script as flexible as possible i'm sure i overlooked a specific problem with /your/ server topology. so, please send me your complains or ideas and, if time allows, i'll try to incooperate them in the next version.

PAT

 

 

 

 

 

 

 

 

 

© pat erler 2001

SourceForge Logo