Finally i had some time to gather all the files together and make third release of my cacti monitoring scripts and graph templates.
Monitoring of application and server health is an important task, while trying to maintain high availability. Without monitoring you dont know what goes wrong and you do not know when exactly does it happen. Some time ago i realized that the graphs i used to use were not perfect. I searched the web, read a bit of documentation and decided to put together a set of simple scripts gathering key server performance metrics.
This bundle includes gathering scripts and graph templates for Memcached, APC, Apache2, Linux file system, Linux memeory, CPU and Network. It should cover all the most important aspects for a typical web server. Graphs are designed to match my expectations and to make analysis easier.
I still dont have postgres, sphinx nor mysql stats included but .... who knows .... maybe in version 4! :- )
New version has a few new metrics as well as some bugfixes. I also added better support for non pure Debian systems
To get the cacti graphs and scripts running you will need the following:
- Cacti (tested on 87e) to import template and draw graphs.
- LAMP stack with APC op code cache and Memcache - you do not need all if you are not planning to monitor them
- Root access to setup one file, if you want to monitor Memcached file descriptors count
- Latest Memcached version (tested on 1.4.4) to have evictions graph working
Monitoring scripts - Generic Installation Steps
- PHP needs to be allowed to call shell_exec()
- writable /tmp folder
- ls command copied to the same folder the scripts are in. Then ls executable needs to be setuid root as it will count files in proc folder. Stats will work without that command but opened file descriptors count will not work. Just cp /usr/bin/ls /statsfolder and chmod it to setuid and owner root (you dont have to use attached ls binary). It cant be a symlink nor bash script as setuid wont work.
- Installed and activated mod_ststus in apache2
- Sample apache2 config
<VirtualHost *:80> ServerName my.hostname.com # to allow access to the apache stats ServerAlias localhost DocumentRoot /var/www/statsfolder RewriteEngine On <Directory "/var/www/statsfolder"> Order allow,deny Allow from all AllowOverride All Options +FollowSymLinks </Directory> </VirtualHost> <IfModule mod_status.c> ExtendedStatus On <Location /server-status> SetHandler server-status Order allow,deny Allow from all #Allow from .example.com </Location> </IfModule>
Monitoring scripts - Additional Debian/Ubuntu specific tips
apt-get install sysstat
- access to commands: pidof, iostat, vmstat, free, top, cat, uptime, apache2ctl (/usr/sbin/apache2ctl)
- verified on Ubuntu 2009-10 and Debian Etch
Monitoring scripts - CentOs 5.4 and other systems might need additional steps
yum -y install httpd php mysql mysql-server php-mysql gcc automake autoconf php-pecl-memcache php-devel libevent yum install sysstat
- Install apc without apxs like this: pecl install apc
- Add some apache config
/etc/httpd/conf/httpd.conf NameVirtualHost *:80 <VirtualHost *:80> DocumentRoot /var/www/stats ServerName stats.centos.local ErrorLog logs/error_log CustomLog logs/access_log common </VirtualHost>
- Enable APC cache
/etc/php.d/apc.ini extension = apc.so apc.enabled = 1 apc.enable_cli = 1
- If you are on VM or fresh setup you might need to flush firewall iptables -F
- If you have SeLinux you might have to recreate security attributes
restorecon /usr/lib/php/modules/apc.so reboot
- If still does not want to load module just disable the SeLinux by
/etc/selinux/config SELINUX=disabled reboot
- Install Memcache
yum install libevent-devel wget http://memcached.googlecode.com/files/memcached-1.4.4.tar.gz gzip -d memcached-1.4.4.tar.gz tar -xvf memcached-1.4.4.tar cd memcached-1.4.4 ./configure make # to run memcache on console with debug ./memcached -u root -vv
- Copy /bin/ls to folder with stats scripts and set owner root + setuid
At this stage you should have all statistics scripts working via browser. You can see them in the browser typing http://some_fake_domain_to_vmware/stats_memcache.php etc.
You might need to create some fake load and traffic to see the stats results changing
Bundle also has latest view_memcache.php and view_apc.php scripts. These are slightly modified APC and Memcache scripts i got from net.
Linux - core subsystems activity
Linux - disk activity
Top disk saturation: if you have 4 drives the highest saturation is picked among them. This way you can have a quick insight into how busy is the busiest drive
Linux - network activity
USER CACHE HITS and FILE CACHE HITS look the same just different name.
Hope you enjoy it! Please let me know if you find any errors or fixes. also Let me know if there are things that could be improved, im really interested in your opinion! Im not cacti expert in any sense .... just trying to use it :- )
About the author
Hi, my name is Artur Ejsmont,
welcome to my blog. I am a passionate software engineer living in Sydney and working for Yahoo!
If you are into technology, you can order my book Web Scalability for Startup Engineers on Amazon. I would love to hear what are your thoughts so please feel free to drop me a line or leave a comment.