PALM v3 - bundle of Php Apache Linux and Memcached Cacti scripts (APC included)
Finally i had some time to gather all the files together and make third release of my cacti monitoring scripts and graph templates. Monitoring of application and server health is a important task while trying to maintain high availability. Without monitoring you dont know what goes wrong nor when. Some time ago i realized that some of the graphs i used to use were at least not perfect. I searched the web, read a bit of documentation and decided to put together a set of simple scripts gathering key server metrics of average web server. I still dont have postgres nor mysql stats included but .... who knows .... maybe in version 4! :- )
New version has a few new metrics as well as some bugfixes. I also added better support for non pure Debian systems
Web Security - Part 1 - Google Hacking
Doing security reviews from time to time of web applications you can find real beauties. You see GET variables in eval, system, include and all sorts of things you would not even imagine that could run live for years without anyone noticing it :- )
I dont really have great english nor writing skills but i thought i will try to write a few parts of the security article just to give something back to the community. Its not really rocket science nor my invention or discovery. Its just day and weeks and years spent working with web apps and reading here and there. What i hope is to write something you will enjoy and maybe put some new spin on some issues.
PHP display_errors => You will be Google-Hacked for sure
Mysql linux shell commands to get tables list and basic statistics
Mysql is so common that its good idea to have some command line scripts or tricks. Sometimes you need to execute some SQL queries on set of tables or all of them or get stats or fix tables etc.
Its good to be able quickly get list of tables and basic stats.
You can make bash script of it or just type in the command line:
# Put your database name here export SET dbname='drupal'; # List all tables in the database echo "show tables;" | mysql -u root $dbname |grep -v '^Tables_in_' > tables.txt # Show tables status (tables' engine, rows count, table size, index size, collation, comment) echo "show table status;" | mysql -u root $dbname > table_status.txt # Get each tables indexes details
How to wget to mirror a website without html file extensions
Wget is a great and powerful tool but it does have some hidden features that are not really common in linux world.
For example it has set of default settings that can be altered via some misterious .wgetrc (good luck on mac!).
How to mirror a website onto local machine?
wget -m -k -K URL
This will crawl all pages in the site, save them on disk and rewrite urls to relative ones so you can browse them locally.
Its cool but it also adds .html extensions so index.php becomes index.php.html. But in my case i wanted to crawl the site to flatten it
and make static html pages. Site was like 4 years old, and i wanted to make it easier to deploy and safer than some freaky old crappy php app sending
emails and god knows what else.
Review - Linux System Programming
I guess its not easy to write an easy to read book about linux system programming but this one is not bad at all.
I have to admit that it is a bit 'dry', tons and tons of system calls explained with parameters and return values
and error codes going on and on and on :- ) But thats the point. You want to know how to write system apps on linux?
well you have to go through it and youd better enjoy!
But book also contains some really nice chapters describing file system handling, cache, memory and process schedulers.
How to compile memcached 1.4 on MAC
I am using Mac just since January and im still not so familiar with it. If you want to install latest memcached on Mac OSX you will need libevent.
Download libevent from this site:
http://www.monkey.org/~provos/libevent/
Then unpack, configure and compile it. Also install it in /usr/local
./configure --prefix=/usr/local make make install
Then obtain latest memcached from:
http://www.danga.com/memcached/download.bml
Then unpack, configure and compile. You need to point to the same folder so configure could find the library.
unpack ./configure --with-libevent=/usr/local/ make ./memcached -u root -vv
And thats all.
PALM - bundle of Linux system, Network, Apache2, PHP, APC and Memcached scripts and Cacti graphs
After some more work on the weekend and playing around with cacti i have fixed up some of the previous scripts and joined input methods with data sources and graphs all together.
I have also added a host template so now adding servers will be much easier you just enter host name and click ccreate graphs ... job done.
Package includes graphs to monitor Linux system metrics, disk IO, network IO, apache status, APC op code cache, memcached.
Im still adding stuff to it so make sure to come back for an update : -)
You can see some of the graphs in earlier versions here:
Linux graphs
Monitoring Linux servers with cacti ghraph
There is a lot going on on the average server and it helps a lot to be able to see how each component performs and under what stress it is. I prepared a few graphs that show most important parameters in a more readable form than default cacti graphs.
First there is memory status showing total phisical memory with swap on top:
Then there is CPU time division to kernel user io itd:
Monitoring Memcached status with cacti graphs on rrdtool
Memcache is an awesome caching server but managing it is a bit more complex than most people think. It used to have quite many bugs in its history plus the algorithms of memory allocation and replacement of items might cause confusion. To manage it efficiently you have to monitor hits and misses ratio:
Then another important information you want to track is evictions rate as climbing evictions might indicate that you cache too much or have not enough memory.
How to create self signed ssl certificate for apache2 web server
We often need to test some sites on local machine or development server. Its good to have ssl environment set so we could test all links etc. For that to work we need a ssl certificate for our apache server. Then we will be able to start apache mod ssl extension.
Passwordless login with ssh - authorization keys generation
It is possible to login to remote hosts via ssh without providing a password. It is especially important while creating automated scripts. Having password less login we are able to execute commands on remote machine without the need to keep passwords unsecured in the script code.
Dual boot linux and Windows Vista from Windows bootloader
It is possible and really easy to install linux on a single partition and tell windows vista to load it on demand. We dont have to mess up our windows installation just make it recognize linux as bootable partition.
Mac OS X default logging helper sucks
Mac has some nice features but some are really lame. Trying to make syslog look cool made my system inresponsive for like 10 minutes. All because of GrowlHelperApp.
How to check file consistency across operating systems
Its very stressful if we have to copy 2gb database dump or archive of important data over network. Even that TCP is supposed to let us send data without any damage it can happen from time to time while copying huge amounts of data through slow networks that some package gets damaged and crc checksum does not detect it (32 bit checksum still gives some chance of collision so the more packages the bigger the chance)
nc (net cat) used as network debugging, sniffing and hacking tool
Netcan can be used in very many ways. Its a low level network testing and debugging tool. As such needs a bit of knowledge how protocols and servers work but once you pass that netcat gives endless possibilities. I show here how to sniff forgotten passwords from out ftp client and how to connect manually to memcache issuing commands. All can be dome with one line commands or put nicely in a shell scrip.
digg