PHP session in Mysql VS Memcached
In my previous companies we used to serve sessions from mysql via a custom made session handlers and to be honest we never really had any problems with it. I cant actually remember the traffics and concurrency rates but it was not that low. Currently the solution im working with uses memcached for sessions storage, the default build in session handler. I am a bit worried about the way it works with memcached.
Problems with memcached sessions in PHP
First of all memcached is a cache storge engine and it was not designed with sessions in mind. The way php extension uses memcache causes a few more problems in the long run. Here are my concerns:
Ultima Online Resources
Shop prices
This is the initial prices list proposal that i came up with.
The main idea is that customers could always see current prices and they would have them guaranteed. So that if we add new item into the shop it will have the price from the file.
- Shows how attractive our pricing is
- Helps users to have some reference while shopping elsewhere
- Helps us to establish reasonable and predictible prices
Using early cache rebuild to optimize web cache performance
Having a busy website requires a few techniques to optimize the performance of caches. It is not that difficult and should help you reduce the load times (pessimistic load times) a whole lot.
Imagine you have home page cache (as fragments or entire page). Its all good and beautiful as long as it sits in the cache. All threads connect to cache, get the content and send to the user. What happens when the cache expires? Well all threads have to rebuild it. Unfortunately on very busy content elements it can cause dozens of processes trying to rebuild the same page!
Cache expiration consequences on busy pages
New decade, Google tools are the coolest
I have just upgraded to the new 3.6 Picasa from google and i have to say its just amazing! It is just hands down the coolest photo editor ever!
I know it does lack a few features but seriously! Not only it has great filters, retouch cropping and automated editorial features! it now comes with face recognition and it is just amazing ! i have scanned all my photos (takes a while) and its just amazing how well does it recognize faces. Cherry on the top is the fact how easy is it to use it.
Just in case you are not using it, try Google Earth with its photos and places, street view, markers, xml export and search .... AMAZING !
Another Google tool i absolutely love is Google Analytics ... i still can not believe its free! the best traffic analysis tool i have seen.
Review - Psychology and life
Now, that was an amazing book.
I got it as recommendation from my brother in law (he has degree in psychology). At first i thought it would be dry and hard to read but it was not like that at all. I was amazed how interesting was that book. I was also amazed that it talks about very serious stuff in a normal way that can be digested by normal people.
Mixed feelings about PHP vs THE BIG GUYS
Lately i was readin some more about current Java based frameworks and development platforms. We even had some presentations from TOP players (Ill call them the Bestests) in Java world. To be honest i was very unimpressed. I started to look more critical on the Java community.
For years i had some Java fascination that was planted in my brain by Bestests when i was in high school. I always though 'well damn! if they have the best people and best minds, if they have shitload of money ... that has to lead to something great right?!'. Well im not sure any more. Maybe its that online content managment is not Java's good side? maybe we were unlucky with the presentations and frameworks we saw so far?
PALM v3 - bundle of Php Apache Linux and Memcached Cacti scripts (APC included)
Finally i had some time to gather all the files together and make third release of my cacti monitoring scripts and graph templates. Monitoring of application and server health is a important task while trying to maintain high availability. Without monitoring you dont know what goes wrong nor when. Some time ago i realized that some of the graphs i used to use were at least not perfect. I searched the web, read a bit of documentation and decided to put together a set of simple scripts gathering key server metrics of average web server. I still dont have postgres nor mysql stats included but .... who knows .... maybe in version 4! :- )
New version has a few new metrics as well as some bugfixes. I also added better support for non pure Debian systems
Web Security - Part 1 - Google Hacking
Doing security reviews from time to time of web applications you can find real beauties. You see GET variables in eval, system, include and all sorts of things you would not even imagine that could run live for years without anyone noticing it :- )
I dont really have great english nor writing skills but i thought i will try to write a few parts of the security article just to give something back to the community. Its not really rocket science nor my invention or discovery. Its just day and weeks and years spent working with web apps and reading here and there. What i hope is to write something you will enjoy and maybe put some new spin on some issues.
PHP display_errors => You will be Google-Hacked for sure
Java MouseClicker - Automated mouse movement and clicking tool
I needed a simple tool to automate mouse movements and delays on my PC. I like playing with Java so i wrote a simple one in a few hours.
Its a really simplistic app and it might have some bugs so please be understanding :-) But i could not find any free tool that would do exactly what i wanted to i thought i could share it to the public. Maybe someone will find it useful.
If you like it and want to modify it for your needs i think i might send you source code.
Setting up mouse control sequence
So to see mouse position click on track button, you can stop tracking if you are done. Then you move mouse where you want on the screen and write down location. Then write up a script like this:
MOVE:1000:750 LEFT_CLK WAIT:1 LEFT_CLK MOVE:680:90 LEFT_CLK
Compile Memcached on Mac OS X
First of all lets make it clear, memcahced on mac? why bother?! Well i have to work on mac at work so i dont really have a choice. I assume if you are interested in that hint you are in same situation. Personally i think its just pain that i dont work on debian but any way.
If you need memcached running locally on mac download sources of libevent from lib event site. Then follow typical procedure of compilation and installation:
./configure --prefix=/usr/local make make install
Loading URL with CURL from specified IP address
Sometimes we have to automate some pages testing or loading content or whatever. CURL is an excelent library for any HTTP interfacing.
If you ever have to connect to a server and ask for particular server name you can do it with curl setopt parameters. If you want to get file /index.php from www.mytopsite.com from particular IP (without adding any /etc/hosts mappings) you can do it like this:
$headers = array("Host: www.myfaketopsite.com");
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_URL, "http://10.1.2.3/index.php");
This is really helpful if you have a farm of 40 web servers and you have to ask each and every one, load some file and process it. Im sure you can think of other usages.
Wget page from server using IP
PHP Memcache extension API Pitfall
It happened again, i spent like 15 minutes trying to figure out what am i doing wrong before i realized that i got caught in a API snare ;- )
Problem i had was caused that PHP extension for Memcached communication has a strange way of interpretation time to live parameter. If you provide 3rd parameter to set function it will set value in cache and mark it to expire after X amount of seconds. Well that all cool and nice but if you try to pass amount of seconds greater than 2.5 mln seconds (about 30 days) it will be actually interpreted as unix timestamp! Oh noooooo! : -)
Just a hint how to validate XML Schema xsd under linux
Its really basic stuff but worth to remember that you can easily validate XML Schema under linux command line with xmllint. It can be especially useful if you have to automate some xml files validation in crons etc.
xmllint --schema /path/to/schema.xsd /path/to/sample.xml
This will show you list of validation errors or a success message.
Top Places In The World to Visit
There are very many places in the world that are worth visiting. To put it in order and make sure you visit most important places or most attractive to you in just one lifetime is a serious task :- ) To have better overview and plan easier i decided to create a list of bookmarks.
The perfect tool for organizing your travel bookmarks is Google Earth. You can see some of photos geo located with panoramio and wikipedia bookmarks. Also world preview is much faster and more flexible than google maps. Thats why i decided to put all my bookmarks on google earth map. Every time i get some catalog or a travel guide i try to add most important places onto the map. With time my list grew and i decided it might be actually useful to someone.
Mysql linux shell commands to get tables list and basic statistics
Mysql is so common that its good idea to have some command line scripts or tricks. Sometimes you need to execute some SQL queries on set of tables or all of them or get stats or fix tables etc.
Its good to be able quickly get list of tables and basic stats.
You can make bash script of it or just type in the command line:
# Put your database name here export SET dbname='drupal'; # List all tables in the database echo "show tables;" | mysql -u root $dbname |grep -v '^Tables_in_' > tables.txt # Show tables status (tables' engine, rows count, table size, index size, collation, comment) echo "show table status;" | mysql -u root $dbname > table_status.txt # Get each tables indexes details
Too many magic methods in PHP code can be a real pain!
Magic methods in PHP can be a bit confusing especially if you do everything via __get and __call. They can be very usefull but adding them
by default everywhere is definetly a bad idea. I dont like __get/__set accessors that let you access enything. That defeats purpose of
encapsulation and promotes cross dependencies.
Recently i came across a silly problem .... i think its interesting enough to have a look at it.
So what would you expect from that code?
var_dump(empty($param->reference_class)); var_dump($param->reference_class);
Any reasonable person would say that there are two options:
1. first one bool:true, second one some value
2. first one bool:false, second one 0,’’,null or empty array
Convertig file encoding to UTF-8 from other Unicode and removing Byte Order Marks
Zend Studio for eclipse has a hidden feature that is very useful. It can show you what encoding is the file you are editing and
you can convert the encoding with a little copy and paste trick : -)
If you work on linux and you get file in utf-16 there is almost no way you will be able to edit in in command line nor do anything useful with it.
Each character is saved as 2 bytes inflating the size and making it encrypted for command line tools. It also can happen that you have
these annoying byte order marks etc.
Any way in eclipse you can convert it using this sequence:
1. open the file
2. ctrl + a
3. ctrl + c
4. menu -> edit -> set encoding -> utf-8
5. ctrl + a
6. delete
7. ctrl + v
8. save
PHP Soap Error - Procedure 'xxx' not present
In PHP while using SOAP extension and calling external service, you might get into problem that PHP keeps on saying that method does not exist.
You are calling a method on a service that has the method and you are still getting "Procedure 'xxx' not present" error.
Error would look more or less like this:
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Body>
<SOAP-ENV:Fault>
<faultcode>SOAP-ENV:Server</faultcode>
<faultstring>Procedure 'gettwotandomitems' not present</faultstring>
</SOAP-ENV:Fault>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
Review - JQuery in action
Its a really great Jquery book. I liked a lot the fact the authors simply let you see the power of jquery not getting too deeply into javascript. There is no cross/browser details or low level tricks.On the other side authors dont skip over important details.
Book has a very good structure and consistently presents Jquery API. I thik there is not much more you could expect from the book in that area.
I also loved the fact that they put javascript appendix to describe more complex ideas. It really gives you good review and puts all bits in place.
How to wget to mirror a website without html file extensions
Wget is a great and powerful tool but it does have some hidden features that are not really common in linux world.
For example it has set of default settings that can be altered via some misterious .wgetrc (good luck on mac!).
How to mirror a website onto local machine?
wget -m -k -K URL
This will crawl all pages in the site, save them on disk and rewrite urls to relative ones so you can browse them locally.
Its cool but it also adds .html extensions so index.php becomes index.php.html. But in my case i wanted to crawl the site to flatten it
and make static html pages. Site was like 4 years old, and i wanted to make it easier to deploy and safer than some freaky old crappy php app sending
emails and god knows what else.
Jmeter used to playback Apache access logs to generate live-like server load
Jmeter is a very useful tool for not only performance tests but also for automation of all sorts of web tests. Its not as powerful as selenium when it comes to ajax etc but it still allows to login, keep session and execute any sequence of http calls.
In addition to ab (apache benchmark) and curl it comes in handy when you want to stress test your applications to see the capacity and possible bottlenecks.
Its difficult to perform reliable tests when your development environment does not mirror live and when you have no traffic at all. 10000 home page calls will perform differently depending on what is the server doing.
To generate realistic load and be able to profile applications and their bottlenecks you can playback your live server logs on test machines.
Review - Ruby on Rails 2.1
That book is actually only in Polish so far, but there is no harm to mention a few words.
Author of the book is a friendly 'ruby-psycho' i work with :-) He is getting me interested in some things from time to time and he
also let me read his book on rails framework.
To be honest i have no experience with rails at all and i just started getting familiar with ruby itself. The book assumes you have quite a bit
of development understanding or at least you know ruby. I work with PHP for years and i have used other languages so analogies
Review - Linux System Programming
I guess its not easy to write an easy to read book about linux system programming but this one is not bad at all.
I have to admit that it is a bit 'dry', tons and tons of system calls explained with parameters and return values
and error codes going on and on and on :- ) But thats the point. You want to know how to write system apps on linux?
well you have to go through it and youd better enjoy!
But book also contains some really nice chapters describing file system handling, cache, memory and process schedulers.
Review - High Performance Mysql
High Performance Mysql is hands down the best mysql book i have seen.
I would recommend it to everyone who wants to learn more about mysql, not only for gurus but for everyone. I would recommend it to sys admins and programmers. Especially if you are concerned by performance you should read this book.
Authors describes in details how mysql works, how to tune it, how to avoid benchmarking pitfalls and how to plan for scale.
Replication, tuning, monitoring, backups, sharding and shinx its all there! :- )
How to compile memcached 1.4 on MAC
I am using Mac just since January and im still not so familiar with it. If you want to install latest memcached on Mac OSX you will need libevent.
Download libevent from this site:
http://www.monkey.org/~provos/libevent/
Then unpack, configure and compile it. Also install it in /usr/local
./configure --prefix=/usr/local make make install
Then obtain latest memcached from:
http://www.danga.com/memcached/download.bml
Then unpack, configure and compile. You need to point to the same folder so configure could find the library.
unpack ./configure --with-libevent=/usr/local/ make ./memcached -u root -vv
And thats all.
PALM - bundle of Linux system, Network, Apache2, PHP, APC and Memcached scripts and Cacti graphs
After some more work on the weekend and playing around with cacti i have fixed up some of the previous scripts and joined input methods with data sources and graphs all together.
I have also added a host template so now adding servers will be much easier you just enter host name and click ccreate graphs ... job done.
Package includes graphs to monitor Linux system metrics, disk IO, network IO, apache status, APC op code cache, memcached.
Im still adding stuff to it so make sure to come back for an update : -)
You can see some of the graphs in earlier versions here:
Linux graphs
Annoying unicode Byte Order Marks in UTF-8 files
There is this annoying thing that happened to me a few times before that some files get additional invisible characters at the beginning of the file. They might look something like this:
It happened to me a few times when i was working with some people from Germany and they used different tools and different encodings so i thought it was some editor related issue. But it was strange as it would only appear in the diff screen of eclipse svn plugin not in regular edit window.
Monitoring Linux servers with cacti ghraph
There is a lot going on on the average server and it helps a lot to be able to see how each component performs and under what stress it is. I prepared a few graphs that show most important parameters in a more readable form than default cacti graphs.
First there is memory status showing total phisical memory with swap on top:
Then there is CPU time division to kernel user io itd:
Monitoring APC op code cache for PHP with cacti graphs
APC is one of the best PHP op code caches. To be sure its performing the best it can you should monitor its status and make sure it has enough memory as well as correct ini settings.
You want to monitor memory allocated for op code cache as well as user stored items. Its important to get it right as wrong settings may reset your complete cache once it gets full:
You should also monitor the hit and miss ratio for op code file cache:
Monitoring Memcached status with cacti graphs on rrdtool
Memcache is an awesome caching server but managing it is a bit more complex than most people think. It used to have quite many bugs in its history plus the algorithms of memory allocation and replacement of items might cause confusion. To manage it efficiently you have to monitor hits and misses ratio:
Then another important information you want to track is evictions rate as climbing evictions might indicate that you cache too much or have not enough memory.
digg