Stress testing our PHP framework

So, today we did a little stress testing on our old and new framework, the aim was to see what kind of improvement in performance (if any) we'd gained in re-writing the framework and adding some new functionality.

How did we stress test?

Initially we were using Apaches built in benchmarking tool, but although it's great for testing, it doesn't really simulate real users clicking a page, reading it, then loading another page, it simply throws all the requests at your site at the same time. It does however give you a good indication of your sites performance, and the requests per second were very similar to our results below.

We settled on using a tool called Siege, luckly it happened to already be installed on our server, but a simple 'yum in stall siege' should do it if it's not already installed. Just to note at this point, it's always best to run the benchmarking tool from another server, otherwise you'll not get accurate results.

Using Siege was pretty simple...

siege -u http://www.domainname.com -d1 -r10 -c10

What the above does, is make 10 concurrent connections to your site, with each connection making ten requests. the -d means a delay of 1 second between each request (to simulate a user reading the page). If you don't specify -d, it defaults to 3 seconds.

Why did we choose 10 requests and connections? After a few attempts at trying difference values, we found that using 10 requests and connections give us a reasonable response time < 3 seconds, higher values yielded response times which would have more than likely affected the user experience. So technically the site could cope with more visitors than our results show, but page loads would have been between 3-5 second (more in some cases), we weren't really happy with that kind of response time.

We carried out three tests, all runnning exactly the same site which makes a number of database connections and rendered a page using a template and view (our own website infact). It's probably worth noting at this point that the site is hosted on a VPS, so a dedicated server would likely show much better results.

Test 1. V1 FW - Our old framework

Test 2. V2 FW - Our new framework

Test 3. V2 FW w/Cache - Our new framework with built in caching enabled

Here are the results...

Looking first at the transaction rate, this is the number of transactions the site was able to get through per second. There is little difference between the old and new framework really, although there is some improvement which is more apparent when you look at how many requests per minute and hour that equates too. 

The biggest improvement comes when we enable caching. With caching enabled the site can serve around 290% more requests per second. Looking at the longest and shortest transaction times you'll also see that the response times from the server are much better.

What have we learnt from this? When it comes to low traffic sites, both the old and new framework will more than cope, but there's probably a more noticeable difference on higher traffic sites. Building caching into the new framework was certainly worth doing, it gives us the opportunity to really speed up a websites performance, while also increasing the amount of traffic it can deal with, meaning purchasing that dedicated server can be put on hold that little bit longer.