Framework benchmarks

There's an interesting blog post floating around today with benchmarks of the various PHP frameworks. In this article an identical (but basic) application is created using each, and then run through its paces using httperf on the same hardware under a number of different configurations. The source code is provided for each, and the article also includes comparative benchmarks from non PHP frameworks such as Rails.

Comparing frameworks is a tricky beast. Any framework will add some overhead to an application, the amount of overheard generally depends on:

• how fully featured the framework is (the more lightweight the framework is, the less that usually has to be required by default)
• how many of these features have to be loaded in in the test
• to what extent the test app takes advantage of these features

For this reason, lightweight frameworks will always come out on top in 'hello world' type tests, since such tests use basically none of the features the frameworks provide. But as the test apps get more complicated, the more fully featured frameworks should start to come into their own. The conclusion of these discussions is always that there is no 'best' or 'worst' framework, you use the right tool for the job.

Having said that, I still think there is value in these benchmarks. I've said before that I believe performance should be one of the main goals for any framework, as the thousands of applications that will be build upon the code can only be as fast as the framework itself is.

Although ZF comes out fairly well in these tests, I was a little surprised that ZF + APC didn't out perform Rails. I'm looking forwarding to seeing what affect (if any) Zend's upcoming performance review has. I also have some ideas of my own on how to tweak the framework to squeeze up the numbers - more on this when I have a big enough app to test some things on.