Derek Greentree, one of our Senior Software Engineers, has agreed to share his thoughts on what he does here at Lijit. You can read the first part of this series here.

When you first start optimizing a website, there are many questions to answer. What are you optimizing for? Raw speed? Maximum concurrency? How will you determine the bottlenecks in your way? How will you test the optimizations you make to be sure they were real and not placebos? How will your optimizations scale? And finally, how much time are you willing to put in for what amount of return? A famous quote among software development people, coined by Donald Knuth, states that premature optimization is the root of all evil. Web applications are no exception.

First, you’ll need an environment where you can test your changes. You should have a test environment anyway, but this environment will exist mostly for benchmarking, which means you’ll be putting load on it, so you may want to have a separate environment just for load testing. Since you’re going to be deploying your optimizations eventually to your production servers, this environment needs to mimic your production architecture as closely as possible. Don’t load test on a machine that serves its content from a network share, when your production servers serve that content from local disk.

When you have an environment ready, your next step is to benchmark what you currently have. Many tools exist for this, but at Lijit, for website benchmarking, we use Siege. After a siege run, you’ll have data such as how many requests completed, the distribution of HTTP status codes, and average number of transactions per second. Be statistically smart about this - run several tests and average them together to get a baseline reading. Try to max out your environment; knowing when your overhead runs out in your current architecture is powerful knowledge. And, if you can max out your environment during the test, you can be sure the limits you’re hitting aren’t limits of the load testers, but the thing being tested. Try to hit your site in a browser during the load test so you can see what the user experience will be like in a high load situation. Anticipate problems before they occur.

Once you have these baseline numbers, you can test optimizations by running the same sequence of load tests and comparing the results. You can see if the various status codes coming back from the test changed. For example, if you deploy an optimization and start getting a lot of HTTP 500 instead of HTTP 200, you broke something.

Getting a good benchmarking environment setup is the essential first step to trying to optimize your site. In the next article, we’ll talk about some basic things to look at when approaching optimization, including whether you need to do it at all, and some specific examples of problems and solutions we’ve encountered at Lijit.

This entry was posted on Thursday, April 17th, 2008 at 10:30 am and is filed under company, technology. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.