Mr Kirkland

Congratulations, your server is melting.

This is a good problem to have (server heat is proportional to # of users). And it’s not so difficult to deal with if make a few preparations in advance.

After launching a number of web services and viral social media apps, some of which grew to hundreds of concurrent users and zillions of hits within days, I’ve had to learn “on the job” about scaling websites. Here I present to you my thoughts and some simple suggestions on how you can prepare ahead of time.

Part 1: Think Modular

Really the most important principal to take on board is modular design. If you look at any high volume platform you’ll quickly see evidence of how the components of the service are split up into separate independent units. Wikipedia is classic example of this, cheek out the architecture behind wikipedia.

If you can separate all the different components of your website/web app, then you can easily apply more resources where they are needed. Just upgrading to a bigger server won’t get you far, long term you need to be able to identify where the bottle necks are and apply resources efficiently. In fact even identifying the bottlenecks can be surprisingly hard, modular design makes this much easier.

Spilt services into independent modules, it’s much more efficient to scale them individually

So some examples of the typical components your website/web app will be using:

• media, images, static files (javascript, css etc)
• db server
• mail
• batch processing (webstats, image resizing etc.)
• front end (normal user area)
• back end (control panel)

Separating these components needn’t be rocket science, here’s some simple examples of how you could apply modularity to the components above:

• media -> move all media and static files on a dedicated domain/subdomain (or better still in a CDN) e.g. <img src=”http://media-server.mydomain.com” />
• db server -> always use a central config file for db settings, so you can easily change the hostname when the time comes to use a dedicated db server:  $db_hostname = ‘mydomain.com’ -> $db_hostname = ‘dbserver.mydomain.com’;
• mail -> again make sure mail settings are in a central config and ideally use a framework/library that allows you to change to an external SMTP server/service
• batch processing -> create an independent system for any batch processes, for example use a standalone phpthumb installation for image resizing.
• separate front end / back end -> make use of different a domain for control panels e.g. www.mydomain.com and controlpanel.mydomain.com, so later on it’ll be easy to move these onto different servers

In addition to thinking modular, always be monitoring and profiling your systems then you know exactly where the bottlenecks are and can deal with them more effectively.

Part 2: Other Key Principles

1. Queuing/Batching

Queuing prevents a server being overloaded all of a sudden

Typically there may be parts of your application that involve heavy processing. If these don’t need to be real time, or can tolerate slight delays then separate the processes and make use of queuing/batching so your web application isn’t held up.

Google Analytics is a good example of this, generally there’s no need to have real time webstats, so the collected stats are batch processed at regular intervals. Another example could be image processing, I’ve written about this in more detail here.

2. Partitioning

Taking modularization a step further and we have partitioning i.e. splitting data up in to smaller manageable chunks which can be stored/operated on separately. If you have a multi user system you could split users across servers, so for example your could have 1000 users per server, or split users by user id odd/even, divisible by X etc. Again you could do this ahead of time with subdomains:

odd user id: controlpanel-a.mydomain.com
even user id: controlpanel-b.mydomain.com


Another example of partitioning is database partitioning, here’s a quick introduction, but this would come way later, having a dedicated db server or db cluster will scale you a long way.

3. Code First, Profile and Optimize Later

Slightly contrary to the idea of scalability, but definitely on topic: don’t start your code design with optimization, this is a waste of time. Not even the mighty He-man could predict where all your bottle necks will be. Always focus on writing easily maintainable code, and then in the later stages of development you can profile your application to find out where you need to optimize your code.

4. Cache Cache Cache

Caching is probably the simplest and cheapest way to dramatically improving performance. There are a variety of stages at which you can use a cache and whilst some require a little care at least a couple are almost trivially easy to implement. It’s good to be aware of all the options even if you don’t need to implement them yet.

Super Easy

• Opcode Cache/Acceleration (like APC, xcache
) -> once installed require minimal maintenance and are completely transparent to your application. They simply keep copies of your complied scripts in memory.
• DB Cache -> again usually very simple to install and will be transparent to your application and keep the result of queries in memory. for mysql users, see here.

Easyish

• Cache server/HTTP accelerator (squid nginx varnish etc.) -> This is very common with high traffic sites, Caches can easily multiply the capacity of your website. Again this kind of cache is outside of your application so is relatively easy to work with. However you may need to take some precautions to ensure certain items aren’t cached, and are expired in good time and you’ll need a higher level of sysadmin skills to set up and maintain a cache server.
• Built in framework cache -> many code frame works come with a cache system out of the box (e.g. codeigniter) this is a lot easier than rolling your own, but still you’ll need to be aware of how it works to avoid caching pages that should stay fresh.

Harder

• Application object cache -> this kind of solution involves using a library/framework that allows you to cache arbitrary objects/variables across sessions. For example you might have a widget that appears on multiple pages, you could cache the object that generates this widget (or even the HTML output itself).
• Setting HTTP headers -> you can of course manually set HTTP headers that tell web browsers how long to cache pages for. This gives the best performance boost as the data is stored locally on the end users machine. However this is no trivial task and requires a full understanding of how browsers respond to HTTP cache headers, here’s more info on the subject

Part 3: Quick and Practical tips

Within 10 mins his app was performing about 10 – 50 times faster. All I did was add a few basic indexes.

1. Database Indexes + Query Optimisation

This isn’t strictly about scaling, but it’s such a good performance tip: Learn how to do good DB Indexes, they are really simple and without them you’re hemorrhaging your DB’s potential performance away. here’s a quick tutorial.

I was having a geek cafe session with my good friend Craig Mod one afternoon and he asked me to look at his application as it was a bit slow. Within 10 mins his app was performing about 10 – 50 times faster. All I did was add a few basic indexes.

As mentioned above, don’t forget to turn on the DB query cache.

2. Use a separate database reader and writer

There are a number of reason why at some point you’ll need a DB master and slave(s) (load balancing, redundancy, zero downtime backups), you can prepare for this now by having two DB objects e.g $db_reader and $db_writer

$db_reader = new db_object(SLAVE_DSN);
$db_writer = new db_object(MASTER_DSN);
//reads go to the slave (or master)
$db_reader->query("select * from user where id = 1");
//writes only go to the master
$db_writer->query("update user set status = 'kool kat' where id = 1");


With a simple change to your DB config this allows you to instantly use a master/slave(s) setup to spread the load on your database.

3. Account For DB Reader Delays

Allow for the DB reader to be slightly behind the master – replicated slaves are likely to periodically lag behind the master even if just for a second or two. If you are using the tip above and sending reads to a slave then this could cause problems in your application. For example say a user is logged into a control panel making changes, if the DB reader is behind the DB writer, then trying to fetch the new changes from the database will potentially show the old values if the DB reader is behind.

4. Separate Server For Media/Static Files

I’ve mentioned this already above, but it’s worth mentioning again. Set up your site to host all images, js and css on a domain different from your main website. I’d recommend you have some sort of central configuration so the static server url is automatically included in all image urls e.g.

//hand coded (not recommended)
<img src="http://images.mywebsite.com/images/1234_an_image.jpg" / >

//or full managed image urls using a special class to generate urls
<img src="<?php echo $this->images->get_image_url($image_id); ?>" />


5. Mail Config

If your application sends emails, make sure the mail config can be changed easily and can use SMTP. 
Part of modularizing + queuing would involve having a separate server for sending, it’s most likely you’ll connect to this from your app via SMTP. Particularly now as there are a number of 3rd party SMTP services that can reduce the increasing headache of reliably sending mail.

If you are using a standard mail library like phpmailer, or as part of a framework, then this is probably set up already – see the docs. if you have your own DIY mail functions, it’s probably best to swap to some established library (e.g. phpmailer) anyway, no need to re-invent the wheel.

6. Avoid local file system dependencies – separate your data storage.

Separating storage of data from your application is an essential part of being modular. Probably most of your data is in a database already so without too much sweating and grunting this is most of the battle won. However you may be storing uploaded files, images etc. which you definitely don’t want to store in a database. But you shouldn’t just write these to the local file system either, if you use a cluster of application servers then you’ll need some sort of shorted storage system. A good example of this would be Amazon S3, there are plenty of client libraries and example code that means it should only take a matter of hours to integrate. An NFS server could be an option, but NFS can be nasty!.

7. Efficient Backups

Large backups can really slow down or even stop a server (particularly large DB backups), but chances are you could significantly reduce this load by

• Not backing up temp/cache files
• Using git/svn etc. and backing up the repository not the web application docroot
• Database dumps/snapshots using a slave, so write locks will not affect your DB server
• For your data storage use a filesystem/system that support snapshots (e.g. ZFS)

8a. Get into the Cloud

Using virtual servers/cloud computing will give you a certain amount of scalability for free. Most virtual server providers make it very easy to up the power on your machines, whilst this may not be the most elegant solution, it saves a massive amount of work compared to the task of migrating dedicated servers.

However that’s not the only scalability benefit. You get many of the suggestions I’ve made here for free with Cloud/Virtual computing, e.g.

• Easy to clone machines (for setting up multiple identical servers)
• On demand computing/per hour billing is great for batch processing
• Filesystem snapshots
• Robust Storage Solutions (e.g. Amazon S3)

8b. Services Above The Clouds

And last but not least I should mention there are now a number of higher level services, RightScale.com, Scalr.net, Google App Engine to name a few that offer automatic scalability. If you are able to work within their constraints (for example App Engine only supports Python + Java) then they can be a very attractive offering. Google App Engine:

Automatic scaling is built in with App Engine, all you have to do is write your application code and we’ll do the rest.

Closing Comments

As technology advances, a lot of scalability issues become easier to solve or disappear – we have cloud computing, easy to use CDN’s and services like Google App Engine. However until we have the Holodeck, I’m pretty confident most of the principles I’ve raised today will still be important in your design considerations.

…oh and hopefully your server has stopped melting now.

Further Reading

Here’s a few books that have helped me:

Check out a real world example, wikimedia:
http://meta.wikimedia.org/wiki/Wikimedia_servers

highscalability.com has some good resources:
http://highscalability.com/blog/category/blog