Learn How To Use Domain Sharding For Performance Optimization

By Branislava Milosavljevic on 2015-08-20 | 2 comments

performance, domain sharding, applications, profiler tools, user experience, optimization, slow loading time, Eton Digital

What do you do when your website or application shows signs of poor performance?

Act fast, and accordingly.

We’ve recently noticed that one of our applications, launched a year ago, saw some performance problems.

The problem first appeared with more than 5000 users accessing the application intensively. Having a large number of worldwide users accessing the application at once resulted in a slower loading time for some web pages and an increase in the number of requests (images, JavaScript, CSS files) sent to a web server.

We analysed the web server logs only to confirm what we were suspicious about in the first place - the part of the problem was high CPU utilization and a weaker hardware. We advised our clients to upgrade to a better hosting package (a 4-core processor, SDD disk and 16GB RAM memory) which improved hardware and application performance, however,  it still showed signs of slow performance, which we weren't pleased with.

To address this problem thoroughly, we concluded that the application also needs an easy-to-implement and cost-efficient Content Delivery Network (CDN) and we decided to use CloudFlare free plan.

Why CloudFlare?

The network operates out of 34 data centres around the world, automatically caches the static files, and distribute these files from the locations that are the closest to the visitor while delivering the dynamic content directly from our production server.

To accomplish this, the network uses a technology called Anycast to route your visitors to the nearest data centre.

CloudFlare integration and domain redirection was simple and fast.  And while we were working with CloudFlare, we came up with the idea for a new optimization strategy - domain sharding.


Web browsers can limit the number of requests sent to a particular web host (see image 1).



Such limitations lead to an increasing number of resources that are blocked while waiting to be downloaded. See the common pattern in the waterfall diagram (image 2):


Domain sharding offers one of the solutions for this problem by splitting content across multiple subdomains to simultaneously download static resources.

With domain sharding, one can decrease the time spent waiting for resources to download and improve page load time for more than 30% (image 3).


However, domain sharding can be implemented incorrectly, and if so, you can expect to see a negative performance of your web application.

To avoid this, you should keep in mind several situations:

  • Domain sharding increases the number of DNS lookups - too many implemented shards can hurt application performance and the application will perform better without sharding
  • Static resources should be consistently served from the same sub-domains - this way you will not lose the benefits of caching
  • The resources should be distributed evenly between sharded hosts - if not, such optimization will have no effect because the number of blocking requests will increase
  • SSL certificates for the sub-domains are needed for a HTTPS site (the added cost of buying)
  • Due to parallel requests sent to the server, you will experience a higher server load
  • The same origin policy - when you move your static content onto a separate domain or subdomain, a code can break if it depends on the data from the same origin

Domain sharding integration is very simple as my web application development team uses Symfony2 framework that offers excellent options for sharding integration.

If you are using Symfony2 and Assetic for serving static resources, you can define assets_base_urls parameter (http and ssl) to manipulate the files (in theory).

However, we had to write a specific Twig function for our application as the images are served separately from Assetic. The function has a different SRC parameter for image input and new output image parameter served from sharding. Twig function shows how to evenly distribute the resources and benefit from caching.

Whether you use Symfony or not, here is a good example of how to implement sharding algorithm:

function shard($url, $shards) {
    $md5 = md5($url);

    $md5_32 = 0;
    for ($i = 0; $i <= 3; $i++) {
        $md5_32 ^= hexdec(substr($md5, $i * 8, 8));
    $shard = $md5_32 % $shards;
    return "http://static$$url";

But know this  - while you are implementing domain sharding, shards should be cookieless.

Moreover, to take the advantage of domain sharding, you should block all server requests but requests for static resources.

If you are using CloudFlare CDN, unfortunately it’s almost impossible to implement cookieless domains. On the other hand, if you are using Apache server, with mod_expire i mod_headers modules you can have better control over resources and, for example, set caching for specific resources, or something similar.

If you want to properly optimize your web page, the first thing to do would be to reduce the overall number of resource requests.

Symfony2 has a great component I already mentioned, Assetic, in addition to uglifyjs i uglifycss filters that can be easily integrated into it. Below you can see the example of combining a larger number of JS databases into one single file using uglifyjs filter.


If you are using Symfony2, you can also use ESI (Edge Side Includes) to additionally control HTTP caching. With ESI, you can cache page fragments independently - this way you will be caching a web page for 60 minutes and embedded sidebar only five minutes. Activate ESI in the main config.yml and call fragments in Twig with render_esi.

If you are using Doctrine, you can significantly improve performance within a database with Doctrine cache. However, to avoid possible bugs, you need to pay attention to properly invalidate Doctrine cache.

We use MySQL indexes to speed up database query - this is a good way to take advantage of MySQL. Symfony toolbar, on the other hand, helps greatly with locating problematic inquiries. The hardest part of implementation is experimenting and adding indexes in Doctrine mapping.

If some of your pages are still loading slowly, and you are having a hard time determining what causes poor performance, I advise using profiler tools.

One such quality tool for PHP is Blackfire Profiler  - easy-to-use, intuitive and interactive.

If you pay attention to the red fields in the graph, you will see the problem.



Good luck with optimization!

If you have any questions or care to know more, write in the comments below.



Polish Letters Test
2018-10-19 at 21:25

2016-10-04 at 03:37

Leave your comment