Introduction

After being tasked with finding a possible replacement for our dieing imageserver (running out of diskspace, no real replication, high load, and so on…), I have spent quite some time studying possible solutions. We have come up with a solution which is fast, scalable, domain independent, has high availability, and is redundant. Which are exactly the kind of qualities you are looking for if you are designing an architecture for a company that has multiple high traffic projects and has these qualities as requirements. The technologies used for the creation of these static file clusters are all opensource and should be available for most relatively recent *nix distributions. The technologies used in this setup are in no particular order: nginx, lsyncd, heartbeat, RR-DNS.

Although the focus of this post is heavily geared towards the distribution of images, it should be clear that the information disclosed in this document can be used to distribute and load balance any static files across any number of clusters. Please also note that I have attempted to explain everything in great detail and that this unfortunately has the consequence that the complexity of the descriptions and terminology used in this post is geared towards technical people with an understanding about *nix system administration.

Continue reading →