You are hereExtremely simple file loadbalancing with statistics
Extremely simple file loadbalancing with statistics
With FOSDEM we have around 70GB of recorded lectures. While this is a good thing for developers that weren't able to come, it can be annoying for the organizers as they have to make these files available.
There are currently two ways to access the video's. Using the FOSDEMTALKS YouTube channel or via video.fosdem.org (http download).
Serving all the files only ourselves would add useless load on our server and internet uplink. That's where Belnet, HEAnet, Unixheads and dotsrc.org; the mirrors play a role. The annoying thing with redirecting everything (include directory listings) to mirrors is that you don't have download-statistics. This is where the following principle is a quick-win:
- Make sure all directory listing stays on our server
- When a file is accessed, redirect to a mirror
The implementation is fairly simple, only two files are required: loadbalancing.php for the redirection and an .htaccess file for sending file-paths to the loadbalancing script.
The .htaccess file looks like
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ loadbalancing.php?f=$1 [L,QSA]The first line tells Apache to only perform the rewrite if the request is !-d, not a directory. The second line sends the url to the php-script
And the loadbalancing.php file:
<?php
# A trailing "/" should NOT be present in the list of mirrors
$mirrors= Array(
'http://ftp.belnet.be/mirrors/FOSDEM'
,'http://ftp.heanet.ie/mirrors/fosdem-video'
,'http://fosdem.unixheads.org'
,'http://mirrors.dotsrc.org/fosdem'
);
$file=$mirrors[array_rand($mirrors)].$_SERVER['REQUEST_URI'];
header('Location: '.$file);
?>
<html><body>
If your browser doesn't redirect you should download the file manually using:
<a href="<?php echo $file; ?>"><?php echo $file; ?></a>
This script just takes a random element out of the array and sends an 302 to redirect the browser to the selected mirror. As every request, except the real download, is now done on our webserver we will be able to get full statistics of downloaded files while letting users use the directory listing feature of Apache. Statistics can be done with your favorite Apache-statistics tool.
Next story: adding geolocation to make the loadbalancing more efficient.
It may be interesting to have a look at http://mirrorbrain.org/