Load balancing and scripted deployment with VoxCLOUD servers

Since scaling and hybrid infrastructure are hot topics around here, I thought I’d give you a snapshot of what’s possible when combining VoxCLOUD, hAPI, and post-install scripts; namely, grabbing the latest release of your application from an SVN repository and automatically addingyour new server into a load-balanced pool to serve web traffic — in under 4 minutes.

Let’s say we’re using the PHP hAPI client (available here in various languages) to create our VoxCLOUD, load-balanced VM. Our code would look something like this:

<?php
include('hapi_client.php');
$hapi = new hapi_client();
$hapi->fetch_key('your_voxel_username', 'your_voxel_password');
$hapi->call_api('voxel.test.echo', array(), HAPI_SIMPLEXML);
if ($hapi->get_error_message()) {
// you have not logged in correctly
}
// setup parameters for the server
$voxcloud_params = array(
'billing_type' => 'hourly',
'hostname' => 'loadbalanced.1.me.com',// this hostname would have to be
// incremented for auto-scaling
'facility' => 'LDJ1',
'image_id' => '16', // operating system id 16 is Voxel
// Server Environment (CentOS5)
'processing_cores' => '1',
'disk_size' => '5',
'swap_space' => '1',
'admin_password' => 'rootpassword',
'ssh_username' => 'myuser',
'ssh_password' => 'mypassword',
'postinstall_script' => $postinstall_script
);

// provision the device
$voxcloud = $hapi->call_api('voxel.voxcloud.create', $voxcloud_params, HAPI_SIMPLEXML);

// get the device id
$device_id = (string) $voxcloud-&gt;device-&gt;id;

// put together the server metrics monitor
$metrics_up_params = array(
'device_id' => $device_id, // device id
'monitor_type' => 'server_metrics', // statistics about the system
'max_system_load' => 10,// if load &gt; 10, send email
'notification_email' => 'alerts@server.me.com', // email to monitoring server
'notification_interval' => 600// wait 10 minutes
);

// create the metrics monitor
$hapi->call_api('voxel.devices.monitors.create', $metrics_up_params);

// put together the bandwidth monitor
$bandwidth_up_params = array(
'device_id' => $device_id,
'monitor_type' => 'bandwidth',// bandwidth of the server
'max_inbound_traffic_rate' => 104857600,// if bandwidth &gt; 100Mbps
'notification_email' => 'alerts@server.me.com', // send to monitoring server
'notification_interval' => 600// wait 10 minutes
);

// create the bandwidth monitor
$hapi->call_api('voxel.devices.monitors.create', $bandwidth_up_params);

$metrics_down_params = array(
'device_id' => $device_id,
'monitor_type' => 'server_metrics', // statistics about the system
'min_system_load' => 1, // system load below 1
'notification_email' => 'alerts@server.me.com', // send to monitoring server
'notification_interval' => 600// wait 10 minutes
);

$hapi->call_api('voxel.devices.monitors.create', $metrics_down_params);

$bandwidth_down_params = array(
'device_id' => $device_id,
'monitor_type' => 'bandwidth',// bandwidth of the server
'min_inbound_traffic_rate' => 10485760, // if bandwidth &lt; 10Mbps
'notification_email' => 'alerts@server.me.com', // send to monitoring server
'notification_interval' => 600// wait 10 minutes
);

$hapi->call_api('voxel.devices.monitors.create', $bandwidth_down_params);

?php>

The above script would create a 1-Core + 2GB RAM VoxCLOUD VM with four monitors–two for maximum system load and bandwidth, and two for minimum system load and bandwidth. An email would be sent to alerts@server.me.com in the event that bandwidth usage to this server went over 100Mbps, if the system load went above 10, if bandwidth went below 10 Mbps, or if system load went below 1. You see where this is going.

The email addresses alerts@server.me.com could be monitored easily for incoming alerts. After receiving an alert, it could perform certain actions like provisioning another monitored server to scale up, or de-provisioning an existing server to scale down. So, how does the server we’ve provisioned get automatically added into the load-balancing pool? That’s where the $postinstall_script comes in.

Before jumping into the code, it’s important to understand how Voxel’s load balancers work. They distribute traffic to customers’ web servers by creating a Virtual IP (VIP). This VIP is what a site’s DNS is pointed to, say 123.45.67.89. The load balancer is configured to take all web traffic sent to that IP address and distribute it to the “real” IP addresses that the web servers are listening on. These web servers are setup to listen on both the real IP address and the virtual IP address. Why? Because it allows the server to respond directly back to the client with the data. The load balancer delivers traffic to the web server; it doesn’t have to send it back. This is important, as it reduces strain on the load balancer.

On to the post-install script contents:

#!/bin/bash

### custom variables
LOAD_BALANCED_VIP="123.45.67.89"
SVN_REPO="https://svn.mydomain.com/svn/repo/branches/RELEASE_1_0/"
SVN_USER="username"
SVN_PASS="password"

### loopback text to gets passed to ifconfig for the load-balanced VIP
LOOPBACK_INTERFACE="DEVICE=lo:0nIPADDR=$LOAD_BALANCED_VIPnNETMASK=255.255.255.255n";

### this is the real IP address of the server
IP_ADDRESS=`ifconfig eth0 | awk '/inet addr/{print $2}' | sed 's/addr://g'`

### install needed packages (for now, PHP and Apache)
### broken into two lines for readability
yum install -y php-devel.x86_64 php-common.x86_64 php-cli.x86_64 httpd.x86_64
yum install -y httpd-devel.x86_64 apr-devel.x86_64

### grab code from SVN repository and put it in our web directory
cd /var/www/html/
wget -mnp -nd --user=$SVN_USER --password=$SVN_PASS $SVN_REPO

### remove the index.html that gets downloaded by wget
find /var/www/html/ -name index.html -exec rm {} +

### make a backup of the current httpd.conf
cp /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf.bak

### edit the just-downloaded httpd.conf with the real VIP and real IP
sed -i "s/REPLACE_WITH_REAL_IP_ADDRESS/$IP_ADDRESS/g" /var/www/html/httpd.conf
sed -i "s/REPLACE_WITH_VIP/$LOAD_BALANCED_VIP/g" /var/www/html/httpd.conf

### move the svn'd httpd.conf to its final resting place
mv /var/www/html/httpd.conf /etc/httpd/conf/httpd.conf

### create the local loopback interface and bring it up
echo -e $LOOPBACK_INTERFACE > /etc/sysconfig/network-scripts/ifcfg-lo:0
ifup lo:0
### finally, start apache
apachectl start

This post-install script assumes that the root directory of the SVN repository has your generic httpd.conf file that you want to use for each new web server. For the `sed` command to work, the httpd.conf would have to have thefollowing two lines in it:

Listen REPLACE_WITH_REAL_IP_ADDRESS:80
Listen REPLACE_WITH_VIP:80

This would tell Apache to listen on both the VIP as well as its own IP address.

I’ve personally used the above to create load-balanced web servers in under 4 minutes–now, you can do the same.