Where staging environments fit in the lifecycle of high availability web apps

One of the more common discussions Voxel sales engineers have with potential clients is around whether staging environments are a necessary part of a hosting deployment. Our answer is always a resounding YES, especially for critical web apps undergoing rapid development. Unfortunately, staging environments often have a bad rep — some developers see them as a throwback to less agile programming days, product managers just feel there are too many extra hoops to jump through to see their latest feature and business owners often just see the cost and think its an extra expense.

Luckily Voxel has a team of great sales engineers who have years of experience and knowledge around managing high performance web stacks. They know that the secret to every great production environment is a solid staging setup behind the scenes and a true commitment to testing each software change against the staging infrastructure.

What is a staging environment, exactly? In the Voxel world, it’s a reproduction of your production deployment – machine configuration, build specifications, application versions, etc. – that is used to “stage” upcoming code releases. All releases are tested first on staging and then pushed without changes to the production environment.

To make swallowing the cost of staging environments more palatable, Voxel leverages our flexible, on-demand infrastructure options to fluidly create, expand and destroy staging environments using automated configuration and deployment tools. This not only streamlines the process, but lowers costs and increases the accuracy of your environment configuration.

In order to build a proper staging to production workflow, however, several other things need to be in place in your software lifecycle. Let’s review them here:

#1 – Develop It
Developing software is where it all begins. The team may be just one person or a distributed set of workers around the world constantly adding code to your application. Either way, how you develop code and manage that workflow needs to account for the staging and production role separation.

Most of our customers develop locally on individual work stations or in the cloud via VoxCLOUD servers and shared dev environments. And in these cases, a virtualized setup can be really advantageous. Each developer can have a ‘production like’ environment without the cost or overhead of operating large server groups for each developer. Or, they can clone an existing dev environment to test out a major upgrade or a set of code changes as part of their workflow. Plus, with the automation behind Voxel’s infrastructure products, many developers simply have Voxel build a pre-configured dev environment on demand each day, powering it off or destroying it when not in use.

#2 – Repo It 
Again, regardless of if you’re 1 developer or a huge global team, using a code repository like SVN and Git is a critical part of the development process. Beyond their uses in the actual development process, these code repos are great for automating deployment, testing new code and most importantly, rolling back to previous versions of code if something breaks. If you’re interested in learning more about various code repo solutions, look for more information in future posts where we’ll be dissecting a few different tools and strategies.

#3 – Stage It
Your staging server should really be for two main things:

• Testing code changes against production level libraries, versions and permissions (where only the code has changed)
• Allowing developers to step out of the deployment process and put operations people (such as Voxel’s ProManaged support engineers) in charge of releases and rollbacks.

A staging environment allows you to separate your development server, with its ever-changing code, from your production environment, which should be the final product pushed to customers. You can catch bugs, performance issues, version conflicts and other critical issues before you put any code into your production application environment. When new code is pushed into production on release day, you can feel confident that those mysterious differences between your dev server and production server won’t exist and you’ll enjoy a worry-free upgrade.

Configuration Management To the Rescue
To those that have managed dozens or hundreds of servers before, they know that keeping the production and staging environments “in sync” can often be a challenge. Over time, one environment might get a slight version upgrade or software patch that the other didn’t. Or a reboot on the staging side may cause a new kernel to be put in place that nobody noticed. This kind of variation completely eradicates the benefits of the staging environment (because it no longer perfectly matches production!) and can be deadly to any kind of testing regime.

To solve this problem at Voxel, we’ve integrated Opscode’s Chef platform for configuration management and cloud infrastructure automation into our internal tools and customer deployments. Chef allows Voxel to quickly and accurately scale server infrastructure and to rebuild any environment exactly the same way every time. This kind of configuration automation saves development time, trouble shooting, and system administration hours. This all leads to saving money as well as having better uptime and quality.

Using a robust configuration management system like Chef is incredibly important when moving from staging to production. It ensures the accuracy of your IT configuration and makes sure everything is installed, configured, permissioned and tasked just like your production side. Once you know your staging environment matches your production side exactly, you can effectively test on your staging environment and feel confident when it comes time to push that new code to production.

Conclusion
Despite popular belief, most downtime is not caused by network outages – it’s caused by application changes, unknown system variances, exploits and conflicts. Staging servers help you mitigate these issues, while also giving your hosting partner the proper process and control to support the uptime of your production application. With per-hour VoxCLOUD and VoxSERVER infrastructure, automated configuration management using Chef and a comprehensive approach to your dev to testing to production code process, you can easily obtain better performance and high levels of uptime for your application.