Humans are fallible. To err is human. These are noble assertions when it comes to living the human experience. But when it comes to application deployment, humanity is the weak link in the chain. In this post I’ll explain why and offer arguments in favour of automation.
We like to be in control. We want to make sure a process is done properly, and ultimately our bosses want to know we provided the necessary oversight. But how many manual deployments have you participated in when someone inadvertently omitted a step? Sure, you documented it clearly in the deployment instructions, and reviewed them with your implementers in the days leading up to the big deploy. Even so, that omission caused some sort of delay or worse, cause a data corruption. Now you are still there late Sunday night cleaning up the mess. There has got to be a better way.
There is a real life story about the consequences of a manual deployment that failed and 45 minutes after go-live the company was facing bankruptcy. Knight Capital deployed an app to all but one of its eight production servers. It was a manual deployment where the implementers followed a set of steps to deploy the application to each server. Some dead code was triggered in the eighth server by an unused flag that was re-purposed for the new version. As a result, the eighth server caused thousands of trade orders to be executed per second. There was nothing to stop this runaway trade execution. After 45 minutes, the company had assumed a liability that exceeded their current assets. They were facing bankruptcy.
One of the big lessons here is to develop and test automated deployment scripts, then introduce these into your production deployment processes. The benefits are threefold:
- Consistent deployment across all instances in your cluster, all the time. Computers are great for doing the same thing every time. Deployment scripts will perform the same actions on all instances every time you run them. Such consistency breeds confidence.
- You’ll notice your deployment time shrinks. The time you spend executing the deployment is shorter than the time you would spend doing it manually. Granted, the first few times you do it you’ll be thoroughly verifying the results. But as you gain confidence, you’ll notice what took many hours takes minutes.
- You can deploy more frequently. Now the deployment effort is much less of a constraint on when you can deploy.
How Do I Get Started?
So how do you get started on automation? If you are still using an application server to host web applications, look for a command line utility from the vendor of your app server. See if you can make shell scripts that invoke this utility with whatever options or subcommands it needs. There are deployment tools such as Ansible, Chef and Puppet that you might be able to use in your situation. Even operating system shell scripts written using Bash or PowerShell are a good start.
The major cloud vendors have solutions available to automate your deployment. Amazon Web Services has their CodePipeline and CodeDeploy services. Microsoft Azure and Google Cloud Platform have their equivalents. Check them out.
Happy automating!
