I believe everyone starting a software project should start their DevOps on the first day [of the build cycle] of their project.
I believe that failure to do this leads to bad places almost every time, and the more complex the solution the worse mess you can get into.
This is my manifesto for getting your DevOps lined up from the start of a project.
What is DevOps?
DevOps is the term used to describe a set of practices used to automate the delivery of software and infrastructure. Most software delivery best practices have incorporated automated build / Continuous Integration (CI) for a long time, but as automation extends from the developer’s code commits up to the point of deployment the range of practices involved has expanded to include scripted provisioning of infrastructure, automated deployment and automated testing.
There is no strict definition of DevOps, but I’d put a stake in the ground to say that if you are manually changing settings on any server in your test or production environments then you need to improve your DevOps.
Manual changes are not repeatable. Without repeatability you cannot achieve consistent quality.
Manual processes are not scalable. Without automation you cannot improve productivity.
No excuses
I’ve been on too many projects where deployment is left too late. I’ve heard a lot of excuses. I’ve yet to hear a compelling one. I just hear that some people aren’t interested in quality.
Excuse: We don’t know what DevOps is.
Retort: Don’t let your ignorance be an excuse for poor practice. Learn. There’s tons of information out there.
Excuse: We don’t want to spend the money on hardware yet, so we don’t need DevOps.
Retort: What? You’re happy spending money building something, and not knowing if it works, but you can’t even stand up a few VMs?
Excuse: We haven’t designed the infrastructure yet.
Retort: What? You don’t even know how you’re hosting your solution yet you’re willing to take the risk building it?
Excuse: We don’t have the expertise to build the infrastructure [or deploy the solution] yet.
Retort: Concentrate on building your DevOps expertise before you start building software.
Excuse: We outsource that to someone else.
Retort: Exactly how will you be in a better place by getting them to do this later?
Excuse: It will take too long.
Retort: Exactly how will you be better off if you burn that time – and more – later on, when your project is at a more critical stage?
Feel free to add more lame excuses you’ve heard in the comments section. I’m fuming already.
When to start
Now.
We all start building a new application with something resembling a “hello world” app. Even if we’ve just initialized a new repo on Git, we can create:
- An index.html with static text for a website.
- An API route that GETS “/status” and returns a 200.
- A background service that writes out to a log.
Literally, within minutes of starting a new software project, you can have a few lines of code that do something trivial that demonstrate running code. It is at this point that you should deploy your code.
Don’t leave it till later. That is a path to bad things. Deploy now. You now have enough code to:
- Set up your source control repo, branching structure, permissions.
- Set up your CI and automated tests.
- Set up your target infrastructure and DevOps scripts.
- Set up your deployment scripts to automatically deploy.
If you wait to add devs onto your project until your quality processes are in place you won’t regret it. Not for a moment. You might also uncover [early!] which developers are used to achieving high quality and which aren’t. And you might teach the ones that aren’t a lesson that will benefit them for the rest of their career.
Impact of cloud
At some stage we’ve all been forced to manually deploy some code onto a server at the last minute because something went wrong. Usually it’s not a pleasant experience.
The world of cloud computing makes the bad habit of manual deployment unsustainable. On the other hand, it makes the exercise of scripted deployment easier as there is no physical infrastructure to provision. Provisioning of infrastructure and deployment of code are all just lines of script. And usually fairly brief.
Cloud infrastructure richly rewards those with good practices and scripted, repeatable processes. You can spin up a test environment in seconds, run a suite of tests, and then tear the environment down when it has only cost you pennies in CPU time. Usually this can be achieved with a handful of lines of script.
Cloud is your friend. Embrace it.
Penalty of leaving it late
As your codebase gets bigger, with more contributors, it gets harder and harder to get your deployments working. All the time you’re trying to deploy, you have colleagues that may break your deployments.
If you can get your deployments and test runs green after your first day of code, make it your developers’ responsibility to keep them green. Human nature being what it is, if you’re trying to deploy to new environments it will be your problem until you get it fixed. If you have a working deployment the responsibility should fall on whoever breaks it. This works in your favour, so get it right on day 1.
Bigger picture
It’s easy to get devs to start cutting some code, that’s what they all love to do. A Solution Architect can identify applications and services, and you can get working on features and everyone’s happy.
It’s a different mindset to consider deployment from day 1. It is often the case that the design a Solution Architect may recommend is driven by features and functional requirements, but it is the physical / virtual infrastructure that will dictate how non-functional requirements are supported.
If you need to put in place realistic infrastructure on day 1 you need to have a grip on how to meet performance, scalability, reliability, security, upgradeability and a whole host of other non-functional aspects of your solution. It helps immensely to get these baked in early so that you are building on solid foundations.
Done means done…when?
One of the main problems with project management on software projects is when people estimate for features based on how long they take to code. If you allow a developer to code a feature as “works on my machine” and declare the task done then you’ve lost.
You can’t even rely on unit tests passing, you have to look at test coverage as well. Unit test coverage should be as close to 100% as possible. I can’t put a number on where you should draw the line, but it needs to be high.
You should also be testing features for completeness as well, so functional tests encompassing user stories or business scenarios will also need to have full coverage. This should absolutely be at 100% for a feature to be considered done.
The more features you have, the more you will be in a position to run load on your system, perform security penetration tests. You will be “upgrading” continuously, as your app will already be deployed.
Done really means done when the feature works in production and passes all tests (with full test coverage). Day 1 DevOps gives you the surest path to this.
In summary
No excuses.
Start now.
Script everything.
Never compromise quality.
