In this episode Andrew and Paul cover the high terrain of DevOps. We have come to understand that great software delivery is more about practices than process, and none are more important than the practices we collectively term “DevOps”. We’ll talk about what we mean by DevOps, what the underlying principles are and note some bear traps you should avoid. The second part of a two-part epic, we wrap up the principles you need to have a great DevOps story for your business.
This is part two of a two-part episode. You can read about and listen to part one here.
This follows on from the first part where we covered the first 5 principles of DevOps:
- Automate everything
- Git is the source of truth
- No sensitive data or values are stored in Git
- Adopt an Infrastructure as Code (IaC) approach
- Adopt an Immutable Infrastructure approach
We get straight onto the content by diving straight into the remaining 5 principles:
- Adopt an Immutable Application approach
- Especially pertinent to testing. We need to make sure that our application is isolated when regression testing so that we don’t pollute our results when other tests are running or other changes are being tested.
- This unblocks our release pipeline because we can test in parallel and not in series.
- A great thing here is that we can reduce our infrastructure costs using Kubernetes as we can create short-lived isolated applications for testing. Previously we might have had to have a large number of test environments to parallelise, at high cost.
- Each infrastructure environment gets its own Git repository
- This is a security isolation, but also good practice because we need to look at the demands on each different environment.
- Use Git branches to represent deployed instances of the same shape.
- We use GitHub Releases to trigger the deployment of changes to deployed infrastructure environments.
- Shared infrastructure services on Kubernetes, such as database services or queuing services, are deployed to the environment as part of infrastructure setup.
- Each application microservice and shared library gets its own Git repository
- Microservices are independently deployable and upgradable. This means that they have to have their own DevOps pipeline. More work to bootstrap a repo but more flexibility in the long run.
- This need not be high cost, DevOps pipelines are usually built on a cookie-cutter basis, using a standard template / pattern.
- Shared libraries can be packaged separately through a package manager or vendoring approach.
- Application configuration is environment specific
- Environment specific properties (not sensitive properties such as secrets, keys, certs etc) are output as part of infrastructure setup and stored in DevOps Kubernetes cluster.
- Application configuration is defined in the application microservice Helm chart as Kubernetes ConfigMap or Secret templates.
- Application configuration is treated like code, meaning configuration changes are subject to the same peer reviews and quality processes as code changes and Git is used to audit the changes.
- It’s the responsibility of the developer to identify configurable parts of their code. It’s the responsibility of us as leaders to coach and teach them.
- Application changes must be backwards compatible
- Independently upgrading individual microservices brings challenges.
- Especially relevant for microservices architectures, but good practice in general, if we’re making changes we need to support backwards / forwards compatibility so that we don’t break our system when we modify an interface.
- Ensure you can rollback when there are failures in an upgrade, or fix forward, depending on the failure and ability to roll back.
- Things to watch out for:
- Message protocols must be backwards and forwards compatible.
- Data changes must be compatible.
- Service interfaces must be compatible.
- Deleting elements from interfaces and name changes are particularly problematic.
- New operations are OK.
- New data elements are OK as long as you can deserialize / ignore them in your old versions.
- Changes that are not backwards compatible requires a new version of the service deployed that runs side by side.
You can watch the video here:
You can listen to the audio here: