In this series of blog posts I am discussing aspects of Enterprise Application Configuration and how we have come across and resolved issues in real-life mission critical systems.
When I talk about Enterprise Applications, I generally refer to applications that are (a) mission critical and (b) distributed across multiple machines. Sometimes (c) geographically dispersed across different data centres also applies.
In the first post in this series I discussed briefly some issues regarding the retention and access to version history for configuration in enterprise applications. In this post I’m going to look at the most valuable aspect of keeping a version history: the ability to roll back to the last known good version.
I have seen operators copying bits out of configuration files, pasting the snippets into notepad (or emails, chat windows etc.), or doing a “copy and comment out” change to configuration files in order to help them have something to roll back to if their (albeit ill-advised manual) configuration change does not succeed, or even makes things worse.
I have also worked in better environments where all changes are automated, but even in these cases the final configuration often comes from a combination of a template + build tokens, each of which may have been subject to change prior to a deployment. Whilst it is possible to roll the entire application back to a previous state it is often not possible to have a previous version of the actual configuration once a new deployment has been made, replacing the old version.
Whilst I think that deployment and DevOps automation is the way to go in almost all cases, the 345 way of thinking about configuration only serves to build on these good practices, not replace them.
We think that an important aspect of any Enterprise Configuration Store is support for versioning and rollbacks, and we have thought deeply about how best to achieve this in practice.
When we developed cloco (Cloud Configuration), our Enterprise Application Configuration platform, we recognised that saving the version history for diagnostic purposes was only part of the story. We also wanted to provide tooling to allow and support rollbacks.
The cloco configuration store is accessed via a REST service, with the clients never needing to actually see the raw data stored there. We also wanted to build on top of this the tooling to make the API transparent, to expose the functions of the REST API in a way that can be consumed readily.
When we developed the cloco Admin Console, we aimed to give cloco users and administrators access to view version history. We always intended to allow them to also initiate a rollback of the configuration to a previous version. When you’re developing software you would expect to be able to roll back a class file in your source control, but to be able to roll back a configuration section in production? We think it’s needed.
We are open minded about how many previous versions we are likely to support (5? 10? 20?) we recognise that for a system to be practical there should be a reasonable number of previous versions available, and that administrators should be able to select any of the previous versions to roll back to.
