NCCM and the unfulfilled promise of closed loop remediation

DSCF0102There are a lot of benefits to Network Configuration and Change Management (NCCM) not the least of which are visibility, control and change automation. But the “Holy Grail” that is often pointed to is the ability to do closed loop remediation where the tools can be used to detect change, confirm its validity, and remediate it directly, if out of compliance. This is good story, but when looking at actually putting it into practice there seems to be a reluctance usually tied to the need to conform to their change management process, often around obtaining CAB approval.

I’ve been discussing this issue with my colleagues as we try to understand what is at the root of the problem. Certainly, it is not the tooling which is lacking. NCCM is an extremely mature area and the technology is more than capable of the demands. The best of the tools not only provide built in compliance and remediation capabilities but API access that allow integration into a larger service management platform. But NCCM, by its nature, is dealing with the traditional infrastructure of the network and so it seems there is still the desire to try and manage change with the traditional processes. For all the benefits that have come from paradigm shifts such as DevOps and Continuous Delivery, the legacy network is not an area where significant inroads have been made to date.

This is not to say that there is not a desire to adapt. In our own experiences we have seen the security groups in organizations willing to embrace the idea of auto-remediation to address compliance issues but still have to overcome the challenges of the IT change management procedures.

Why this reluctance to adapt? One of my colleagues and resident NCCM expert, Andrew St. Jean, noted that perhaps the reason for hesitancy is a historical one. In the past, unscheduled changes were made but often undocumented. An admin would make a change to a device to deal with a real time operational issue but there was no record of it having happened. At best this contributed to uncertainty and configuration drift, at worst service failures. In order to combat this, all changes were pushed through the CAB which, in addition to providing a venue for sober second thought, provided the likely more important effect of documenting the changes that were occurring.

But in a world where NCCM provides an audit trail for any of the change that it invokes, the documentation is built in. Furthermore, even if modifications are made out of band, NCCM is still capable of identifying and capturing any changes that have occurred.

So, perhaps it is time to start thinking more seriously about the process of change management and how to apply it in a world where change is becoming the norm rather than the exception. Is it really necessary to force all network change through the CAB? Can pre-approved change be utilized more effectively to allow closed loop remediation to become a reality? Maybe we need to start having the same sorts of discussions around change in the network that have driven advances in modern server and application deployment. Maybe it’s time for the process to catch up to the technological capability and allow NCCM to live up to its full potential.

This post was originally published on the Infoblox Community Blog.