The Challenges of Technology ComplexityApril 27th, 2008
Recently I was asked to write a blog piece on IT complexity for a Dell website. It should be posted pretty soon, but I am posting it here also, and will add a link to the Dell site, once it’s there.
The complexity of IT — of the applications, hardware and management practices making up an IT portfolio– has become a huge challenge for IT professionals. We see the symptoms everywhere: from the growing percentage of IT budgets devoted to “keeping the lights on” to the increasing challenges of building new products and services on top of those already delivered.So how did this happen? How did things get so complex that our IT resembles the Winchester Mystery house? And what can we do about it?In part IT is getting more complex because it needs to be more complex. Every business is constantly looking for new products or process improvements, with IT invariably key to making this happen.Often we do this by extending old systems not intended for this new purpose, such as an old batch processing financials system suddenly asked to support real-time access or a Web interface.
Traditionally IT dutifully cobbles together the needed solution as expediently as possible — that is, cheaply, and quickly. And then we do it a gain, and then again. But over time the result can be hundreds of different applications running on different operating systems, hardware platforms and application servers, and woven together by a spaghetti-like web of infrastructural technologies, interface models, and data standards. And it potentially requires an army of C++, Cobol, PowerBuilder, SAP, Java and Smalltalk developers just to keep it all running.
And suddenly the board is wondering why is it so expensive to maintain all this, or add new features!
Even new, tech-savvy companies can fall into this trap Think for example of a successful four or five-year-old tech startup. When such companies are successful, they typically drive for rapid growth by rapidly acquiring competitors or ecologically-related firms: sometimes to acquire technology, but often to acquire customers or market share.
And what’s the result? Very quickly this small technology-focused firm has ten different customer information systems, seven application servers, eight ERP platforms, hundreds of incompatible data standards, all running on disparate hardware in twelve data centers and all of which need to be quickly integrated to support continuous rapid growth. And soon the focus is not on building new products – it’s on figuring out how to integrate all this stuff together, just to keep the business running.
It’s enough to make a grown programmer cry.
So how do we tackle this problem? How do we make IT ’simpler’, while continuously delivering more capabilities for the business we support? And how do we get the business to see the size and scope of the challenge, and pay to fix it?
The short answer is that there are no short answers: every business’s IT and strategic challenges are different, so an approach that works for one will likely be a non-starter for another.
But there are a few simple patterns that work for everyone, and that (almost) always lead to a solution.
The first is the ‘wall’ pattern. This is when the company hits a wall: things are so complex that they simply can’t execute their strategy, such as acquire another company or build out new features to retain parity with their competitors, because the technology makes it impossible to do so.
A second is the “it’s obvious” pattern. Sometimes (albeit all too rarely) the scope and nature of the problem becomes too obvious to miss. For example, if you’re a mid-sized company with three ERP platforms and ten data centers it’s not hard to build a scary diagram and a financials spreadsheet that will convince the board of directors that they have a problem!
But most companies aren’t so unlucky as this – most are suffering, but not to this degree. In these cases, more subtle approaches are needed. And these approaches often require a far more disciplined approach to IT management and metrics than is common in most IT shops today.