As someone going through this right now, you are right. No companies that sell software as a service avoid the integration problem, at least in the industry I'm investigating. Some offer to do it for you, but then you have all of the joys of outsourcing ("the requirements weren't clear," "there was scope creap," "the business needs changed in the middle.").

Where the utility model fails is that almost no one differentiates themselves on the kind of electricity they use (and those that do don't rely exclusively on the power grid - e.g. IT data centers, which have the modern equivalent of the great water wheel in every building - they just use it as backup). Every mid-size and up company differenciates itself in some delivery capacity that is reliant on computer technlogy to make happen. And no technology currently exists, and is even hard to imagine in theory, to make such a wide range of options programmable by someone other than programmers. The complexities always get to the point where programming is required, even if they built their own custom language or GUI to do it.

ANother way to look at it is the interface. Electricity has a defined interface protocol, actually more than one: 120 volt, 60 cycle, single phase; 240 volt, 60 cycle, three phase; 220 volt, 50 cycle, single phase. Hmmm, ever try to use a US spec appliance in Europe?

I suppose it is theoretically possible to write code that would auto-negotiate an interface with any other module -- but Note Pad would probably run to a million lines of code for a simple text editor. Why would any software company ever have a financial interest to write and maintain such a thing?

When "utility computing" reaches the level of simplicity and reliability from the point of view of the consumer that electricity does, we can talk about security and all of the other issues you mention. Not that those issues are unimportant, but worrying about security, regulatory compliance, etc. if you can't even get basic access to your data consistently, reliably and easily is a waste of effort as the whole thing is a non-starter.

What does consumer need to do to use electricity?

You need to choose your vendor (if you have a choice), figure out what devices need what kinds of UPS or surge suppressor, and then plug things in. That's it. 5 9s of reliability, and it just works. Yes, you still need an electrician, but if he's done his job right, you'll be seeing him rarely, and most of the jobs are routine.

And, it's worth noting, that really big firms often do NOT "outsource" their electrician functions, because for them economy of scale works in the other direction. If the simplest of simple to use utilities can still support an "in house" support function in some cases, the idea that something orders of magnitude more complex won't sounds a bit far fetched, to say the least.

I can see how it can be hard to fast forward several years and boldly state, "this is where things will be." I don’t necessarily think Carr is wrong, but more than likely, it'll be some shade of that truth and its perhaps a bit more nearer to reality that Bob’s currently giving credit for.

I haven't read Carr's book or predictions myself, but I definitely get the general picture from what Bob's analysis. I first joined a company in 1999 that set out to deliver HR end-to-end solutions in ASP model (utility computing for HR!). We lost our angel funding and suffered the all-to-common dot-bomb consequences of the day.

Businesses will find a way to implement their processes ever more efficiently and that is the key. A decision maker will look at options at the time (s)he needs to make a decision which direction to go in and decide either a) build it all in house b) buy something off the shelf and run it in house c) buy a hosted solution d) a hybrid of the three.

Hosted solutions have gained a lot of steam in last ten years and I see no reason not to see more and more offerings that begin to put in-house solutions at risk.

If you want to see where technology is heading and what the likelihood of a company adopting an "all utility" model where they have nothing in-house and everything outsourced, just look at two year old companies each year and see what the best of the breed are doing. These companies will be most open to embracing externally hosted solutions and integrating them to fit their needs. They will be agile enough still to formulate their processes around the technologies or at least aggressively pursue finding external solutions to plug the obvious holes in their business needs.

When I look at these companies, I see a lot of progress towards such a model, but there’s also still a lot of room for growth, changes, and adoption. However, I also see a correspondingly increase in number of companies providing the latest hosted solution for a particular area that makes utility computing ever-more appealing to established institutions and new-comers alike.

For our company, I am responsible for our network services (data center, infrastructure software, and tools that help everyone get the job done). I also do daily software development and meet with our clients on a regular basis, so I tend to see the picture end-to-end while being quite hands-on with the technology itself.

We build software specifically targeted at medical researchers and we provide it in two flavors, a) installed on-site in our customer’s data center where their team manages the servers and systems and b) utility hosting model where we host the system in our data center.

Utility computing is not necessarily an all or nothing model and I think most analysts in the media completely miss the boat here time and again. We outsource all our communication service 100%, and that includes a hosted VoIP, hosted email/groupware and cellular service. Believe it or not, our voice mails can be sent to our emails and our emails and calendars are easily synced to our smart phones and desktops. And this is despite our service hosts knowing little to nothing about each other. We’ve simply selected a suite of services that work together through a common denominator.

As a counterpoint to the above, we don’t likewise outsource our CRM because our approach currently isn’t really modeled well with what’s out there now, but we can munge services together with a select few open-source and proprietary tools to effectively give us everything we need to drive our sales effort.

Another case in point: we have one particular client that employs both solution models we offer. It just so happens one department has an already over-taxed IT support group while the other division that purchased the same software system has a very large IT support team that could absorb our system onsite in their daily operations.

The pattern of change is there. Utility computing is on the way and more and more companies will take advantage of that. We’ve had lively debates within the company on whether to continue to grow our own data center or take advantage of those large “electronic computing cloud� data centers that are popping up. Only the future will tell which way we shall go. Right now, our main hurdle is not technology or pricing. Our challenge is HIPAA compliance and ensuring our partners are, too.

We’re virtualizing nearly everything these days and can commission a new server and service running on it in a morning. Once every server is virtualized, what’s to keep us from seamlessly moving that service anywhere in the world?

If more and more firms are likewise heading down this “virtualize everything� road, then watch out as today’s most talked about model of utility computing where the emphasis is usually service or sector specific (outsourced email, hosted CRM, etc.) is supplanted (or more likely simply supplemented) by service agnostic solutions to the point that all we’re really buying is reliable processing capacity, electricity, and Internet connectivity. At that point, *onsite* backups becomes most critical to data and system recovery rather than *offsite* backups!

How complex is the job you ask of your car? Least common denominator:
* Go faster
* Go slower
* Go backwards
* Turn left/right
* Enable/disable headlights/taillights/brake lights/turn signals
* Set/adjust/disable cruise control
* Set/adjust/disable wipers

Now look at the number of interfaces different cars use to accomplish those functions. Controls in the dash, on the stalk, on the wheel, voice control, iDrive, etc. etc. etc.

Next, take a look at the set of remote controls for a well-stocked home entertainment center. Still a discrete set of functions you want to accomplish, but the controls need manuals of their own.

But what's the set of functions you want a computer system to accomplish? Change. You most important thing you want from your system is the ability to change its behavior over time. It's a general-purpose device.

Why does anyone think there can be a single unified interface to a general-purpose device specifically designed to execute arbitrary instructions?

A truly fascinating discussion…

I’ve experienced what Mike Swaim pointed out… IT systems may induce a “dumbing down” effect among IT system consumers. At first, consumers are reasonably knowledgeable about the systems they use, probably because many of them were involved in requirements gathering during system development. Attrition tends to erode such understanding until you have a work force that just knows they are supposed to “push certain buttons” to perform routine activities. Exceptions? The lead analyst (substitute supervisor, team lead, rehired retiree, etc…) handles all those.

I now see the same thing happening in IT circles. The most knowledgeable developers eventually move on and then the IT system is supported/administered by analysts who have little or no knowledge of its intricacies. So now we have ignorance twice removed! And from there, it may get worse exponentially. The end result is a bit like monkeys finger-painting on the Mona Lisa…

It’s no wonder management yearns for turn-key, commodity solutions (just never mind they are largely responsible for creating this dilemma…). And it’s also easy to get caught up in the insanely frustrating challenges within such an environment. BUT it is also a time of great opportunity. While everyone is busy imitating each other, the amount of low hanging fruit increases noticeably for anyone who can still think for themselves and has the guts to act.

Hi Bob,

Your posts here and elsewhere about cloud computing and other IT stuff make refreshing reading. It's great to see the words "business", "IT" and "engineering" in such close proximity!

In order to discuss some of the issues surrounding The Cloud concept, I think it is important to place it in historical context, looking at the Cloud's forerunners and the problems they encountered before being adopted.

On my blog I've tried to do that in my Cloud Computing post, where I've linked to you and some others.

One of the current barriers in the way of The Cloud is economics. I argue that,

'Telecom prices have fallen and bandwidth has increased, but more slowly than processing power, leaving the economics worse than in 2003'.

And 'I'm sure that advances will appear over the coming years to bring us closer, but at the moment there are too many issues and costs with network traffic and data movements to allow it to happen for all but select processor intensive applications, such as image rendering and finite modelling'.

Your comments and feedback are very welcome.

Turning back to "engineering", in my view IT exists for one reason: to manage the flow of data between business assets.

This is a concept which I believe will be important for business and IT in the years ahead.

If you take this approach, then techniques used in the Oil & Gas industries for many years can be applied in any sector.

In Oil and Gas, the introduction of digital monitoring equipment means flows of product are analogous to flows of data. These flows are displayed, monitored and trended in dollars per second.

This means business processes can be optimised around value, and the contribution each asset makes to the cost/value of the flow can be evaluated in business (monetary) terms.

Today, business resources (which includes people) and IT assets are either providers of data, consumers of data or provide the conduit through which the data can flow.

The role of IT is to support, process and optimise the flow of data to maximise business performance.

OK, what's that got to do with "engineering"?

Well, one of the main differences between professions like engineering or architecture and IT is that for many years they have been fully documented.

Like IT, the very complex projects they manage involve many related assets, processes and people. Yet unlike IT, the business and the professionals can easily understand each other and these days disasters are fairly rare.

Why? Because they have simple means of communicating with each other. After all, how could complex things like skyscrapers or bridges be built without blueprints or engineering diagrams?

It is this easy to understand big picture of the business and IT relationship that has been missing.

To create this picture, and enable business and IT to speak a common language, understanding dataflows is critical. It is the understanding, documenting, and engineering of them which is key to managing complexity.

If we have a simple picture of how each dataflow moves across and through the assets of the business the responsibilities, roles, risks and costs of every IT resource (or group of IT resources) employed in support of each business activity (and/or set of business activities) can be clearly visualised and, thus, understood.

By attaching value meta data to data flows and cost information to IT assets, we can start to assess the ratio between IT support costs and the value of the contribution of IT to the business.

Which means IT can speak to the board in the language it understands, that of money. It also means that IT will be fully documented, providing a standard for governance and a foundation for professionalism.

If you would like to read more about the above you might like IT exists for one reason.

And you can see a couple of the 'big pictures' here. These B&ITs (Business and IT diagrams) were inspired by the P&IDs (Piping and Instrumentation diagrams) used by engineers the world over.

Regards

PJW