MORE ENTRIES

November 2007

November 28, 2007 | Comments: (0)

Joe McKendrick does a good job in highlighting a new notion: "How do you measure SOA success? Try IT backlogs." This was from a Podcast posted by Hub Vandervoort, CTO of Progress Software. Clearly, that's the case, but just looking at IT backlogs could be a bit too rudimentary when considering SOA.

"In companies implementing SOA, the overall backlog of IT requests is starting to shrink, Hub pointed out. Companies are seeing drastic reductions in project cycle times and consolidation in purchasing across the enterprise."

The fact of the matter is that the core benefit of SOA is agility. If you have agility, then you have the ability to change the architecture as the business needs changes. Thus, if you have IT backlogs, naturally they will decrease because it's easier to make these changes. Seems logical to me.

The trouble with using IT backlogs as a core metric is that it's after the fact. Thus, we can measure ROI after we've implemented our SOA, which is great; however the rubber meets the road in determining the ROI before you fund your SOA. Indeed, most business leaders are going to demand a business plan before they will provide funding, and you'll need clear ROI for the business plan. Just promising reduced IT backlog won't be enough, you need to define the increased efficiencies at all levels, including business and IT. SOA is promising much more than just reduced backlogs, at the end of the day. That's just one benefit.

In thinking about the ROI around SOA we need to think at a bit more sophisticated level. While positive outcomes such as reduced backlogs will indeed be a result, there are many other things to consider as well, and they need to be considered before, and after the projects. Measuring afterwards means that we're being reactive, rather than proactive. That's not good.

Posted by Dave Linthicum on November 28, 2007 07:37 AM

November 28, 2007 | Comments: (0)

Review of my SOA Predictions for 2007

Download file

Posted by Dave Linthicum on November 28, 2007 03:59 AM

November 21, 2007 | Comments: (0)

Dimensions of SOA testing

There are many dimensions to SOA testing. They include services, processes, and performance.

Service-level testing is the most important, since core services are the foundation of the SOA. However, services are written very differently, depending upon the developer, with some services perhaps too course-grained, and some service too fine-grained, and many other services poorly designed. Services may also be built on top of existing interfaces and APIs, and thus are even more complex and more in need of quality assurance testing since you're placing an interface layer on top of an interface layer. There is no magic here; it's a matter of validating the services for their intended use, verifying that the interfaces function correctly, and validating both WSDL and schema. Also, you need to consider diagnostics for design-time and run-time, and make sure to address those older, but important notions of unit, functional, and regression testing.

In addition to service-level testing, we also have to test the way services are abstracted into processes and composites. Since these are typically exposed as services themselves, it's just a matter of testing another level up from the core services, as units, and regressing down through the services that they leverage (unit and system). This is very much like testing object-oriented systems, but these guys have binary interfaces and heterogeneous development and run-time platforms, thus the complexity is much higher.

Performance testing is perhaps just as critical, considering that most of the quality problems I run into when deploying SOA relate back to performance. Here is where you test against the SLAs established within the project, and learn how to spot bottle necks, such as slow services, that can bring your SOA down to a crawl. Performance testing in the world of SOA is a matter of testing at the service, composite, process, and system levels. You look at overall performance first, then decompose the architecture down to functional primitives to isolate the system's problem components. You need to create an ongoing performance testing approach since so many performance issues develop over time as message and data traffic increases or changes.

What's key here is to remember that you're testing architecture, and not an application. Thus, the complexity of the system, and the approaches and tools used for testing, goes way up. It's important that you have a solid test plan, an arsenal of testing tools and techniques, and the time needed to test the architecture properly to correct any problems before they are found by the end user. Once the end user becomes the tester, it's difficult to get your creditability back. Consider the systemic and business critical nature of the architecture. Any quality issues that exist within your SOA will be amplified.

Posted by Dave Linthicum on November 21, 2007 05:03 AM

November 18, 2007 | Comments: (0)

How to Consider Orchestration

No Podcast this week due to the holiday, so perhaps some more heady topics for those of you, like me, are working.

The best way to consider both notions of service and orchestration (and process integration in general) is to think of them as independent layers, where the process layer is the calling application to the services to extract both data and behavior needed to form cohesive orchestrations. This interaction allows the architect and the developer to place certain things into certain domains. The orchestrations that define the way the services are leveraged, and the services that…well…provide service.

Orchestration is a necessity if you build a SOA, intra- or inter-organization. It's the layer that creates business solutions from the vast array of services and information flows found in new and existing systems. Orchestration is a godlike control mechanism that's able to put our SOA to work, as well as provide a point of control. Orchestration layers allow you to change the way your business functions, as needed, and to define or redefine any business process on-the-fly.

For our purposes we can define orchestration as a standards-based mechanism that defines how web services work together, including business logic, sequencing, exception handling, and process decomposition, including service and process reuse. Orchestrations may span a few internal systems, systems between organizations, or both. Moreover, orchestrations are long-running, multi-step transactions, almost always controlled by one business party, and are loosely coupled and asynchronous in nature.

Orchestration encapsulates and leverages services, binding them together to form higher level processes and composite services. Indeed, orchestrations themselves should become services, and may be leveraged as services.

Here are a few things to consider:

• A single instance of orchestration typically spans many instances of service- and information-oriented points of integration, perhaps many domains and even organizations. Thus, you typically have one orchestration for many services.

• Most orchestrations leverage public standards such as BPEL. However, there are a few competing standards, such as process and workflow standards, and technology that can do very much the same job…binding services into processes to form solutions. Thus, when considering the process/orchestration solutions, you have many to choose from that provide process-based control, using or not using BPEL.

• Orchestrations may be public, or available to everyone, private, available to just the owner, and shared, for supply chain integration scenarios. Thus, orchestrations can span businesses, supporting supply chain integration and other B2B activities.

• Orchestrations are usually driven from a single party; they are not always collaborative in nature.

• Orchestrations themselves may become services available for other services or orchestrations (as mentioned above).

• Orchestration defines a meta-application, of sorts, that has visibility into many encapsulated services as well as application information that may be bound to those services.

• Orchestration is independent of the services they are leveraging. Changes can be made to the orchestration without having to force changes to the services. In other words, this architecture is a loosely coupled, services to orchestrations.

• Orchestrations may be decomposable down to base processes, and finally services. The hierarchy further provides architectural control to the architect or the developer in support of the business solution they are automating.

• Orchestration, in context to a SOA, is strategic, leveraging business rules to determine how systems should interact and better leverage the business value from each system through a common abstract business model.

Posted by Dave Linthicum on November 18, 2007 06:29 AM

November 15, 2007 | Comments: (0)

Here is the video of my presentation at the SOA Executive Forum...Enjoy

You can see the video here.

Posted by Dave Linthicum on November 15, 2007 04:45 AM

November 14, 2007 | Comments: (0)

When Not to SOA

One of the things that I say that take many people back is that "SOA is not right for all enterprises." What? How can that be?

Truth-be-told SOA is just a way to do architecture, and while valuable within most problem domains, it indeed may not be right for you. That's why I always suggest that you do a business case before investing in SOA, making sure to justify the systemic change that SOA typically drives, and should. However, the value of SOA is clearly dependent on the characteristics of the business.

While your problem domain/enterprise may have some unique attributes, typically there are some general guidelines when considering SOA. When to use it, and when not.

SOA is probably a fit:

When the enterprise is changing, and there is a high rate of adjusting core business process for new markets, products, acquisitions, etc.
When the existing architecture is heterogeneous and in need of integration.
When the value of change is high.

SOA is probably not a fit:

When the enterprise does not change that much.
When the existing architecture is homogenous.
When the value of change is low.

Pretty logical, but there are a few of you out there that leverage SOA without regard for need. While, SOA may look good on your resume, it may not be good for your company. SOA is a powerful weapon, it can provide huge value, but needs to be directed towards the enterprises where SOA can provide the most bang for the buck.

Posted by Dave Linthicum on November 14, 2007 10:18 AM

November 13, 2007 | Comments: (0)

My Take on the SOA Executive Forum

Download file

Posted by Dave Linthicum on November 13, 2007 04:07 AM

November 10, 2007 | Comments: (0)

More on SOA as a Corporate Responsibility

I had a lot of reaction around my post last week "Why Enterprise Architecture is a Corporate Responsibility." This is also the topic of my ZapFlash I wrote the week before.

Mike Walker liked the post, and had a great follow up post here. I urge you to read it.

"In summary this post does a great job at the overview of the issue. I do think that this is a very serious problem in the industry. There is no easy fix to this. Enterprise Architecture alone will not save the day but does offer a compelling set of methodologies and frameworks to mitigate these risks as much as possible."

He expanded a lot on the notions I'm putting forth, including:

"I have seen a great deal of this in the EA space and is a huge challenge. I hear these challenges from customers and from personal relationships in the industry. The trend here though is that EA is evolving from this Ivory Tower approach where EA's are traffic cops making sure you are building the right systems to a more collaborative and self empowering governance model. The challenge with the political and budgetary is an issue as we move into matrix style organizations where separation of duties force logical separation of roles. As far as EA's are concerned, this is addressed in two ways. First, there needs to be buy-in by CIO and there needs to be "Teeth" in the process. Secondly EA's should have great people skills. I think it is good that EA's do not have a budget it keeps the EA honest. To be clear, this issue is not to be taken lightly as it hinges on your business model, culture and organizational maturity, there is no easy fix here."

However, Dan Foody had a rather grumpy reaction in his recent blog post.

"While I enjoy reading David's writings, there were a few things in this article that made the hair on the back of my neck stand up."

"To me this sounds very self defeatist - it's an excuse someone uses when their SOA strategy fails: 'The man never gave me what I needed to succeed.' The reality is that you don't need budgetary authority and you can create political clout (certainly no one can give you political authority - it's not a transferable commodity)."

Just to set things straight, it's not "defeatist." Were I going there I would say: You're fat, and you'll never get in shape. Instead, I'm saying: You're fat, there is the treadmill, here is how you run, now get to work. There is a huge difference.

The core issue is that EA is and was a neglected area in most enterprises, and needs to be fixed. So, go fix it. And, indeed, this has been a symptom of way too much focus on the short term tactical, and not enough on the long term strategic. I'm not advocating focusing only on the strategic; you have to create a balance. Right now we're making a mess, and are not making plans to clean it up.

Dan continues:

"Just call me the Tony Robbins of SOA. :-)"

No, you didn't just say that!

Posted by Dave Linthicum on November 10, 2007 04:16 PM

November 09, 2007 | Comments: (0)

Observations from the InfoWorld SOA Executive Forum 2007

I'm back from NYC. Took the train back, which is always nicer than waiting for a delayed flight. However, trying to get my truck out of Union Station is always a bit of fun; the gates are so narrow I actually have to fold in my mirrors to get out. Time to get that hybrid, I'm thinking.

My tutorial, despite being held in the middle of the conference, and not at the beginning or the end was well attended. Great group of people, and great questions…the day flew by.

A few observations from the event, at least the sessions I saw:

More case studies around just a bunch of Web services (JBOWS), and not moving towards a true agile architecture state. I'm assuming these are architectures in transition, and not at a final state.
Too much focus on the tactical, and not enough on the strategic. I would like to have seen some more SOA in the context of the business…the ROI.
The analysts are not helping, indeed seem to be confusing things for those looking to move towards SOA. Need to define a process and approaches, not buzzwords. You do it, you don't buy it.
I would like to have seen more candid, real world presentations, such as what to do when things go horrible wrong.
Overall, people seem to be learning and the content was much more sophisticated than before.

Good conference overall. If you did not make this one, make the next one.

Posted by Dave Linthicum on November 9, 2007 09:22 AM

November 07, 2007 | Comments: (0)

Notes for SOA Executive Forum, Day 1

This is the 4^th SOA Executive Forum I've spoke at. It always amazes me how quickly these guys can pull together this event, and still get the attendees. This event is packed, and has some new and interesting content.

The highlight of this morning was Pfizer's Martin Brodbeck, who talked about "Laying the SOA Foundation." He was very data focused, which I think took the audience back. I mean, why talk about data when you can talk about services? The truth is data becomes services, thus the starting point of data, in Pfizer's case a "Master Data Architecture" is a great place to begin. You work up from there to services, data and transactional.

Martin's presentation was great validation for me that indeed SOA can't exist without a clear understanding of the data. Also covered was the importance of SOA governance, inclusive of the ability to track the enterprise IT assets. All of this leads to SOA success.

My panel is at 2:30, and my keynote at 4:00. Looking forward to mixing it up a bit.

Lunch is served…it's a buffet.

Posted by Dave Linthicum on November 7, 2007 09:44 AM

November 06, 2007 | Comments: (0)

SOA as a Corporate Responsibility

Download file

Posted by Dave Linthicum on November 6, 2007 05:09 AM

November 05, 2007 | Comments: (0)

When Data Abstraction Meets Open Source

If there is anybody who is a champion for data abstraction when it comes to SOA, it's me. Moreover, if there is anybody who is a champion for open source SOA technology, it's me. So, how about if there was a company who provided both?

That's the concept behind XAware's new open source offering, which follows a few other SOA open source players, including open source ESBs and application servers. XAware, who has provided a data management and data abstraction technology for years, has recently moved into the open source world with their XAware 5 offering. What's cool about this product is that it lowers the cost of entry into the world of data abstraction for SOA, and puts the end user in direct control. XAware is looking to build a community around this product, as well as a support infrastructure, which is critical to any open source play.

XAware provides data abstraction tool that allows the architect to create a logical database before linking existing physical data stores to it. Thus, this allows the architect to work from the design to the implementation, and provides complete independence from the physical instances of data. This is critical when following my recommendations that first you have a complete semantic understanding and logical structure, before hooking up the physical data sources. These in turn become the data access layers for the services, providing agility at the data layer. If you think that I think that data abstraction is a clear need for most SOAs…you're right.

I think open source, when it comes to SOA, provides two major advantages:

First, it's typically much less expensive than the tools and the technology that are proprietary.

Second, they are typically much more simplistic and easier to understand and use.

To the first point, SOA technology is expensive. I'm talking ESBs that cost as much as Bentleys expensive. Thus, anything that can reduce (or eliminate) costs makes good sense when they are considered with the mix of technology you need to build a SOA. Guys like XAware, for instance, can provide you with key SOA technology at a very low price point. Mule Source is an example of an open source ESB player with a low price point as well, and there are other open source SOA players out there, all offering their wares at a discount.

To the second point, simplicity. The open source SOA vendors seem to take a much more rudimentary approach to SOA, and their tools seem to be much easier to understand and, in some cases, use. While some people want complex, powerful tools, the reality is that most SOAs don't need them. If you're honest with the requirements of the project, you'll see that good enough is, well, good enough. As a result, you end up with less expensive technology that provides only a subset of the features and functions of the larger big stack players. If you don't need them, they only make things more complex, and SOA is complex enough as is.

Posted by Dave Linthicum on November 5, 2007 10:12 PM

November 02, 2007 | Comments: (0)

Why Enterprise Architecture is a Corporate Responsibility

In these days of corporate responsibility and compliance, we need to consider the efficiencies of our IT infrastructure along with other corporate assets. For most companies enterprise architectures are the single most limiting factor to their business economics since they don't have the ability to adapt to business changes in a timely manner.

While many in IT have accepted this as one of life's realities, we could be entering an era where such a reality is unacceptable. Indeed, as innovative corporations learn how to leverage concepts such as SOA to liberate their IT from a static and fragile architecture, less innovative companies could find themselves quickly losing market share to competitors that have embraced the agility value of SOA.

Moreover, the companies who choose to maintain an inefficient architecture could find that their stock price and investor relations have suffered as well, perhaps even becoming targets for shareholder lawsuits as the mood shifts around enterprise architecture as a true corporate responsibility. If investors hold management accountable for accounting irregularities and missed sales, then lost dollars around the ineffectiveness of IT can't be far behind. Thus, the ability to maintain effective and agile enterprise architecture is indeed a corporate responsibility, and those who don't focus on their IT architecture could find that they suffer in the short term, and die in the long-term.

While few will disagree that the inefficiencies of existing enterprise architectures have reached a critical level, many count on "flying under the radar" of those who look at the efficiencies of the company. Let's face it; enterprise architecture is very technical and difficult to understand by the layman. The well publicized corporate scandals focused on shady accounting practices and corporate mismanagement, while IT received a pass in the past years. This will no longer be the case. Today, those who look to value and invest in a company take a critical look at all aspects of the corporation, especially any areas of inefficiencies, including IT. In other words, dysfunctional enterprise architectures could and will devalue the corporation overall, and could put the business at risk.

Enterprise architectures, however, are not fixable by simply bolting on new technology or building connections between systems. Architecture is, well, architecture, and it requires a great deal of planning and analysis to create a strategy that rejuvenates the enterprise architectures into something that lives up to the corporate expectations of business agility and efficiency. Thus, fixing your architecture could take years, and now is the time to begin the steps toward creating an architecture that serves the needs of the business, and not the other way around.

How Have Things Gotten This Bad?

Architectures are like archaeology; in essence, layers upon layers of systems, applications, databases, and connections, typically built or procured to solve a tactical problem. Many corporations talk a good game and brag about the strategic long-term direction of the enterprise architecture that serves the business. The fact is, tactical needs have trumped strategic direction over the years. Thus, layers upon layers of technology on top of technology are the end result, and an architecture that is inflexible, static, fragile, and thus difficult to change along with the business requirements. This is the norm, not the exception.

In reaction to this dilemma, enterprises created positions called enterprise architects. These are single individuals, or groups, within the organization who have the responsibility to drive the enterprise architecture strategy going forward. While a good idea in theory, the reality is that many of these enterprise architects simply don't have the political or budgetary authority within their companies or government agencies to make much of a difference. In many instances, they have been relegated to those who create reports and presentations that nobody reads, and provide direction and guidance that's easily ignored.

Thus, without good architectural governance and ongoing corporate management pressure to redirect resources to tactical IT projects, the enterprise architectures continue to become more unnecessarily complex, static, and fragile. What was a mere annoyance only a few years ago, is today a clearly limiting factor in the businesses' ability to create shareholder value. The company can't easily shift into new and emerging markets, acquire companies, and adjust major business processes without a great deal of latency. In some cases, they are completely unable to change. In other words, things are bad and getting worse.

Posted by Dave Linthicum on November 2, 2007 07:37 AM

November 01, 2007 | Comments: (0)

SOA Performance Modeling

I'm not sure I'm giving birth to a new TLA, figure somebody has used this by now. However, I wanted to touch upon the concept of performance modeling and its importance with SOA.

Performance modeling is nothing new; however its use within the problem domain of SOA is very important. So, how do you go about it?

Performance modeling is creating predictive models that will allow you to determine the performance of your system, in this case SOA, under different loads and configurations. It's in essence a mathematical model, considering real variables, that effect overall performance. Thus, if done right, your model should provide a pretty good prediction of the impact of changes on the architecture as it relates to performance. Let's face it SOAs are always going to change, and performance is always going to be an issue.

While performance modeling was fairly simple in the days of centralized systems, or perhaps systems that were slightly distributed, SOA is a very different animal. Indeed, SOAs are widely distributed, loosely coupled, and deal with different data sizes and data rates, and all of the systems have very different performance characteristics. Thus, SOAs are complex, and difficult to model.

Despite the complexities, it's clear to me that performance models are necessary for SOA. This means that performance should always be a consideration during the design, proof-of-concept, deployment, and operation of an SOA. However, most SOA architects have no idea how to go about doing SOA performance modeling, or distributed computing modeling, with is basically the same thing.

The core concepts are:

Gather heuristics around all systems within the SOA.
Profile communications.
Profile the process layers.
Profile the services layer.
Profile the data layer.
Profile changing behaviors under loads.

Profile all core components.

Once that is done, you construct a distributed computing model that best represents the architecture, test it to determine its accuracy, and then leverage that model as a check point as your SOA changes over time. Pretty simple, but most SOAs don't have one. Do you?

See me live at:

InfoWorld's SOA Executive Forum

Building a Foundation for Continuous Change

November 7-8, 2007

Millennium Hotel, New York City

Posted by Dave Linthicum on November 1, 2007 06:29 PM

REAL WORLD SOA PODCAST

Listen to the latest podcast:

MP3 •