Real World SOA | David Linthicum | September 2006

MORE ENTRIES

September 2006

September 28, 2006 | Comments: (0)

I've been receiving a lot of questions around service design, and I'm finding that those building SOAs are not focusing on this as a discipline.

First, services are not applications. They are small parts of an application. Nor are they subsystems, they are small parts of subsystems as well. Indeed, services are more analogous to traditional application functions in terms of design, and how they are leveraged to form solutions.

Each service has a specific purpose, and they are not complex nor naturally dependent on other services. Thus, they are easily abstracted into composite applications, in essence leveraging these services as if they are functions local to the composite.

Second, services should exist with a high degree of autonomy. They should execute without dependencies, if at all possible. This allows you to leverage the service by itself, and designing the service with this in mind, no matter how course or fine grained the service is.

With the movement toward SOA, and the use of services to assemble and integrate software, we have to pay particular attention to design. Indeed, services have many unique design principles, a bit different than traditional software systems (structured and object-oriented).

So, now that we understand the common design and testing patterns we must follow, the question is: How does one design a service? There are certain steps architects and developers can follow. Here are some suggestions.

First, you need to define the purpose of the service. What will the service do, and who is the intended user?
Possible Artifact: Service Definition Document

Second, you need to determine the information to be bound to the service, including both metadata and schemas. This means you need to understand how information is leveraged by the service, and what functions require what data. Typically, services are storing information outside of themselves, thus you also need to design the mechanisms for accessing outside data sources. Moreover, you need to define data policies here, including data validation constraints and dependencies.

Possible Artifact: Metadata and Schema Document

Third, you need to determine the functions (methods) encapsulated inside the service; in other words, the behaviors you would like to expose. For instance, if our service checks inventory it may expose:

CheckInvenotoryOnHand(product_ID)
CheckInventoryReorderPoint(product_ID)
CheckInventoryOnOrder(product_ID)

It's also at this step where we define each function, including how the function breaks down using a traditional functional decomposition chart. This means that we define the higher level functions by defining all lower level functions. Structure charts or decomposition diagrams are very helpful here.

Possible Artifact: Structure Chart

Fourth, we need to define any interfaces into the service; both machine and human. This means we need to determine how the service will interact with the calling applications, and through what mechanisms. While Web services define the mechanisms for both interface discovery and communications (e.g., WSDL and SOAP), we need to determine what those interfaces are and what they do. In many instances they map directly back to the functions, but not always.

Moreover, with the use of rich client interfaces, you may be interacting with a human through a portal-type interface. You need to define that here as well, including design of the user interface.

Possible Artifact: API Design, User Interface Design

Finally, we need to define how the service is to be tested, using the suggestions above. This is a very important but often neglected step where you define how those leveraging the service will test the service within the context of their usage pattern. You need to define test information, service invocation, and validity of results. Even performance profiling should be included in this step.

Possible Artifact: Test Plan

Posted by Dave Linthicum on September 28, 2006 05:59 AM

September 27, 2006 | Comments: (0)

US Government...SOA to the Rescue?

In this announcement the "Merlin Federal SOA Coalition," a consortium of SOA solution providers (aka, product vendors), announced the results of its "SOA What? - Who and What is Driving SOA Adoption in the Federal Government?" study. The survey was of 196 Information Technology (IT) decision makers in the Federal government.

"Indicators point to the fact that Federal IT professionals overwhelmingly support the SOA concept with 56 percent reporting they believe their agency would benefit from a SOA. Among those who have experienced a SOA implementation, 73 percent would recommend other agencies follow suit and adopt a SOA approach. Forty-nine percent of respondents have heard of SOA. That said, when quizzed, 70 percent of those aware of SOA chose the correct definition and business value proposition. Seventeen percent of respondents note SOA implementation experience at their agency - 11 percent have completed the process while 62 percent of projects are currently in design, planning, and deployment phases. Against this backdrop, only 22 percent note participating in a successful implementation that met the project's identified goals - 78 percent note partial or no success."

First of all, "SOA What?" means it's time to get some new marketing people. I would say: "SOA you're fired." That's almost as bad as "Flying in the face of SOA." But, not as bad as "SOA 2.0."

Second, not sure there is anything interesting here, other than the government IT guys, like the rest of the world, are moving towards SOA.

The fact of the matter is the US government is very interested in any concept that will fix their core IT architectures, which were built over many years of procurement cycles and thus, have many layers of complexity and are about as agile as a pile or rocks. I've seen them first hand in my travels over the last few years, and let me tell you the IT architectures in place are down right scary.

So, SOA to the rescue? Sure, why not? However, there needs to be an understanding that a bunch of products, such as those mentioned in the "SOA Coalition" won't provide as much of a benefit as a detailed set of planning steps to move from a dysfunctional state to something that resembles an efficient architecture. Therefore, the additions of governance systems, BPEL engines, or whatever, will only add to the complexity...if not a part of a larger more strategic SOA journey (notice I did not say project).

I think that the government can benefit most from SOA considering the nature of their business and the underlying need to have many systems interoperate. However, there needs to be a realistic understanding of the issues at hand, and perhaps they need some new approaches other than building architectures that look like archeological layers of past IT contracts. I do think some of the more spectacular SOA successes will come from the government side.

Posted by Dave Linthicum on September 27, 2006 04:56 AM

September 26, 2006 | Comments: (0)

BPEL Issues and Step 10 of my 12 Steps to SOA

Download file

Posted by Dave Linthicum on September 26, 2006 03:48 AM

September 23, 2006 | Comments: (0)

More on the "BPEL Thing"

Fred Holahan, from Active Endpoints, Inc. did a good job in taking me to task in response to my BPEL post last week. I can understand his frustration and defensiveness giving that BPEL is core to their technology, and the fact that BPEL has been getting hammered recently in the press and blogashere. However, I'm really discussing the quality of the core standard here and not a product instance.

What's more, this is "Real World SOA." It's very important to talk about the issues around technology, approaches, and standards, as well as the benefits, else you're just drinking the SOA hype Kool-Aid. Truth-be-told, standards have good points and bad points, and discussing both does not mean that you're a "naysayer," only that you're seeking a clear view of the reality.

What's most frustrating about the issues here is that orchestration is indeed a core feature of SOA...the configuration component that makes orchestration that part of the architecture providing agility. Quoting myself:

"Orchestration, at least the notion, is a necessity if you are building a SOA. It's the layer that creates business solutions from the vast array of services and information flows found in new and existing systems. Orchestration is a godlike control mechanism that's able to put our SOA to work, as well as provide a point of control. Orchestration layers allow you to change the way your business functions, as needed, to define or redefine any business process on-the-fly. This provides the business with the flexibility and agility needed, and the core value of SOA."

The notion that portability or interoperability was never a promise of BPEL 1.1 does not jive with what has been said and written by the BPEL vendors, analysts, and bloggers. Here are a few examples:

"...therefore the promise of BPEL portability has not yet arrived,..."

"The key goal of BPEL is portability."

"The Business Process Execution Language for Web Services (BPEL4WS or BPEL for short) is an XML-based language for defining business processes that provides an interoperable, portable language for both abstract and executable processes..."

You get the idea, and thus the selling point behind most standards...that your IP will be protected because it's transportable to other technologies (albeit they almost never are).

Moreover, orchestration, and BPEL attempting to provide an enabling standard for orchestration are much more complex and far reaching than database query standards or data interchange standards, that's the problem here. In essence you're defining a standards layer as a "platform" for meta-applications existing above native processes and services. With BPEL 1.1 many of those capabilities are missing, that is why many vendors added them on as proprietary extensions...the issue at hand.

So, you put your hopes on the next release. However, Wikipedia:

"This change in name and version number reflects the significant and in many cases incompatible differences between BPEL4WS 1.1 and WS-BPEL 2.0."

Or, David Ogren:

"On the other side, the vendors know that there will be a huge chasm to cross between BPEL4WS 1.1 and WS-BPEL 2.0, and that even WS-BPEL 2.0 is only a tiny subset of what a BPM vendor needs to include in their product."

Enough said, and there is much work to be done.

Posted by Dave Linthicum on September 23, 2006 06:49 AM

September 21, 2006 | Comments: (0)

SOA as an Investment

I've been receiving a lot of e-mails this week. Not from technology guys looking at SOA but from investors looking to invest in companies that have SOA technology and services. Indeed, the stock market has been raging lately, and while I don’t think we’re filling up a bubble with hot air again, there is much money to be made in technology over the next few years.

You know you're in a cool space when Motley Fool talks about you, and that's what occurred in this article "SOA: The New Software Bazaar" explaining both the benefits of SOA and how to invest in this technology.

"This is the SOA promise -- and CIOs believe. Merrill Lynch surveyed CIOs and found 87% agreed SOA is the "next big thing" in enterprise software. So, what benefits are they buying?
Technology benefits: reuse of existing IT assets; quicker development of new software; simplified, integrated, and standardized IT portfolios; and leaner technology departments, organized around processes, not packages.

Business benefits: create more flexible businesses, supported by streamlined, automated business processes and software services.

Financial benefits: quicker responses to market changes boost revenue; reuse lowers maintenance and development costs; requires lower total capital invested in IT; and risks reduced by not building software from scratch."

Clearly, SOA is getting some play on Wall Street, and for a good reason. It's a legitimate problem; the existing bad architectures within most enterprises are limiting the capabilities of business. SOA...the notion, the approaches, and the technology has the potential to solve this problem. Thus, it makes sense to invest in companies that provide either SOA services or technology.

I've found that the best SOA technology is found inside of smaller private companies, and unless you have the major duckets to be an angel investor, or dump some bread in the larger VC funds, you won't be able to cash in on this investment trend, at least as a pure play. However, with the trend toward acquisitions, this means that many of the larger public companies are adding these companies to their portfolios...Oracle, BEA, and IBM, just to name a few.

SOA is a sound investment opportunity, but you need to think long term and focus on the business value of fixing the dysfunctional enterprise architectures you find today. There are more out there than you think, and those are examples of bad investments.

Posted by Dave Linthicum on September 21, 2006 05:47 AM

September 20, 2006 | Comments: (0)

WS-BPEL 2.0 Not Backward Compatible?

Let's face it, BPEL 1.1 was a bad standard that left so much out that many end users and vendors found it useless. In response, the vendors put a ton of proprietary extensions in their BPEL 1.1 - based products, thus diluting its value to the point of "why bother."

While I've been ranting and raving about this for some time, Dave Chappell does a much better job in explaining the limitations.

"Promoting BPEL's portability helps significantly in the first of these goals, since customers like the idea of not being locked in to a single vendor. But actually making customers successful has typically required extending BPEL in proprietary ways, which works against the language's promised portability. While BPEL purists might argue that all of these extensions should be provided via programming language-neutral web services interfaces, this isn't what’s actually happening in the products."

With huge investment in BPEL by the larger SOA players out there the dirty little secret is that BPEL 1.1 never really worked as advertised, and the amount of custom and proprietary extensions required making the technology useful meant that the money, time, and effort was pretty much wasted if standards and portability was the goal.

Enter WS-BPEL 2.0, and another opportunity for vendors to get BPEL right. You can find the draft specification here, which I read, and discovered that this spec was much improved, but with many issues that remain. For instance, WS-BPEL 2.0 has considerable differences to its previous 1.1 version. The major differences include syntax changes to the language, inclusion of new features including parallel for-each, and modifications to the semantic of existing constructions, such as compensation handling.

Hmmmm. So, what if I already implemented orchestrations using BPEL 1.1, what do I do now? In short, you purchased Beta and the world is moving to VHS (yes, I'm old). I thought that this article by BEA's Alexandre Alves did a good job in summing things up.

"For all but the simplest business processes, migration from BPEL 1.1 to BPEL 2.0 is not an easy task. Some of the syntactic changes can be automated as shown [in this article], however the semantic differences, especially when dealing with links, messaging, compensation handling, and data manipulation, will demand a comprehensive and time-consuming process."

I suspect the BPEL sales guys will get some angry calls around Christmas time from their users, perhaps more so considering the use of many proprietary extensions to get around the limitations of BPEL 1.1. Not a good way to start a standard, I guess that is why my old boss told me never buy products until release 3.0.

Posted by Dave Linthicum on September 20, 2006 06:57 AM

September 19, 2006 | Comments: (0)

WebMethods Gets Infravio and Step 9 of my 12 Steps to SOA

Download file

Posted by Dave Linthicum on September 19, 2006 05:54 AM

September 18, 2006 | Comments: (0)

Ubiquitous Computing and SOA

I had a birthday recently, and I received a pair of these O ROKR Bluetooth sunglasses. While these are not new, this is my first pair and I'm finding that it is way cool to listen to my iPod then have the glasses switch automatically to a phone call coming into my Blackberry, at the same time looking stylish.

While a bit of a leap, this has me thinking more about wearable computing. I mean, we have the technology to place a small screen on my glasses as well, perhaps put a keyboard on my arm and all connected through a satellite or cellular network. There are applications for this kind of technology today, including outfitting war fighters with complete information systems strapped to their bodies for use in combat. I mean, it can't be much more expensive than the Diesel Jeans I just purchased. Also, you got to love a technology where you can check your stocks, e-mail, update your blog, and fire a round at somebody, all while running from building to building in some far off land.

At the essence of this, is the notion of ubiquitous computing. Or, as Wikipedia defines it:

"Ubiquitous computing (ubicomp) integrates computation into the environment, rather than having computers which are distinct objects. Other terms for ubiquitous computing include pervasive computing, calm technology, things that think and everyware. Promoters of this idea hope that embedding computation into the environment and everyday objects would enable people to interact with information-processing devices more naturally and casually than they currently do, and in whatever location or circumstance they find themselves."

You have to admit this day is coming quickly, if not already upon us, including cell phones that are now really small PCs, Microwaves that can set their own time and calculate the cooking time for 3 potatoes, and information systems in the cars we drive that not only tell us when the oil needs changing, but schedules the appointment at the dealer, and calls somebody, providing a location, when your airbag deploys to remove you from the wreckage.

Considering the notion of ubiquitous computing, the larger question is the paradigm for communications in and between all of these different devices that no longer look like traditional computers. I'm asserting that the basic principles behind SOA are the most applicable here. If you consider all of these new devices as a collection of services and abstracted data, then the level of service use between them will be the next logical step.

Considering this architecture, we simple deal with each device as a system until itself with a physical data structure, abstracted data, services, composite services, and perhaps some processes as well. From there the services existing in the device are available to other devices, computer systems, or most importantly to an orchestration or process layer where the interaction with all of these devices and computer systems can be defined in terms of business solutions.

Thus, an inventory system showing the lack of a specific product can reach out and invoke a service on the logistics system to figure out how to get that product back in the warehouse. Then, the logistics system can schedule a pick up by invoking services inside the computer systems of the company's fleet of trucks. In turn, you can track the progress of the pickup, and estimate to the customers when the product will be back in stock, and thus when you can ship, bill, and when the money will reportable as income to the accounting system. You get the idea, including all computing power in your SOA to make your architecture that much more valuable.

This not science fiction, this is doable today. Companies just need to step up and learn how to leverage all of the computing power that is already at their finger tips, traditional and non-traditional.

Posted by Dave Linthicum on September 18, 2006 07:03 AM

September 14, 2006 | Comments: (0)

Inflated Expectations Cause the "Dark Side of SOA"

I happened to catch this article on the DDJ site which does a good job analyzing the reaction to the over hyping of SOA, and thus the inflated expectations.

"According to an InformationWeek Research Web survey of 273 business-tech pros last month, 24% of respondents using SOA and Web services say the projects fell short of expectations. Of those, 55% say SOA projects introduced more complexity into their IT environments, and 41% say they cost more than expected. Out of all respondents using SOAs and Web services, just 7% say the results exceeded expectations."

Why is this happening? Well, it's a matter of promising X and delivering Y. Personally, I put the blame on the SOA vendors for pushing SOA technology as a cure-all for the years and years of bad architecture and lack of integration. The reality is that SOA is hard, complex, takes time, and no number of layers of technology will fix bad architecture until you understand your own business issues, and align IT as the solution, and have a realistic process for getting their.

However, the downsides of SOA, does not mean there are not huge upsides to consider.

"Still, nearly half (45%) of survey respondents rate SOA projects as very important to their companies' business goals. The most common reasons for adopting the technology include standardization, automating business processes, cutting operating costs, and increasing customer satisfaction. Among companies using SOA, 72% say they hope to achieve increased application development flexibility, 61% list the ability to more quickly create apps that use Web services as important, and 58% say they want more modular software. But just 26% of respondents say they're pursuing SOA very aggressively."

Thus, most enterprises continue to move forward with their projects, despite the dark sides of SOA, perhaps reevaluating their expectations around what real benefits they can obtain and how long it will take them to get there.

Some of my advice would include:

1. Understand that the upfront analysis work will have to occur before you select your technology.
2. Make sure to employ a proof-of-concept early on to determine what the technology will really do, and its value within your problem domain.
3. Make sure you have the right people for the job.
4. Don't be afraid to change course in the middle of the project if needed.

Posted by Dave Linthicum on September 14, 2006 05:17 AM

September 13, 2006 | Comments: (0)

WebMethods Gets Infravio...More Consolidation

Speaking of governance, WebMethods announced the acquisition of governance vendor Infravio for $38 mil. The sale of Infravio follows the acquisition of other several deals. Last month...BEA Systems acquired Flashline. Earlier this year, Mercury Interactive bought Infravio competitor Systinet.

Both companies have good products; indeed WebMethods has one of the best traditional EAI products that are looking to become "SOAtized" and modernized. However, sales recently have been lackluster based on them missing a quarter. But, a good company is a good company; they typically recover from stuff like that...speaking from experience.

WebMethods has been on the acquisition track for some time, starting with the acquisition of Active Software many years ago, and has done many more between Active and Infravio. I’m sure some technologies adding value, some not as much. So, what about this deal?

I suspect that the driving force behind this acquisition is the fact that the other major integration and SOA players are adding governance to their respective stacks, either building or buying. What is more, many in the market are asking or governance solutions of part of a complete SOA product offering, and this is their response to both protect their market position and generate new name sales.

The challenge will be the integration of all their technologies to form a complete solution that will be acceptable in the marketplace, and with components the market is looking for. The risk is that the components won't be best-of-breed, and thus considering the all-or-nothing approach, a few bad subsystems will lower the value of the better SOA components. Governance, for instance, is a new concept (with old approaches, really), and changing approaches, enabling technology, and standards could mean that while you’ve built and amazing sports car, the lack of a modern drive train limits the capabilities of the complete car. Avoiding this takes a lot of planning, and some good luck. That will be the challenge at WebMethods, I suspect.

Posted by Dave Linthicum on September 13, 2006 05:01 AM

September 12, 2006 | Comments: (0)

Do We Need SOA Governance Reform? (Part II)

So, what’s missing, and required by those building SOAs. My short list is:

The big one...focus on architecture, which brings agility, and less on reuse. Agility trumps reuse every time, and we've beaten that to death here, and on my Podcast. This means focus more on the process abstractions first, and only then the notions of policy management, metadata management, and dependencies. At the end of the day, governance tools should bring some new concepts to the SOA party, as of now they don’t seem to bring much at all other than management.

Focus on the ability to leverage "stranger services." While the name may induce a chuckle, it is the best term to describe dealing with services that you have not created nor do you host. The fact of the matter is that in a few years, services created and hosted outside of our firewalls may be a larger part of our SOAs than we think, thus we need the ability the management and leverage those services.

Focus on standards, and I mean really get it done in a short period of time. Come to an agreement on standard(s) and change your products to support those standard(s) in 6 months, else don't bother. I mean how many times are we going to through the cycle of the joint press release, 6 months passes, the draft, 6 months passes, the revisions, 6 months passes, the infighting, 6 months passes, and nobody cares any more?

Vote yes, on SOA governance Reform in November, or better yet call up your governance vendor and tell him or her that you're not going to take it anymore...it's architecture that drives agility first, not reuse that drives development. Write that on your hand.

Posted by Dave Linthicum on September 12, 2006 06:25 AM

September 12, 2006 | Comments: (0)

The Limitations of SOA Governance and Step 8 to my 12 Steps to SOA

Download file

Posted by Dave Linthicum on September 12, 2006 04:25 AM

September 11, 2006 | Comments: (0)

Do We Need SOA Governance Reform? (Part I)

In working up another article this weekend I was going through the Web sites of the larger SOA governance guys to see what they have been up to, also to better define this notion of "SOA governance." In this process, a few things came to me:

First, everyone's definition of SOA governance is a bit different. Thus, what they offer fits their definitions. However, most are directories, repositories, or registries, at their essence, and may or may not include policy management.

Second, there is no clear architectural use case for most of these tools. In attempting to figure out how these tools fit into the "SOA problem" is difficult, when considering that we’re dealing with architecture, and not just a collection of applications and services. Best, these are application development support tools, akin to configuration management and metadata management of years gone by.

Finally, it appears to me that for these tools to be truly valuable to those building a SOA, they need to change their approaches...or SOA governance reform.

So, if I'm going to boil all of this down to common patterns, I would say that: SOA governance tools are really repositories that store information on what software components have been written and whether there are dependencies among different programs. In addition, they collect policy information, such as security access rules, and connects to project management programs. SOA governance tools are meant to improve oversight of ongoing application development, a design approach that advocates modular, reusable programs, and not overall architecture.

More later...

Posted by Dave Linthicum on September 11, 2006 06:23 AM

September 06, 2006 | Comments: (0)

SOA and Reuse and Step 7 of my 12 Steps to SOA

Download file

Posted by Dave Linthicum on September 6, 2006 06:20 AM

September 05, 2006 | Comments: (0)

Can the Old EAI Dogs Learn New SOA Tricks?

Many of the "traditional" integration/EAI guys have been contacting me recently looking for some direction. It's typically sales and marketing folk, who are tasked with selling the stuff, and are having trouble these days. As it seems, while SOA is taking off, many of the companies who defined the EAI/integration space (well, at least product-wise) in the late 90s are finding that users are passing up their products for SOA up-starts.

I pointed this out before, there are really two camps: SOA Old School and SOA New School, and there seems to be a chasm between the two that keeps growing.

"Old school SOA vendors are those with 'legacy' integration or application development solutions that 'SOA-tized' their stuff, basically adding Web services interfaces, enhanced security, etc.."

In fact many are finding that the "traditional technology" is coming up short when you consider the new complexities of SOA. I think that SOA is being defined by the marketing machines in the well funded upstarts, and the older guys are finding that they are being made obsolete. No matter if it's perception, or reality. In the world of SOA, it really does not seem to matter right now. They hype cycle is raging.

However, there are product pattern differences that are causing issues as well. Here are a few that I've noted, generally speaking:

- Older products are more information-oriented and don't consider the notion of a service within the product at the management, development, and repository levels...it seems to be an afterthought.
- The process or orchestration technologies delivered with the older integration technologies are not innate, but seem to be loosely coupled. Perhaps in many cases "architecture through acquisition."
- They are "enterprisy" and don’t consider services outside of the firewall when building a SOA, nor have facilities to leverage mashups.
- They are leveraging out-of-date standards that are just no longer interesting, nor work well in the context of a SOA. I already pointed out the issues around J2EE in my last blog.
- The existing customer use cases are more about information replication adhering to a loosely defined process, and not about architecture agility or the ability to drive integration around service abstraction.

So, is this a trend? Are the older integration guys with old products, thousands of customers, and yes, gulp, profitable, able to keep up with the young Turks who seem to be defining the market now? Tough to say, but it does not look good when considering the current set of events, including a growing market and declining or slow growing sales with a few of the players.

Clearly, their stuff works and is proven. They just need some strategy alignment and product changes. Been there, done that, have the tee-shirt. It's not easy.

Posted by Dave Linthicum on September 5, 2006 05:08 AM

REAL WORLD SOA PODCAST

Listen to the latest podcast:

MP3 •

•

• Archive •

Technology White Papers

InfoWorld Technology Marketplace

» Technology White Papers Library

Technology White Papers by Topic

Technology White Papers E-mail Alert