Real World SOA | David Linthicum | October 2006

MORE ENTRIES

October 2006

October 31, 2006 | Comments: (0)

Traditional EA and SOA & Governance Issues

Download file

Posted by Dave Linthicum on October 31, 2006 06:02 PM

October 31, 2006 | Comments: (0)

Is it time to have a TOGA[F] party?

In this release.

"The Open Group, a vendor- and technology-neutral consortium focused on open standards and global interoperability within and between enterprises, today announced the creation of The SOA/TOGAF(TM) Practical Guide Project."

This is a collaborative effort by members of The Open Group who recently formed SOA Working Group and The Open Group's Architecture Forum. The new project was created to provide guidance for enterprise architects on the use of The Open Group Architecture Framework (TOGAF) in developing a Service-Oriented Architecture (SOA).

"A primary outcome of the project will be a practical guidebook for performing SOA with TOGAF. The practical guide, which is scheduled for release in early 2007, will leverage the industry's leading research and perspectives on SOA preparation and implementation for architecture practitioners, business executives and IT staff alike. Additionally, the SOA/TOGAF Practical Guide Project will be led by some of the industry's most innovative and knowledgeable enterprise architects, taking advantage of the combined contributions made by The Open Group's SOA Working Group and Architecture Forum."

It get it, they are thinking SOA now. Good for them. Indeed any thoughts around procedures, approaches, and methods to build SOAs are an improvement upon the vendor and hype driven mania that I see out there now. While I don't know the details yet I'm sure the Open Group has something practical and reasonable.

What you have to remember about things like this is that the real value is the discipline that they outline, but really don't provide. You, the architect, must learn how to leverage this approach, or others, for the problems you're looking to solve. If you don't do that, all the approaches, frameworks, and methodologies won't help you.

Posted by Dave Linthicum on October 31, 2006 06:21 AM

October 27, 2006 | Comments: (0)

InfoWorld SOA Executive Forum...Be there or your SOAs will fail.

Okay, perhaps not fail, but you won't know as much.

Don't forget I'll be speaking at the SOA Executive Forum in New York on 11/8. You can register for this event here, as well as find out more about the agenda, the event, and the venue.

I speak at most SOA conferences, but this is the best event to attend. At the very least, you should go to meet me. :-)

Have a great weekend.

Posted by Dave Linthicum on October 27, 2006 12:48 PM

October 27, 2006 | Comments: (0)

Agility vs. Reuse...Debate? Listen in now.

I did participate in that agility vs. reuse debate last Friday with eBizq.net. Elizabeth Book was the host, with me, Ronan Bradley, Neil Ward-Dutton, and Joe McKendrick, all SOA bloggers. You can download the Podcast here.

The Podcast was looking to shed more light on the value of reuse in the context of SOA. I'm not sure it was much of a debate, in fact plus or minus 10 percent I think that Ronan, Neil, Joe, and myself agreed that agility was the golden goose of SOA, with reuse a linked benefit.

I thought that Joe's blog did a good job in defining the Podcast.

"Dave Linthicum agreed that reuse was not the endgame of SOA, but rather a means to take 'architectures are completely unworkable messes for the most part' and 'turn them into something that's better aligned with business, that's changeable as the business needs change, and something that's workable and cost effective in the organization. ...Reuse is going to be a side benefit of that.'"

Remember, the value of SOA is the ability to get your enterprise in an agile changeable state, and thus better able to align with business. Most IT departments can't do this today, and that's limiting their value to the business. That's how this SOA stuff, which is very expensive, makes you money. Reuse is possible, also valuable, and will occur if you do SOA right. Keep that in mind.

Posted by Dave Linthicum on October 27, 2006 05:42 AM

October 27, 2006 | Comments: (0)

Are you getting a D in Governance?

In this article it was revealed that:

"Progress Software (NASDAQ: PRGS) has announced the results of a SOA Governance survey that gathered feedback from over 300 IT respondents in 21 industries, including financial services, banking, government, insurance, healthcare and pharmaceutical.

The survey revealed that governance is not keeping pace with the adoption of service-oriented architectures (SOAs) at most organizations."

While this survey is clearly self serving in terms of adding value to a particular vendor it is important to note that the notion of governance has not made a big impact within most organizations building SOAs. No news there.

Why? Well, most organizations are just starting down the SOA road and heavy and expensive notions and tools such as governance are not yet on the to-do list as they muddle through the first instances of a SOA. However, governance is an important concept and I'm sure there will be a natural migration towards governance as SOAs become more complex and far reaching.

But, as I pointed out before the governance tools I've seen still have more baking to do. Perhaps that's the essence of the issue with adoption at this point, not the end user organizations majority. Indeed, perhaps the vendors get a D and the users get an Incomplete.

Posted by Dave Linthicum on October 27, 2006 05:24 AM

October 25, 2006 | Comments: (0)

Notes for the "Enterprise Architecture Conference"

Just heading home for the EAC in San Diego. The conference was very well attended, and I enjoyed interacting with everyone...talking about SOA for the most part. Just a few things to note:

First off, there is still a huge divide between the "traditional EA guys" and the "SOA guys." I was joking during my talk that service-oriented architects are enterprise architects with earrings. Perhaps that's more true that humorous.

The fact is I heard little about SOA during the entire conference. Even the EA magazines and vendors did not mention SOA, and I'm not sure the attendees understand the synergies between the two disciplines. In fact, I think they are one in the same; SOA at its essence is "good enterprise architecture."

Second, while I've been living in the world of traditional enterprise architecture for some time, the notion of EA seems to have morphed into abstract management concepts, and not as much about understanding, problem solving, and technology.

The bottom line is that SOA needs EA, and EA needs SOA, and the conversations between the two camp don't seem to be occurring. Different conferences, different people, different approaches, different tools,…same problem looking to be solved. What gives guys?

Posted by Dave Linthicum on October 25, 2006 07:01 AM

October 24, 2006 | Comments: (0)

SOA and Application Semantics

Download file

Posted by Dave Linthicum on October 24, 2006 12:20 AM

October 22, 2006 | Comments: (0)

Step X:Understand all Application Semantics in Your Domain

In working on Steps I've been getting through the application semantics analysis phase and came up with the following meta-process:

Understand all Application Semantics in Your Domain.jpg

In essence we are doing a few primary sub-steps:

First, perform metadata analysis from existing legacy systems (stuff we have now), and systems existing within other enterprises (partners), but still make up our data domain.

Second, define the data abstraction layer. There are many many sub-steps under this one, but the idea is to create another metadata layer above the physical for use within the SOA.

Finally, define the data services to support the data abstraction layer.

Posted by Dave Linthicum on October 22, 2006 02:07 PM

October 19, 2006 | Comments: (0)

Considering Semantics

Okay, here we go on the initial steps.

The management of semantics has always been a key consideration in the world of application integration and now SOA. Indeed, we need to manage the meaning of information between systems, accounting for the differences for use in transformation layers and within services, as well as managing common notions and concepts that map back to physical records, and in some cases instances of information. To date, however, we've done a poor job in managing semantics lacking the understanding, the discipline, as well as the enabling technology and standards. Perhaps it's time we finally get a handle on it as we move to SOA.

A semantic mapping/management layer, when combined with a competent SOA tool, allows organizations, trading communities, vertical industries and even micro economies to create something I call "semantic domains." These domains are really a repository of concepts, with each concept having properties and attributes that map back to physical schemas that exist in source or target systems.

For instance, a hospital has a physical record that describes a procedure known as procedure_comp, this may map to a concept known as procedure in the semantic domain known as HC_Vertical_Semantics. Or:

Procedure_comp->procedure->HC_Vertical_Semantics

Other health care providers may in turn map their physical record to that concept, assuring that all organizations leveraging that domain are leveraging the same semantic understanding of the data. For instance:

Prod_1->procedure->HC_Vertical_Semantics

Note that the 'procedure' concept many have hundreds of properties that defines it, and some of these properties will map back to the physical record. Also, note that it is possible for 'procedure' to belong to other domains as well, inheriting the properties and attributes from the concept or concepts. Moreover, these domains can be either public or private, and are sharable among any semantic domain user on the network (if they have permission).

In my view of the world there are three types of semantic domains in the world of SOA: customer defined, community, and vertical.

The customer defined domains are a collection of semantic definitions and mappings for a particular customer for use within a single enterprise.

Community domains share semantic definitions and mappings supporting a closed community of users, such as a trading community around a single automobile manufacturer.

Vertical domains are definitions and mappings supporting whole industries.

Each domain should support the notion of abstraction, making it possible for domains to become sub-domains and leverage semantic definitions, concepts and mappings already created, through an abstraction mechanism exposed as services.

For instance, if we're creating a trading community for an automobile manufacturer we would begin with the public semantic domain for that vertical, abstract the semantics, and then customize the semantics as needed in another instance of a semantic domain that links back to the concepts and properties of the base domain.

What do you think?

Posted by Dave Linthicum on October 19, 2006 09:36 AM

October 18, 2006 | Comments: (0)

Trials and Tribulations of Creating a SOA Methodology

I've been working on a new version of the 12 Steps, now just called SOA Steps. In my career I've written many methodologies, including structured application development, component development, object-oriented development, and now the larger, more holistic, but much more valuable SOA. I continue to learn, and refine my approach.

Methodologies are double-edge swords. When I was working for larger consulting firms in my youth they were really too cumbersome, designed primarily to turn the crank of billable hours more so then creating good information systems. However, not using a methodology could mean you're missing some very important steps. Thus, you need to strike a balance between too much rigor and not enough. That is what I'm attempting to do this time.

A few things to note at this point:

The core issues continue to be the ability to have a semantic level, service level, and process level understanding of the problem domain. No getting around that and that's the core work to be done when defining and designing a SOA.

The data resides in a data services layer which is really an abstract data layer existing above the physical databases. That layer is design to serve the services layers providing information bound to behavior. The abstract data layer is really just a re-instantiation of the physical schemas and data model into something that's more useful for the SOA, and the services that make up the SOA.

Process is still defined as common notion, and not defined as orchestrations as of yet. Albeit you can define your processes as orchestrations it's a bit early in the game to figure out if the process layer will be a process integration tool or an orchestration tool in its end state. Perhaps it should always be a common notion.

That's all for now.

Posted by Dave Linthicum on October 18, 2006 04:17 AM

October 16, 2006 | Comments: (0)

Agility vs. Reuse

Download file

Posted by Dave Linthicum on October 16, 2006 07:44 PM

October 15, 2006 | Comments: (0)

Will SOA Cause Developer Unemployment?

There is a lot of talk about how SOA will significantly lower the need for developers; thus the savings of SOA. This will be accomplished through the promise of reuse that's driving many toward the SOA light. However, I'm not sure we'll see a reduction in development with the advent of SOA, but perhaps a redistribution of talent in the longer term. At the end of the day, the reason for leveraging SOA is agility. Reuse and development savings are a secondary benefit, if they happen at all.

Truth-be-told, we've been considering the demise of the developer during many "hype phases" over the last 15 years. This included the "component development" phase where I heard not one, but three software executives, in keynote speeches, talk about how "applications would be assembled like Ford assembles cars, from pre-built component parts," thus, the need for fewer developers. Same goes for the distributed object phase, the intranet phase, and now here we are in the SOA phase. The issues are exactly the same, with perhaps the technology being a bit more compelling

SOA has three realities to consider:

First, it's something that really has not happened yet; people are talking about it, and in some instances, playing around with it, but true killer SOAs are few and far between right now. This is due to the fact that it’s complex, a huge change in thinking, and those things take years to role out in most enterprises. It's more people issues than technology, by the way. Thus, it's too soon to understand what real savings will be realized from the use of SOAs. Or, in other words, we're a bit early to think about how many developers we can fire.

Second, if history is a teacher, we'll find that we actually need more developers--at least at first--with the promise of savings through reuse in the future. However, we've yet to get reuse right with all of the past opportunities such as object-oriented development, distributed objects, and component-based programming, so we're assuming we'll get it right with this technology, standards, and approaches. I'm optimistic, but I'm also a realist here, understanding that true adoption runs about two years behind the hype.

Finally, the use of services over the Internet will create a new generation of developers who build services for applications they'll never see. They build portions of applications for use in many applications as services, typically delivered over the Web, and that industry will be huge. All you need to do is to look at the growth of the major service providers out there and the emerging Web services marketplaces. So, you guys who get fired by the enterprises will have better jobs in these emerging areas.

Posted by Dave Linthicum on October 15, 2006 06:03 PM

October 13, 2006 | Comments: (0)

SOA in Paris

Sorry to be short on blogging this week, I've been in Paris most of the week speaking at a conference sponsored by Oresys. A few observations from our friends in Europe:

- They call enterprise architecture "city planning." Fitting, I think.

- They take a much more practical approach to SOA, and don't go with the hype. They have to justify costs, good lesson for many companies back here in the states.

- The approaches to enterprise architecture are much more formal and methodical.

Other observations:

- There is not a damm Starbuck’s in the city, plenty of Café's however. You have to sit down and drink from a ceramic cup…my god.

- Wine costs $3 a glass, Coke is $3.50.

- Nobody was rude to me, and I speak no French and wear loud clothing with an iPod blaring.

- The eggs and bacon are not the same eggs and bacon we eat in the states. Can't tell you why, they are just different.

- They ambient temperature in most offices and hotels is about 8 degrees higher than we like it in the US, and everyone still wears sweaters.

All and all a good week to share SOA information with new friends in Europe. I hope to get invited back soon, perhaps there will be a Starbuck's in Paris by then, but I'm sure they will serve wine like McDonald's does in Paris.

Posted by Dave Linthicum on October 13, 2006 02:27 PM

October 10, 2006 | Comments: (0)

Is SOA Understood?

Download file

Posted by Dave Linthicum on October 10, 2006 12:55 PM

October 09, 2006 | Comments: (0)

Patterns for Good SOA Vendors and Technology

As the SOA biz begins to become more mature, I'm seeing certain patterns emerging that are separating the good versus the bad vendors, and their technology. These are patterns that you should look for as a user of SOA technology, or if you're a vendor this is some good self analysis. Here are just 3, I have many more.

Pattern 1: The ability to be self critical.

Denial isn't just a river in Egypt, and I've seen SOA technology providers, time and time again, ignore the elephants in the room. These issues can range for failure of standards to not understanding the essence of SOA as a concept, but not addressing these issues quickly can lead to the failure of the technology, and thus the company itself. Typically, the causes here are culture as well as market forces. You know...that guy who never wants to hear bad news.

To avoid this pattern promote a culture of self criticism, and make sure the issues are understood and corrected before the market or the customer gets wind of them. Seems easy, but it's a systemic problem within many SOA technology companies today.

Pattern 2: Promote a culture of quality.

In my past lives at CTO, quality was always in the top of my list, even though many consider testing boring and a drain on resources better spent on building new functionality. The reality is that SOA technology, by its very nature is complex, and testing this kind of technology is problematic since you have to consider so many domains and usage scenarios. However, not testing means that critical information will be lost by the end user, or the stuff just won’t work as advertised.

To promote a culture of quality, quality must be an issue from the top down, and part of the overall culture of the company. This includes incentives, and allowing anyone in the organization to bring up quality issues at any time, even if it means delaying delivery.

Pattern 3: Be honest about the capabilities of the technology.

At this stage of the game SOA is not well understood by those selling it and most have never seen a problem domain that they don't like, and thus lead with the technology, not understanding the requirements until later in the sales process. This is a huge mistake.

The reality is that all SOA problem domains are different, and SOA technology is very different as well (governance, integration, orchestration, identity management, etc.). So, you must first understand the problem patterns before defining solution patterns, and thus technology to leverage. While your product may be a fit in some instances, in many instances it won't work. You need to understand when that fit is not there and learn to walk away, and spend your sales energy with customers who will find the technology valuable.

Let me know if you want more.

Posted by Dave Linthicum on October 9, 2006 07:10 AM

October 06, 2006 | Comments: (0)

Benefits of SOA not widely Understood

In this article by Clive Longbottom , Head of Research Quocirca, published this week. He looks at the amount of knowledge transfer that's occurred to the rank-and-file IT guys pertaining to the benefits and the notion of SOA.

"Service-oriented architectures (SOAs) are the subject de jour with IT vendors, who have been using the term as if the concept has been totally understood by the buying audience and is well along the way to general implementation.

However, research carried out by Quocirca on behalf of Oracle earlier this year shows a rather different picture. From a sample size of 1,500 respondents representing a mixture of technical and business people, more than 30 per cent said they have absolutely no knowledge of what SOA or service-oriented architecture means. More than 25 per cent more said they have minimal knowledge of it and only 20 per cent stated they have a fair understanding or a good working knowledge of what SOA is all about."

What was more disturbing was the response from the business players, typically the stakeholders.

"This split is even more pronounced when we just look at the business respondents—55 per cent said they have absolutely no idea what SOA is about and only 10 per cent said they have sufficient knowledge to understand what impact SOA would have on their business."

The article goes on to look at the responsibility for the lack of knowledge. They noted that the research shows the majority of people feel information coming from vendors is okay and that the respondents are happy with the information. However, the proper expectations don't seem to be widely understood.

"But if the basic understanding of SOA is so poor, is it that this 'consistent' information is just consistently misleading? Is it down to the analyst community overselling SOA as a concept and focusing far more on the technological impact, rather than the business impact? Is it the media, looking only for the day's headline, and neglecting the actual issues?"

While the author goes on to some obvious conclusions, all which jive with my experiences, it's clear that SOA, as a notion, has been hijacked by the vendors, and the vendors are doing a poor job in explaining the downsides as well as the upsides. In essence they are not depicting SOA in the "Real World." Moreover, they don't do a good job in explaining the organizations impacts; indeed many vendors are actually scaring the IT rank-and-file with talk of "system replacements" and "upgrading IT talent." In most organizations major change is not welcome, and many more savvy players see major spending and risk.

"Quocirca does believe that SOA is the future. Like many technologies and technological approaches before it, however, it runs the risk of being a golden goose killed before it has a real chance. Poor quality implementations, lack of suitable granularity in functional components, a lack of adequate understanding of what an SOA offers at a business level and the Wall Street pressures on application vendors to maintain revenues through selling tightly coupled application suites could result in perceptions that SOA just doesn't do what it says on the box."

That's the danger here. SOA is complex and hard, no matter how good the technology is that you toss at it. As soon as the end users understand that this is a journey, not a project, then the true benefits can be understood, and SOA will be what it really is...a way to optimize your business for success.

Posted by Dave Linthicum on October 6, 2006 05:10 AM

October 04, 2006 | Comments: (0)

Information vs. Services...Know the Differences

Services are very different than points of information production or consumption because they denote behavior. Indeed, they are application functions that you may extend from one application to another by leveraging some type of service delivery standard such as Web services.

Having said that there are many that still view the ability to expose information, albeit through services, as service-oriented. This is a common mistake, and we must keep in mind that the notion of SOA and SOI is all about the ability to share behavior and information bound to those behaviors, and not just information. The rise of messaging technology with Web services interfaces have lead to this confusion, while they are indeed leveraging services they are doing so to move information across a queue, not aggregate and manage remote application functions.

Services are a bit more complex to expose due to the intricacies of the native interfaces, if they indeed exist. Thus, how you expose existing services requires some creativity and perhaps some good technology. For our purposes I like to classify existing services in one of these categories:

- Existing-and-Exposed
- Aggregated and Single Transactions
- API Services
- Recast Standard Interfaces
- Ported Services

Existing-and-exposed types of services, are just what they sound like. They already exist and are already exposed as Web services. Examples of this include the newer versions of SAP and PeopleSoft that have redone their interfaces as Web services, and thus are ready to plug into a SOA. In some instances you may have to do some fine tuning for compatibility, but the idea is that your major enterprise software vendors do the service-enablement work for you. This, of course, is the best case scenario and least expensive and least risky.

Aggregated and single transactions are transactional systems that are exposed as services. Examples of these types of existing services are CICS, Tuxedo, and even J2EE transactions that are seen as Web services externally, typically through software provided by transactional container vendors. For instance, in the case of CICS, IBM's WebSphere is able to expose these transactions as services and almost all J2EE tool and container providers also offer Web services capabilities. This tends to be more of a natural 1-to-1 fit, since transactions are well defined and granular, and so should be Web services. This is less costly and less risky.

API services are APIs that are "wrapped" so they appear as services. This means leveraging an API-to-Web services translation layer, such as mechanisms some middleware vendors provide, or creating a translation layer on your own. The translation layer needs to manage the mediation between the exposed Web services that represent the API, and the API itself. This includes exception handling, reliability issues, as well as security.

Recast standard interfaces, like API services, means that we're looking to leveraging an existing standard interface, such as JMS, JCA, ebXML, RMI, etc., and expose them as Web services. In essence you’re recasting one standard for another, and the same problems that you solve when considering API services you need to solve here, including managing the mediation between the standards and exposed services, as well as all operational aspects.
The cost and risk tradeoffs are the same as with API services.

Finally, ported services, the last resort, is a rewrite exercise where you take existing application functions and rewrite them as Web services. This means a new architecture, as well as hand coding the existing interfaces so they appear as true Web services. As you may have guessed, this is the most risky and time consuming of all of these categories since you must suffer through the redevelopment, testing, and redeployment of the systems just to support Web services.

Posted by Dave Linthicum on October 4, 2006 06:43 AM

October 03, 2006 | Comments: (0)

Service Design and Step 11 and 12 of my 12 Steps to SOA

Download file

Posted by Dave Linthicum on October 3, 2006 05:53 AM

October 02, 2006 | Comments: (0)

BPEL Monday

There is more fallout from my posting a few weeks ago pertaining to the issues around BPEL 1.1, and moving to BPEL 2.0. It seems have started a debate here, and other bloggers have been picking up on it, either for or against.

For instance Joe McKendrick:

"The world has been having a love/hate relationship with BPEL — Business Process Execution Language. It is seen as the glue that will make all the moving parts of SOA work together, but often criticized as laden with vendor extensions and lacking flexibility."

Bruce Silver chimed in, and questioned the future role of BPEL for BPM:

"If you look at why BPEL 1.1 isn't portable for BPM, it comes down to three basic limitations in the language: no support for human tasks, no support for subprocesses, and pitiful data manipulation. BPEL 2.0 mostly fixes the data manipulation part, but not human tasks and subprocesses. So how can you use an orchestration language without support for human tasks and subprocesses? For creating business services! You get more than Fred’s "knowledge-portability." You get actual runtime portability, and a choice of engines at a commodity price. So it has real value there."

The bottom line, if you're a BPEL vendor or a consulting firm doing BPEL, you love BPEL and will defend any issues that guys like me bring up. However, if you're eyes are open, than many questions still emerge...I'm clearly there.

What bothers me most about this issue is not the fact that the standard, at least in some respects, is letting SOA down, but that orchestration is such a powerful need within SOAs, and there are few other alternatives that offer a better approach, standardized or not. Thus, end users who are tasked with building solutions using emerging SOA standards such as BPEL should begin to lean on their vendors now, as well as get involved more with their respective standards organizations.

Posted by Dave Linthicum on October 2, 2006 05:45 AM

REAL WORLD SOA PODCAST

Listen to the latest podcast:

MP3 •

•

• Archive •

Technology White Papers

InfoWorld Technology Marketplace

» Technology White Papers Library

Technology White Papers by Topic

Technology White Papers E-mail Alert