Real World SOA | David Linthicum | December 2006

MORE ENTRIES

December 2006

December 25, 2006 | Comments: (0)

SOA Expert Podcast Archives Out There Now.

Remember the SOA Expert Podcast? It came before the InfoWorld SOA Report, and many of you have been requesting access to archived Podcasts. As promised, here they are.

EDIT: I took out the request for user info.

Just click here for access. Get them now before I run out of allotted bandwidth.

Hope everyone is having a nice holiday break.

Happy listening.

Posted by Dave Linthicum on December 25, 2006 08:45 PM

December 22, 2006 | Comments: (0)

"SOA is hard to do?" You're Kidding Me.

I found an interesting article over on Tech Target pertaining to SOA adoption which I found to be a bit vindicating. The article basically highlights research done by the Aberdeen Group.

"As the lead for Aberdeen's fact-based research on SOA, the analyst looks at data he has gathered from a thousand-plus companies and concludes that if 2005 was the year of SOA romance, by the end of 2006, the honeymoon is over. The promise of SOA cost-savings remains its biggest lure, he said, looking over the research he has gathered during six trips out into the field where he visited IT professionals who are actually trying to make SOA work."

Clearly, SOA is a complex thing, and those promoting the quick fix are quickly found out as organizations seek the value beyond the complexity. SOA is a worthwhile venture for most organizations, but at the end of the day is a complex and systemic change in architecture, thus painful. We'll continue to get these data points back from the field in 2007, and I'm seeing them in my practice as well.

The article goes on to say:

"He notes that most of the IT departments he's visited are in the first year to six months of trying to work with the new technology. In many cases, they do not yet have the skills and knowledge to build a complete SOA infrastructure, but he found some that have made meaningful progress. While it is difficult and will take years, a big payoff looms for those who put in the effort."

The key points for a successful SOA are:

- Smart people who understand the concept holistically, and can execute tactically. In many instances you need to change out the talent, or infuse the project with mentors.
- A key pain point for the organization you're addressing. What business issues will SOA address?
- A realistic expectation of results, including a longer term outlook on the value of the approach.

SOA is indeed "hard," but nothing worthwhile is easy. Indeed SOA is a journey, not a project, and the sooner we understand that the better we'll be able to execute to the benefit of the business.

With that, I'm done blogging for the year. I'll post again after the 1st.

Until them happy holidays, and keep working on that SOA.

Posted by Dave Linthicum on December 22, 2006 08:11 AM

December 22, 2006 | Comments: (0)

"The Danger of Automatic Service Creation" Beef

It's always good to end the year with some sort of controversy, and thus one has sprung up. I'm talking about Ronan Bradly's post about The danger of "automatic" service creation and how it's counter productive to reuse.

From Ronan:

"However, my gripe is more fundamental: The idea you could even consider generating any kind of service automatically from code must be the antithesis of SOA."

"The automatic approach to service generation tends to result in hardwired ties and dependencies significantly reducing the likelihood that the service will be reused."

Ronan was pointing to the latest product release from Iona, but other are taking the same approach.

Of course Joe McKendrick put on his white shirt and bow tie and jumped between the two, and had some really good points about the topic, and good 3 good blog days out of this issue. This included some good points made by Iona's Steve Vinoski

At the essence of this is code generation, or in the case of Iona code transformation (JAX-WS 2.0.). This approach, at least the underlying concept, has been around for years, while the enabling standard is relatively new.

So, my take? While this approach will be compelling for some with words like "automatic," the reality is that generating services ties you to the technology that generated it. No matter what standard or agreements you can put forth. Thus, if you're generating services, those services are going to be coupled to the product that generated them. Therefore, reuse, agility, loose coupling, will all be things that are more difficult to achieve.

Thus, the notion? Bad. Indeed, it's a nice thought but it just does not work that way in the real world. I hope people get this concept before any damage is done.

Posted by Dave Linthicum on December 22, 2006 07:29 AM

December 20, 2006 | Comments: (0)

Reflecting on SOA in 2006

So, was 2006 the year of the SOA? Hardly. I think it was the year of talking about SOA, and perhaps dong some minor SOA projects. I think 2007 will be more of the same, but with an increasing amount of implementation work done towards the end of the year. Mark my words. Okay, I think I'm marking my own words.

Some of the positive things when considering SOA in 2006:

SOA has increased awareness of the underlying architectures within enterprises, and how they may or may not limit business. In essence we're shinning a light on bad architecture when dealing with the notion of SOA.
SOA has driven a new set of startup companies, with some pretty good ideas. These include governance, process management, and security, as applied to SOA.

I attended better and more informative SOA conferences this year. We seem to be getting beyond the hype, into the meat of the topic, and understand better the work that needs to be done.

Some of the negative or silly things, when considering SOA in 2006:

"SOA 2.0" The last time I'll say that one.
Most of those implementing SOA don't really get the concept yet, as I'm finding. Needed is more education and mentoring.

Perhaps too much consolidation in this space, and too soon. Most of those SOA fledgling companies never got to spread their wings.

The hype was too high. One needed to scream to get above the noise in 2006. Unfortunately, I think it's going to get worse before it gets better.

Posted by Dave Linthicum on December 20, 2006 04:57 AM

December 19, 2006 | Comments: (0)

My SOA Predictions for 2007

Download file

Posted by Dave Linthicum on December 19, 2006 04:18 AM

December 17, 2006 | Comments: (0)

Want to Join the Sigma Omega Alpha Fraternity? ZapThink is Taking Pledges.

Hard to believe that Animal House came out in 1978. I think it was the first R rated movie I say in my life, and I had to sneak in, the old buy a ticket for a Disney flick and switch theaters. Gezz, I hope the statue of limitations has run out on that one.

In any event, the best line from the movie:

"Otter: But you can't hold a whole fraternity responsible for the behavior of a few, sick twisted individuals. For if you do, then shouldn't we blame the whole fraternity system? And if the whole fraternity system is guilty, then isn't this an indictment of our educational institutions in general? I put it to you, Greg - isn't this an indictment of our entire American society? Well, you can do whatever you want to us, but we're not going to sit here and listen to you badmouth the United States of America. Gentlemen! [Leads the Deltas out of the hearing, all humming the Star-Spangled Banner]"

Truth-be-told there are not a lot of people out there who really get SOA. Out of the many that I know I would say that there are less than a dozen who understand the value and the issues holistically. Thus, you could call them a small fraternity...the Sigma Omega Alphas (SOAs). Sweatshirts and beer steins on order now.

Two people certainly in my fraternity would be Jason and Ron at Zapthink, both of whom have been working in the SOA/Web services space for some time. Recently, they have launched a certification program for SOA called Licensed ZapThink Architect (LZA). Thus, you could have Ron and Jason whacking you with a paddle in no time, and "Is that a pledge pin on your uniform!"

Jason and Ron describe the program as:

"Are you an enterprise architect or an aspiring architect looking to get third-party backing for your SOA capabilities?
- Are you not finding what you want in self-serving, vendor-specific "certifications" that don't provide the credentials you need to advance your architecture skills, career, and future direction?

- What does it even mean to have SOA on your list of qualifications?

- Do you want to leverage ZapThink's brand, reach, reputation, and network to advance your SOA efforts?

If so, then the Licensed ZapThink Architect (LZA) program is for you."

What I like about LZA, or at least the notion, is that "Knowledge is Good," and that most aspiring SOA architects need to focus more on the basics, versus the management-by-magazine crowd that I run into these days. Thus, any program that will take you through the basis of understanding of your own issues, and how the emerging approaches and technologies map into that, is a good thing. Thus, LZA, or programs like it, that increase understanding of the holistic notion of SOA could save millions by reducing the number of bad decisions. Mentors and/or good knowledgeable SOA consultants are effective as well.

However, certification programs do come with some bad history and those were very discrete technologies, such as Novell and Microsoft certifications. I never found those with certifications that much more knowledgeable than others without the certification. In other words they could memorize information, but that's not really engineering. Same could be said for the project management certifications, software engineering certifications, etc.

Architecture is much more complex and far reaching, and thus it will be difficult long term to create a program that relates to all enterprise issues I'm seeing today. Those skills are obtained more through experience than classroom training and test taking.

This is not to take the wind out of the ZapThink sails, it's an interesting program and I'm looking forward to the data points from this in months to come. However, if you see them approach you with a large block of ice and a jar of olives, run.

Posted by Dave Linthicum on December 17, 2006 06:22 AM

December 15, 2006 | Comments: (0)

"SaaS and SOA: Together Forever"

Now that I'm an independent I've been working with a number of authors and reporters attempting to figure out SOA, and while some are doing average jobs, some are dong great jobs. Case in point a nice article entitled "SaaS and SOA: Together Forever" written by Intelligent Enterprise's Doug Henschen. Doug does a great job linking these two notions which are indeed becoming one in the same, and also tell you how to begin the movement towards synergy between SOA and SaaS.

"From a SaaS delivery perspective, SOA is what separates the current generation of SaaS providers, epitomized by the likes of Salesforce.com and NetSuite.com, from the failed application service providers (ASPs) of the dot-com era. From a consumption perspective, you certainly don't require a SOA to use SaaS, but if you want to effectively mix and match external services with on-premise assets and services, SOA will make it possible to efficiently build, deploy and manage composite apps."

The article goes on to talk about the synergies between SaaS and SOA, including the notion of mash-ups and how they are changing the way we deal with SaaS and thus providing more value for the enterprise. Or, moving from a visual application delivered model to that of services and composites. Also, how you build up to SaaS and SOA, and the steps it takes to get there.

At the heart of this issue is how SaaS delivered services exists within the context of SOA. Doug describes how much planning is needed to get there, and how this is an emerging concept. Doug was nice enough to reference my SOA Meta Model as a point of reference.

"In other words, as Sinatra would put it, "the best is yet to come, and babe, won't it be fine." In this model of a services-oriented architecture, data sources and legacy apps (bottom tier) are wrapped in services and exposed along with original, new services. The data abstraction layer ensures consistent, virtual access to any data source. The enterprise services bus addresses data services/ messaging. SaaS and more granular Internet-based services join the services layer and may be orchestrated as part of larger composite apps."

I can't argue with that. Also, it's exciting to see this concept emerging, it's will indeed change the way we build business applications going forward. It's about time.

Posted by Dave Linthicum on December 15, 2006 05:12 AM

December 13, 2006 | Comments: (0)

When to Consider SOA Performance

Making solutions scale is nothing new. However, the SOA technology and approaches recently employed are largely untested with higher application and information and service management traffic loads. SOA implementers were happy to get their solutions up-and-running, however in many cases scalability is simply not a consideration within the SOA, nor was load testing, or other performance fundamentals. We are seeing the results of this neglect now that SOA problem domains are exceeding the capacity of their architectures and the technology.

What is more, many SOA technology vendors have not focused on scalability within their solutions. Instead, feature/function enhancements are the rule of the day. It's more important to add orchestration features and more adapters to your solution than to figure out how to pump more information, and manage more services, reliably with your product.

Eventually the number of connected services and the information traffic will saturate the available resources of the integration or application server (memory, processor, disk). The saturation point is dependent on the efficiency of the vendor's product as well as the how course or fine grained the services are as they exist within the SOA. Indeed, using a single server, means that the saturation point is fairly static.

So, what do you do?

First, SOAs are all about distributed services, and the ability to leverage and managed those services. Thus, the more processing you can place at the origin of the service, the better your SOA will perform. In many SOAs, the architects abstract the services to a single server, and performance can be somewhat problematic in larger implementations.

Second, many services are built on top of more traditional legacy APIs, and as such the translations between legacy APIs to expose them as services may cause performance problems.

Third, use of too many fine grained services may cause performance problems. Indeed, you should not be afraid to leverage fine grained services within your SOA. However, you need to understand the performance issues with doing so, taking careful consideration of the network bandwidth and how other applications leverage the services.

Forth, make sure to consider performance when selecting your orchestration layer. Many BPEL engines and process tools are notoriously poor performers, and can become the bottleneck for the SOA.

Fifth, understand the basic rule that while the value of a SOA is the ability to leverage many remote services, the more services you leverage the more problematic that your SOA will become. It's like links in a chain. This is just an architectural tradeoff that you need to consider, no magic formula here.

Finally, performance modeling and performance testing are not just good ideas, they are essential. No matter how much thought that went into your SOA, you won't know how it will behave until you put it through its paces, before it goes into production. Modeling allows you to predict behavior, and you should use that during the design. However, testing is the final validation. Make sure that your test plans consider real-life scenarios, simulating production as much as possible.

Posted by Dave Linthicum on December 13, 2006 08:40 AM

December 13, 2006 | Comments: (0)

Importance of Data

Download file

Posted by Dave Linthicum on December 13, 2006 08:35 AM

December 11, 2006 | Comments: (0)

Are SOA Architects Neglecting the Data?

After hanging out with a number of SOA builders over the last few months, it's come to my attention that many are not focusing holistically when it comes to their architecture. Specially, what's missing, is both an understanding of the data and thinking about the notion of SOA agility when it comes to data. Let's face it, if your schemas are not agile than your SOA will never be, thus you need to focus from the bottom up, making sure that all layers are changeable, meaning coupling is kept to a minimum.

The issue is that the "SO" in SOA is "service oriented," thus that is where the architects are focusing...service enablement, service management, service governance, etc. However, the base of any SOA is the information, and how it's both managed and abstracted is critical to the success of any SOA.

So, what do you do? Well if you've been following my approaches I want you to first have a semantic understanding of your problem domain. This means analysis of the underlying data sources, including enterprise applications, and databases new and old. Once that occurs, and you do indeed have a semantic understanding, now it's time to group the attributes logically, in essence designing a new abstracted database that better represents the data to the services layer, and eventually the process/orchestration layer.

A data abstraction, or a data services layer, provides the loose coupling between the services and the underlying databases/information stores. As such, provides a point of configuration to both deal with the differences between the data as physically represented, and the preferred logical representation to the SOA.

Moreover, as both the data and service requirements change over time to align with the business, the data abstraction layer can be easily reconfigured to account for the differences, and since not coupled, require a minimal amount of work. For instance, the physical databases in many instances need not be changed, just the abstraction. In essence you've created an agile data layer, to go with your agile services and process layers. Without this you'll find that holistic agility with your SOA is difficult to achieve.

Posted by Dave Linthicum on December 11, 2006 06:25 AM

December 08, 2006 | Comments: (0)

When the ERPs Attack

Lately I've been hearing of a lot of ERP vendors making references to their dominance in the SOA area. Okay, their future dominance in the SOA area. Indeed, SAP just rattled their saber according to Joe McKendrick's blog.

"Shai Agassi, president of SAP's product and technology group, about SAP's emerging SOA strategy, who promised that SAP would be leading the enterprise SOA market by the end of 2007."

That's so, 1995. Indeed, if the ERP guys understood anything about SOA they would know that there is no way they can lead that market because it's all about systemic change to the enterprise, not layering in more processes, services-enabled ERP systems, or middleware. This kind of stuff drives me nuts.

Don't get me wrong, ERPs have value and SAP has a good one. However, at the end of the day SAP is all about buying business processes, not the interconnection of those processes/services, and certainly not about working and playing well with other processes/services, no matter how many complex stacks and charts you can put in front of me.

However, they could have the effect of freezing the market for a bit, as SAP loyalists wait and figure out where their "SOA platform" will go. That will only damage those companies, if you ask me. When they finally get with reality, they will be a year or two behind...and that's like being an hour 1 late to a marathon, and expecting to win. To some extent, this as already occurred.

So, here's my advice to the ERP companies:

1. Focus on what role you'll play in most SOAs. Don't focus on being the entire stack. It won't work, and you'll look silly trying.
2. Service-enabled your platform so it works and plays well with other technology, and provide tools for tweaking if it does not.
3. Understand that you'll now be competing with other enterprise services and composite applications providers, and in many instances you'll see some of your services being pushed out of the way for processes or services that are a better fit for your customer.

And, here's my advice to the ERP users:

1. Always consider the role your existing ERP and CRM applications will play within your SOA. Keep to the strategy, make sure to understand your own issues, and then look for the technology that fits.
2. Test the service interfaces now before you get too far down the road with some assumptions.
3. Communicate with your ERP and CRM vendors as to what their issues are in the context of your SOA, if any.

Posted by Dave Linthicum on December 8, 2006 06:11 AM

December 06, 2006 | Comments: (0)

Notes from the OMG SOA Information Day and Should you Fire your EA?

Download file

Posted by Dave Linthicum on December 6, 2006 08:33 AM

December 04, 2006 | Comments: (0)

Should You Fire Your Enterprise Architect in 2007? Take the Test.

Now working directly on SOA projects, I'm exposed to many more organizations than I did when I was building technology. As such, I see some common patterns or issues emerging.

The largest and most disturbing issue, and as I mentioned a few times here on this blog, is the fact that there seems to be a huge chasm between the traditional enterprise architecture crowd, and those looking at the value of SOA. Indeed, enterprise architecture, as a notion, has morphed from an approach for the betterment of corporate IT to a management practice, at least for some. Thus, the person that is needed to understand and implement the value of SOA is sometimes not the current enterprise architect in charge.

The core issue is an add-not-change approach to architecture. While adding applications, directories, databases, to an existing architecture is easy and risk adverse, changing architectures around systemic notions such as SOA is hard and does come with some risk. Thus, many are choosing to ignore it. In many instances it's the culture, with some organizations promoting a "you fail you're fired" approach, versus a "let's try new things and seek improvement."

Another issue is that it's easier to stay high level, than do actual work. Drawing diagrams, doing presentations, and writing reports is much easier than actually going out and making real changes with real benefits. Again, from above, that carries with it the notion of risk. Implementing SOA takes a lot of up-front work, as well as many changes. However, in many cases the benefits outweigh the risks by a large margin.

So, should your enterprise architect be shown the door to make room for others looking to promote a better and more agile IT architecture, and are willing to do what it takes to get there? Here are a few questions to ask.

1. Has somebody compared the current architecture with best practices in you industry, looking to spot issues that need correcting, such as the architectures inability to align and keep us with the business?
2. Has somebody done a ROI analysis of the value of SOA, or other approaches for that matter, for the current architecture and reported it to management?
3. Do you have a complete service-, semantic-, and process-level understanding of your enterprise?
4. Do you have a common abstract model for key elements, such as customers, sales, inventory, transactions, etc?
5. Are systems well integrated and communicate in real time where needed?
6. Can you change your architecture to accommodate business changes at the speed required by management and the marketplace?

Basically, if you answered no to any of the above, it's perhaps time to look for some new blood...New Year, new opportunities, new value, new architect.

Posted by Dave Linthicum on December 4, 2006 04:34 AM

December 01, 2006 | Comments: (0)

Has SOA Reality Hit?

It seems to me that many of the organizations looking at SOA are now moving from the conceptual and study phases, to real projects. The analysts are looking at this trend as well, and I think that 2007 is going to be the year of the major SOA project upstarts.

Unfortunately, when you move from the hyped phase of looking at a new notion, such as SOA, to the realities of actually doing something, you typically find that the hype somehow falls short. While I have not heard about widespread disappointment yet, it's pretty much always an issue in the lifecycle of a TLA. This does not undermine the value of the notion of SOA, but that you can't really get "SOA in a box," nor does "SOA 2.0" exist, perhaps because we never figured out what SOA 1.0 was, really.

So, what will the realities be?

First, that SOA is a systemic change in the way we do architecture, and layering on technology won't provide enough change to make SOA work. You need to drive the concepts down deep into your enterprise, down to applications built years ago. That's where things get a bit tough.

Second, you need smart people to pull this off. Thus, your existing IT guys may have to go, or at least get completely retrained. The people issues here, I'm finding, are the stumbling point for most organizations.

Finally, you need to lubricate the skids with cash. SOA is not cheap, and most of my clients are understating the cost of this "systemic change." The value is there, once complete, and you will get your money back. However, the startup costs are pretty high.

Let these sink in, and you won't have a "Reality Bites" moment next year.

Posted by Dave Linthicum on December 1, 2006 05:28 AM

REAL WORLD SOA PODCAST

Listen to the latest podcast:

MP3 •

•

• Archive •

Technology White Papers

InfoWorld Technology Marketplace

» Technology White Papers Library

Technology White Papers by Topic

Technology White Papers E-mail Alert