Real World SOA | David Linthicum | February 2007

MORE ENTRIES

February 2007

February 28, 2007 | Comments: (0)

Reference Models, Reference Architecture, and Enterprise Architecture...Oh My!

Some of you may have tuned into the Shared Insights Webinar on "Marching Toward SOA: Does EA Lead the Band?" I can't find the archive of it, by the way. However, there is a good summary article here, by Rich Seeley.

So, this is the larger issue, as I see it, and is very visible to me working both within the world of SOA and the world of Enterprise Architecture. So, why are they different worlds? Moreover, what is Enterprise Architecture, and how does it fit with reference models and reference architectures, as discussed in my two previous blog posts?

From Wikipedia:

"Enterprise architecture is the practice of applying a comprehensive and rigorous method for describing a current and/or future structure and behavior for an organization's processes, information systems, personnel and organizational sub-units, so that they align with the organization's core goals and strategic direction. Although often associated strictly with information technology, it relates more broadly to the practice of business optimization in that it addresses business architecture, performance management, organizational structure and process architecture as well."

Thus, the notion has morphed from something that's been more of a technology concept to a management concept, if you're using this definition. I've blogged about that issue several times. Indeed I think the government is driving some of this, thus expanding the definition and reach of the notion of Enterprise Architecture.

"Enterprise architecture is becoming a common practice within the U.S. Federal Government to inform the Capital Planning and Investment Control (CPIC) process. The Federal Enterprise Architecture (FEA) reference models serve as a framework to guide Federal Agencies in the development of their architectures. The primary purpose of creating an enterprise architecture is to ensure that business strategy and IT investments are aligned. As such, enterprise architecture allows traceability from the business strategy down to the underlying technology."

Also, from Wikipedia.

So, if we go with this definition we are moving from a high-level management notion, including strategy, business optimization, budgets, etc., to information technology, including SOA. EA is more holistic, where SOA is a more specific approach for building an IT infrastructure.

I'm not sure that I can do anything to change the definitions here, or eliminate the confusion when considering the number of ways everyone is slicing and dicing SOA and EA. At the end of the day, it does not matter as long as enterprises find the right approach that works for them. Typically that takes some effort and planning. Truth-be-told, there are no canned SOA solutions out there, at least, not ones that will have significant strategic value.

Besides ending the debate about unified semantics and vision, it's also helpful to consider the "loop" as well, or, mechanisms we put in place to understand and create best practices based upon experiences with building and deploying SOAs. The key issue here is sharing information, something we've not been good at in the past. I can understand within certain industry-specific competitive situations where it's not in the best interest of the company to share information.

There are a few issues as I see things:

First, as already stated, we need to clearly define use cases for SOA-RA and SOA-RM, and learn to adapt these models to the very different problem domains I’m seeing out there. Like any new notion, this will come with use, and these models and architectures will only be as valuable as the data points that are fed back to the creators, and the changes made to accommodate the "real world." As a SOA practitioner I'm finding that a huge chasm still exists between those who define the concepts of SOA and those who actually do the work. That chasm needs to narrow significantly.

Second, what's the uptake thus far when considering the concepts of SOA Reference Architecture and SOA Reference Model? It appears to me that most of my clients are pressing forward with their SOAs, formal models and architectures be dammed. Unfortunately, defining something in a formal way does not mean the rank-and-file will use it. I think the recent debate is a clear indication of that. I can go back in history and point to many instances where good formal models existed, but nobody leveraged them, opting for models that were more ad-hoc, understandable, and available. I do promote the standards effort, however. Please don't misunderstand me.

Finally, SOA, at least in my mind, is a bit more systemic to enterprise architecture than just an "architectural approach." Indeed, if SOA is to be successful, those responsible for enterprise architectures need to understand SOA. Many do, some do not. If SOA is going to provide the proper benefits, SOA needs to layer deeply into business processes, business strategy, information technology, and even capital planning. In essence, it becomes a new foundation in many respects, and as to the point I made many times before...a good SOA is a good enterprise architecture, and not a mere instance of technology or another silo. We need solutions, not another "bolt on" that just adds complexity, and hinders agility.

Hope this helps. On to more technical topics next.

Posted by Dave Linthicum on February 28, 2007 04:49 AM

February 26, 2007 | Comments: (0)

To SOA, or not to SOA, that is the Question

Download file

Posted by Dave Linthicum on February 26, 2007 09:01 PM

February 26, 2007 | Comments: (0)

Understanding SOA Architectures and Models...SOA RA

Continued from my last blog...while there are SOA Reference Architectures all over the place, including mine, the best known SOA Reference Architecture (SOA-RA) is defined by OASIS.

So, what's a reference architecture and how does it relate to a reference model? In short, a reference architecture is a description of how to build a class of artifacts. An architecture describes how to build a particular artifact. The appropriate way to write the description for a reference architecture depends on the particular artifact.

While the definition is changing according to those writing the standard, the SOA-RA provides a bridge between the concepts and vocabulary defined by the SOA Reference Model (SOA-RM) and the implementation of a SOA. In other words, the SOA reference architecture models the abstract architectural elements for a SOA independent of the technologies, protocols, and products that are used to implement a SOA.

I have to agree with this, albeit it is a bit confusing. They are describing a high level of abstraction to define a SOA, the "reference architecture," and the "architecture" as an instance of a SOA. I get that. However, the larger issue is the fact that the problem domains I'm seeing are not as similar as you think, thus the questions is: Can you define a single class of artifacts, and thus provide a sound "jumping-off-point" for the instance? I think a few use cases will prove this out. I could not find many, so send them to me if you have them...I'll post them here. However, to be fair to the creators of the standard, this is still a work in process.

Also confusing is the number of SOA Reference Architectures you see out there, including this one from webMethods.

What’s more, you can find more vendor-created models going by different names, but basically attempting to define the same thing..a reference architecture for SOA. However, most appear to define the same notions as put forth with the SOA Reference Model (discussed next).

Here are some others:

Burton Group SOA RA

IBM SOA RA

BEA (+ some consortia) SOA RA

Some of the key issues, as I see it, are:

There really needs to be some fundamental discussions about the use of the Reference Architecture and the Reference Model in the real world. Based on what I found out, as an outsider, there seems to be an impedance mismatch between the way the architecture and model is defined and what's currently going on in the world of SOA. I'm assuming that will "self correct" over time.

It's unclear as to how all of this reaches up into the domain of the Enterprise Architecture...perhaps not as a replacement, but an augmentation. If so, how do we approach that considering the other frameworks employed?

Like many written standards, the approach is somewhat confusing. Not that the standard itself is bad. I don’t think that's the case; but it’s difficult for those tasked with building a SOA to see how it will mesh with their current architecture and their current thinking. Over the years I've found that to be as important as good concepts.

Once again, NASA JPL's Jeffrey A. Estefan assist with fact checking this post.

Posted by Dave Linthicum on February 26, 2007 07:07 AM

February 22, 2007 | Comments: (0)

Understanding SOA Architectures and Models...SOA RM

A few people who have been reading my blog and listening to my Podcast, as well as reading other SOA blogs and articles, have become a bit confused pertaining to the notions of:

- SOA Reference Model(s)
- SOA Reference Architecture(s)

And how all of this works and plays with

- Enterprise Architecture

In fact one of those creating the standard, NASA JPL's Jeffrey A. Estefan, who responded to a past blog post, assisted me in fact checking this series of posts since some of this information was a work in process, and Jeff has better context as an insider.

So, I spent a few hours of my weekend attempting to research and define these concepts a bit better, in essence, taking everyone's opinions and normalizing them so they make better sense. What I found were many of the same notions, defined differently, but all attempting to solve the same problems. Seems to be a common theme within the world of SOA...but I digress.

Indeed, there are many definitions for the above concepts (not those terms specifically), that are now being defined by guys like me, standards organizations such as OASIS and the Open Group, and vendors such as IBM, BEA, webMethods, and TIBCO. Sometimes they align; most of the time they don't.

So, who's right? I'm not sure this is a matter of right and wrong, but perhaps it's time we come up with some common definitions and shared vision, as I've been saying. I do think we are moving in that direction, albeit slowly, and I think that just agreeing on semantics will be a huge accomplishment in this emerging space.

I'll tell you what I know; you can make your own choices. Let's begin by exploring the concepts being put forth by the standards organizations over the next several blog-days. This will be backing up a bit from the recent posts and columns that have basically restated and commented on the news...I'm attempting to provide better context.

Also, to avoid the many e-mails and comments I get when writing about this stuff from those who write the standards and define these terms, I'm going to be quoting more than I typically do. Of course, I have to add my 2 cents. Here goes...

SOA Reference Model

First let's explore the concept of the SOA Reference Model, also a product of OASIS, and how SOA Reference Model and SOA Reference Architecture relate to each other. I'll talk more about the SOA Reference Architecture in my next post.
From the OASIS Reference Model for Service Oriented Architecture.

"A reference model is an abstract framework for understanding significant relationships among the entities of some environment. It enables the development of specific reference or concrete architectures using consistent standards or specifications supporting that environment. A reference model consists of a minimal set of unifying concepts, axioms and relationships within a particular problem domain, and is independent of specific standards, technologies, implementations, or other concrete details.

As an illustration of the relationship between a reference model and the architectures that can derive from such a model, consider what might be involved in modeling what is important about residential housing. In the context of a reference model, we know that concepts such as eating areas, hygiene areas and sleeping areas are all important in understanding what goes into a house. There are relationships between these concepts, and constraints on how they are implemented. For example, there may be physical separation between eating areas and hygiene areas.

The role of a reference architecture for housing would be to identify abstract solutions to the problems of providing housing. A general pattern for housing, one that addresses the needs of its occupants in the sense of, say, noting that there are bedrooms, kitchens, hallways, and so on is a good basis for an abstract reference architecture. The concept of eating area is a reference model concept, a kitchen is a realization of eating area in the context of the reference architecture."

Furthermore:

"The goal of this reference model is to define the essence of service oriented architecture, and emerge with a vocabulary and a common understanding of SOA. It provides a normative reference that remains relevant for SOA as an abstract and powerful model, irrespective of the various and inevitable technology evolutions that will influence SOA deployment. The concepts and relationships defined by the reference model are intended to be the basis for describing references architectures and patterns that will define more specific categories of SOA designs. Concrete architectures arise from a combination of reference architectures, architectural patterns and additional requirements, including those imposed by technology environments."

Again, abstraction, one is an instance(s) of the other, if I understand this correctly. I urge you to download and read this 31 page document to get some additional details. Also, get involved, if you like. I always love the comeback from standards bodies who deal with criticism and debate around the concepts they put forward..."If you don’t like it, join us and change it." Can't argue with that.

I would, however, recommend that OASIS make this a bit easier to understand for the typical end user, as I stated before. Jeff sent me the graphic attached to this blog, which helps in figuring out the relationship between the reference model and the reference architecture (discussed in my next blog). Again, perhaps provide published use cases for the RM and RA, and thus make the notion a bit more consumable. I'll consider this an open-ended definition for now, so please send your use cases and I'll post them here. In essence, how the reference model will be used in a sentence.

You have to give them an A for effort, but you can see where debates such as the one I blogged about are going to pop-up. It's really a matter of understanding...a marketing issue really. I understand different levels of architectural abstractions, but what is really needed is a reference framework or model, and a set of steps to figure out how to build something that's proper for your problem domain. I've been building that recently, and I'm happy to map them back to this reference model if it makes sense for the project. The uses cases out there are like snowflakes, and one size, or one model, does not fit all. You have to account for that within all this formality.

However, to be fair, if you Google "SOA Reference Model" or "SOA Model," or things of that nature, you'll find that most SOA vendors and consulting organizations have their own SOA (reference) models, SOA frameworks, SOA Roadmaps, or SOA reference architectures. They are a bit different than what OASIS is proposing, but again, looking to solve the same sets of problems.

My best advice is to leverage the architecture, models, roadmaps, or frameworks that will work best for you. At the end of the day, I think clarity of approach will win over complexity, as long as you can do that without diminishing the value.

Next, we'll talk about the SOA Reference Architecture in more detail.

Posted by Dave Linthicum on February 22, 2007 04:02 AM

February 21, 2007 | Comments: (0)

3 Things I Forgot the Mention in the Mashup Article

If you spotted my article, "Enterprise mashups meet SOA," in this week's InfoWorld you may have noted some of the core ideas, such as:

"The line is blurring between the enterprise and the Web. Mashups live on that porous perimeter, offering the reusability of an SOA plus very rapid development using prebuilt services outside the firewall. Soon, we may live in a world where it's difficult to tell where the enterprise stops and the Web begins. It's scary - and exciting at the same time."

And

"Mashups and SOA are part of the same continuum. By linking the new components of Web 2.0 with our own sets of information and services, mashups provide a quick and easy way to solve many of today's simple business problems — and should scale nicely to solve more complex and far-reaching problems in the future. They make the value of an SOA much more visible over a much shorter term."

However, there are a few things I should have added to the article, and one of the great things about having a blog is commentary on my own content after the fact. So, here they are:

1. You really need to solve a specific problem when thinking mashups and SOA...such as mashing up accounting services with logistics interfaces to determine the best routes for sales on an ongoing basis. Complex mashups should be avoided, perhaps approach those as complex composites.

2. As you inventory your SOA components, you need to think "what's mashable," or how the technology will work and play well together. Many assume this will be the case, perhaps because the use of "standards." However, more often than not the technology is incompatible and thus not "mashable." I have a client who is dealing with this now.

3. Mashups unto themselves have the potential to become sources for other mashups, or services. Thus, this notion should be considered within the design.

Posted by Dave Linthicum on February 21, 2007 08:27 PM

February 19, 2007 | Comments: (0)

More on SPO...

Continued from my last blog on SPO...Service Performance Optimization

Second, Experiment and Test

Many of those who focus on the discipline of performance within complex distributed systems such as SOA will first steer you toward modeling. Unfortunately, we don't know enough about how services behave to model how they will perform, so it's a good idea to test the services that will make up your SOA before you build your performance model; otherwise, you're just guessing.

So, how do you test services you've not yet built? It's called a "proof-of-concept," meaning you stand up very raw and simplistic versions of the services (either existing abstractions or new services) for the purpose of proving that they work and to illustrate their operational characteristics. This is typically done in parallel with existing design work, and the proof-of-concept is largely a throw away after you gather your data, but nonetheless important to your understanding of the final product before you complete the design and development.

Testing services, even proof-of-concept services, means that you simulate operational characteristics during the test, or, how you intend to leverage the service. You do this by building or buying test harnesses that can load the service as needed for testing. You should utilize low use, medium use, and high use scenarios to determine how the service behaves under an increasing load, and make sure you have some sort of monitoring mechanism to gather the data for analysis.

What you'll find, in most cases, is that the service will reach a saturation point where performance drops off significantly as the load increases. The saturation point is largely dependent on the patterns of the service. For instance, transactional service should be able to support a much higher load than light weight services.

Creation of a SOA Performance Model

SOAs are not unlike any other distributed computing systems, and thus designing a performance model should be nothing too new. At this point we understand exactly how each service behaves under an increasing load, and we have enough data to plug into a model. Now, it's just a matter of building a model.

There are very expensive performance monitoring and simulation tools that are for sale in the market, but sometimes the least expensive and most simplistic tools work best...in many cases, just a spreadsheet will do. For our purposes, we need to consider both information and behavior in the context of performance, also core features of a SOA.

Information Movement Modeling, typically asynchronous in nature, means we're attempting to simulate how information moves from point-to-point, point-to-many points, or, many points to many points.

Based on the information we accumulated we know:

- Information production rate from a service.
- Information consumption rate from a service.

For example, an instance of a service is able to produce 52 messages (or similar groupings of information) per second…the source service. An instance of a service is able to consume 34 messages per second...the target service. This is a simple point to point relationship, but keep in mind that multi-points to multi-points are always possible and those are a bit more complex to model since you have to determine patterns of movement between multiple points vs. all messages produced by a single service that are consumed by another.

Moreover, keep in mind transformation and routing latency is typically an issue here as well, and needs to be modeled along with consumption and production. You should have test data from these services, but the performance of transformation and routing services will be largely dependent upon the complexities of the transformations and logic associated with the routing. What many do when creating performance models is to model very complex, complex, and simple transformation scenarios, and the percentages of each.

Service Invocation Modeling, typically more synchronous in nature, means we’re attempting to determine the number of times a service is able to provide a behavior (application function) in an instance of time, typically a second.

For instance, you may have a service that provides a risk calculation for the insurance business, and is perhaps abstracted into several different applications (composites). We know through testing that each composite can invoke the service up to 100 times a second before it hits a saturation point, meaning the performance of the service quickly diminishes as additional load is placed upon it. This saturation number plugs into the model, as well as the number of applications that are abstracting this service. You have to model all of these services in the same way.

Models are important because they allow you to predict performance under changing needs without having to actually build and test the system. Models, of course, are not perfect, and you must constantly adjust assumptions and modeling information as you learn more about the behavior of the architecture.

Designing for SOA Performance, Monitoring, and Optimizing

So, now that we know how to diagnose the performance of a SOA, as well as model for it to determine how it will behave under a changing environment, how do we design a service and SOA with optimized performance? Here are a few tips.

- The more processing you can place at the origin of the service, the better your SOA will perform. In many SOAs, the architects abstract the services to a single server, and performance can be somewhat problematic in larger implementations.
- Many services are built on top of more traditional legacy APIs, and as such the translations between legacy APIs to expose them as services may cause performance problems. The ability to leverage existing legacy systems as services is a powerful notion. However, you must be careful in selecting the proper enabling technology to do this. Service invocations that take a second or more to produce behavior, or information bound to behavior, will cause big problems when you align them with hundreds of other services that are doing the same thing.

- Use of too many fine-grained services may cause performance problems. Indeed, you should not be afraid to leverage fine-grained services within your SOA. However, you need to understand the performance issues with doing so, taking the network bandwidth and how other applications leverage the services into careful consideration.

- Make sure to consider performance when selecting your orchestration layer. Many BPEL engines are notoriously poor performers, and can become the bottleneck for the SOA.

- Understand the basic rule that, while the value of an SOA is the ability to leverage many remote services, the more services you leverage, the more problematic your SOA will become.

Core SOA Performance Issues

Making solutions scale is nothing new. However, the SOA technology and approaches recently employed are largely untested with higher application and information and service management traffic loads. SOA implementers were happy to get their solutions up–and–running, yet, in many cases, scalability is simply not a consideration within the SOA, nor was load testing or other performance fundamentals. We are seeing the results of this neglect now that SOA problem domains are exceeding the capacity of their architectures and the technology. It does not have to be this way.

Posted by Dave Linthicum on February 12, 2007 04:12 AM

February 09, 2007 | Comments: (0)

Service Performance Optimization (SPO)

I've been running into a lot of "early SOAs" that I'm looking at on behalf of my clients . These SOAs just don't provide the performance my clients had expected.

Why? Well, I guess you can blame a lot of things for poor performance; however I've found the core architecture and design issues are the most relevant variable. So, I figured we would focus a bit on service design for performance...what works...what does not.

There is indeed a right way and a wrong way to design a service. Also, there are things out of your control that you must consider during your design. As with anything else, you need to do your homework, allow enough time for design, and do some experimenting and proof-of-concept testing to determine the best path.

First, Know Your Service Patterns

A few patterns are beginning to emerge. We can categorize them into larger buckets, including: legacy abstraction, simple composites, complex composites, and new autonomous services.

Furthermore, we can put them into behavioral sub categories, including: transactional, data services, light weight and heavy weight services. Notice there is no mention of fine grained and course grained, I'll blog about that next week.

Legacy abstraction services are services built on top of existing services, including elderly technology such as Cobol and CISC on the mainframes, or perhaps services liberated from mini computers, or even enterprise class Unix systems. You can toss ERP and CRM applications into this mix as well. The notion is that you somehow are able to externalize these internal processes as services and leverage them as modern Web services, no matter how ugly and arcane the interfaces are.

Simple composites are one or two services that are bound together in a new service. Complex composites are many layers of services that are bound together, perhaps a composite that's made up of other composites. New autonomous services are services that are created for a single purpose such as a Web service, and are typically not based on other services (non-composite).

Transactional services can be a simple or complex composite, or even new autonomous, but they support transactional characteristics including ACID. For those of you who have not seen ACID as many times as me, Atomicity refers to the "all or nothing" quality of transactions. The transaction either completes, or it does not. Consistency refers to the fact that the system is always in a consistent state, regardless of whether or not it completes the transaction. Isolation refers to the transaction's ability to work independently of other transactions that may be running in the same environment. Durability means that the transaction, once committed and complete, can survive system failures. With new standards such as WS Transaction, how you build a transactional service should be more consistent. For now, developers are taking their own unique approaches, typically leveraging TP monitors or application servers.

Data services, as you might expect, are services that are built to produce and consume data. These could be Web service abstractions on top of call level interfaces, or simple services exposed out of an ERP system that produces data. These are very simplistic services, with schemas, access controls, and the encapsulated data. Almost always these are services built on top of relational database, but other database types are leveraged as well. Moreover, through a data services abstraction layer, you can emulate database types to meet the needs of your SOA.

Light weight services, as the name implies, means that you're doing things with a light volume (typically less than 10 invocations or messages-per-second), and the size of the message that the service is passing is small (typically less than 50 KB). Heavy weight services, in contrast, do heavy volumes (greater than 10 invocations or messages-per-second, but more typically 100-300 invocations and message-per-second), and can transmit and consume huge messages.

Okay, get service patterns? Over the weekend I'll dig into design.

Posted by Dave Linthicum on February 9, 2007 03:58 AM

February 07, 2007 | Comments: (0)

SOA/EA Convergence is Coming

I've been thinking a bit about my post on Sunday night, specially the issues that seem to be arising between the EA community, and the SOA community. I think the Open Group's Chris Harding did a great job in his response to my post:

"David, you are spot on with your comment that SOA and EA are both Architecture. I think the problem is that there are two different communities of architects, and they talk to each other less than they should. But there is hope that they can, and in future will, communicate."

Okay, when will that happen, and who gets hurt in the meantime? It's the businesses we serve, I'm thinking. Moreover, this kind of discourse could be causing more harm than good, and perhaps it's time that we make strides to come together on this stuff sooner rather than later.

I've long said that service oriented architects are enterprise architects with earrings. While that's meant to be funny, there is a certain notion of truth there. Indeed, those that focus on SOA are architects in their own right, but typically work on smaller problem domains, usually a subset of the enterprise. However, if not running the entire architecture, and have influence and resources, they can't be effective longer term. For instance, agility won't have that much value if you're only making 6 out of 200 systems agile.

Thus, if SOA is a better approach than current practices, than it will indeed prove its way into the enterprises today, but will be as it should be, systemic to enterprise architecture, and not replacing it.

However, it's going to take some rethinking and reevaluating on the part of many of the enterprise architects out there, as well as some additional work from those promoting SOA within their own firewalls. At the end of the day, there has to be a convergence, else we won't be doing what we're paid to do, enhance the businesses we support.

Posted by Dave Linthicum on February 7, 2007 03:08 PM

February 06, 2007 | Comments: (0)

What is a SOA Reference Model?

Download file

Don't forget about the free "SOA Jump Start Webinar" next week. Sign up here.

Posted by Dave Linthicum on February 6, 2007 03:58 AM

February 04, 2007 | Comments: (0)

Open Group debates SOA Reference Architecture...

I did not make it out for the Enterprise Architect Practitioners conference in San Diego last week, but I wish I did. From this article and reports from a few of my friends at the conference, looks like there was a bit of a "discussion" about the SOA Reference Architecture.

"...when Chris Harding, Forum Director for The Open Group's SOA working group, proposed an SOA reference architecture at the organizations Enterprise Architect Practitioners conference in San Diego this week, the room erupted in a surprisingly vigorous debate."

"And that's where the debate began. Some in the audience confused the maturity model with a reference model.
Others questioned whether the world needed yet another reference model, saying that SOA should be just another "view" or slice of an enterprise architecture model. The rationale is that both EA and SOA both aim for a similar goal, of integrating and harmonizing processes and technology standards, and that developing a separate reference mode for SOA would degenerate to reinventing the wheel."

Yep. This is why we need some additional leadership here, and as I discussed many times on this blog and Podcast.

There seems to be two worlds out there, the world of enterprise architecture and the world of SOA. The funny thing is that those in each world thinks that they can do the other world's jobs. The end result...there is not a lot of synergy there, and I fault both sides. This debate is an indication of this.

The traditional enterprise architects have not done a stellar job in understanding the opportunities within SOA, generally speaking, and the SOA guys have not figured out how SOA meshes with existing enterprise architecture standards, notions, and practices, again generally speaking. Thus, when new ideas come about, on either side, no matter how valuable they are, they just serve to muddy things up even more.

Some smaller issues:

"There was debate over whether agility belongs in an SOA reference architecture, or is simply the result when you follow the reference architecture and have the right internal climate. The question was whether reference architecture should include business practices, or simply provide a recipe for the technical pieces that can enable you to achieve the desired business result."

I get the debate here. Agility is a valuable byproduct of a well designed and implemented SOA, but not really a part of the architecture. No biggy.

"Furthermore, with pieces such as standards in SOA still a work in progress, many in the audience questioned whether it made sense to define a reference architecture when nobody knows what the final pieces will be."

It's okay to define a reference architecture without the use of standards or technology. Keep in mind the technology and standards will always change, good reference architectures should transcend technology and standards, and the concepts around SOA are well defined at this point.

Back to my original point. SOA is architecture, and enterprise architecture is also architecture. The objectives are pretty much the same, so we better figure out how both work together, else we'll just see more of these debates and nothing will get done.

Posted by Dave Linthicum on February 4, 2007 03:41 PM

February 02, 2007 | Comments: (0)

Should Your SOA Include a Persistence Strategy?

Truth-be-told traditional approaches to integration are really about keeping persistence at the points, within the source or target systems, and replicating data as needed. However, with the use of true services there is a clear advantage, in some cases, in keeping some persistence at a central-tier, for any number of legitimate reasons. Let's explore this in the context of a SOA.

Indeed, as we become better at building services, we need to understand the infrastructure that the services will leverage including orchestration, security, management, and data, and where those functions need to reside in the architecture. While SOAs are like snowflakes, everyone a bit different, it's helpful to understand the requirement patterns that will lead to specific architectural decisions. Keep in mind that once these decisions are made, they are not easy to undo later.

Thus, you end up with composites or processes created out of services that may exist over a dozen or more different systems, and as such persistence becomes more complex if done at the points. So, in these types of situations, which are becoming more common, it makes good sense to centralize the persistence for the composites and processes, as well as some supporting services to a central data tier or central data service. This data tier exposes a custom schema view or views to the composites, and may also abstract some data at the points as well. This is done to provide a data service to the composites directly, or perhaps using a data abstraction layer such as data abstraction middleware.

If the composites are doing most of the processing, and it's really a center-tier process abstracting remote services, than it makes sense to collocate the data as close to the data processing as possible. This done for both manageability, reliability, and for performance.

Integrity will also become less of an issue when leveraging this type of center-tier persistence. No need to lock a dozen or so tables when you can simply lock one. Moreover, security becomes an easier process as well, since it does not need to be as distributed.

We all understand the value and leveraging a message warehouse, or the storage of information flowing between systems. Having persistence at the central-tier allows architects to store transactional information for many purposes, including analysis and integrity management issues (logging). Also, with the new focus on compliance and support of audits, this seems to be a likely place to store that type of information as well.

I would also include this notion in support of state management information for services, processes, or composites. This includes supporting long term transactions, or multi-party transactions. Again, the controlling data is maintained at a center, shared tier.

We all know that we need to understand and manage application semantics. Leveraging central-tier persistence allows the SOA architects to get a better handle on this issue, due to the fact that we can place abstracted and composite data elements at the central tier. Also, this is a prime location for a central repository and for the management of application semantics, perhaps using standards such as the Semantic Web.

This is not to say that all SOAs will require a central data tier, but it may be a good idea for some. Again, you have to consider your own requirements. Common requirement patterns to watch for include the need to control metadata, state management, heavy database process by the composites, and security issues. The data may reside in any data storage mechanism such as a relational database, object, XML database, or through an abstraction layer. The choice is determined by the requirements within your SOA, including accommodation of existing legacy systems and schema management.

Posted by Dave Linthicum on February 2, 2007 06:20 AM

REAL WORLD SOA PODCAST

Listen to the latest podcast:

MP3 •