Whipping IT data into shape

Publishing those transformations can help later services understand the context of the data they are working with, IDC’s Morris says. But that can also flood the system with very large data files and slow down each service. IT needs to consider carefully how much context is passed through as aggregated parameters versus limiting that metadata and having the service interface look for exceptions, Kanbay’s Shah says.

Free IT resource TechNet: More ways to know it, share it, and keep it running. Sponsored by Microsoft Free IT resource Attend the SOA Executive Forum: Breaking SOA Bottlenecks SOAExecForum.com/may2007 Sponsored by InfoWorld Return to special report DOWNLOAD PDF Click here to download InfoWorld's special report Managing data

The return of master data
The rise of SOA has given vendors reason to revisit their tools to simplify data management, for both SOA and non-SOA environments. Many are now promoting MDM (master data management) tools to help ensure that applications or services use only correct, current data in the correct context. “Master data” incorporates not only the data itself but attributes, semantics, and context (metadata) needed to understand its meaning for proper use by a variety of systems. (Some vendors call these systems enterprise information integration, or EII, tools.)

Although not new, the concept was largely relegated to after-the-fact data systems such as data warehouses and business intelligence, notes Bill Swanton, research director at AMR Research. Before SOA, enterprises could largely get away without worrying about master data because most information resided in application suites, where the vendors had at least an implicit, internal master data architecture in place. IT could thus focus just on transmitting processed or raw data between application suites -- by creating connectors -- and allowing the applications to handle most of the contextual issues, he notes.

SOA’s many-to-many architecture no longer allows IT to leave the problem to application vendors and to limited integration conduits. Even non-SOA environments, though, benefit from moving from the one-off approach of creating connectors to a more ration-alized data architecture that makes integration simpler, Swanton says.

Some providers -- including IBM, Informatica, Oracle, and Siperian -- approach the issue as an operational data warehouse, providing one or more data hubs that services access both from stores of cleansed data and from services that generate validated data from other applications as a trusted broker. These emulate the hub-and-spoke architecture common to traditional enterprise environments. Others -- such as BEA Systems, i2 Technologies, and Xcalia -- approach the issue at a more federated level to better mirror the loosely coupled, abstracted nature of an SOA.

Analysts and consultants warn that today’s technology is very immature and at best can help only specific data management processes. “There is no silver bullet,” says Shawn McIntosh, senior manager at consultancy AGSI. For example, Starwood’s Park notes that his IT group is hopeful that IBM’s planned Systems Integration Bus will provide a way to manage the data services in the hotelier’s SOA. “But we can’t wait for the tools to come out,” he says.

Many of the data hubs currently offered are geared to one data subject, such as customer or product information. That’s fine as an initial building block; later, however, IT will have to generalize the hub or work with a federation of specific data hubs, says Satish Krishnaswamy, senior director of MDM business at i2. “We won’t ever get to one single hub, so IT should instead work toward a standard canonical [hierarchical] view” of data across its sources, IDC’s Morris says.