Whipping IT data into shape

Many enterprises have avoided such data architecture efforts because there’s no obvious ROI, notes Common Sense’s DePalma. Some remember earlier-generation efforts such as custom data dictionary creation, which also involved understanding the organization’s data architecture; by the time they were completed, they were already outdated. Fortunately, IT can approach the data understanding incrementally, creating the rules and metadata around the information used for specific applications’ or services’ needs, says Marcia Kaufman, partner at Hurwitz Group. Over time, the enterprise will build up a complete data architecture. “It’s a long-term journey,” says Hurwitz.

Free IT resource TechNet: More ways to know it, share it, and keep it running. Sponsored by Microsoft Free IT resource Attend the SOA Executive Forum: Breaking SOA Bottlenecks SOAExecForum.com/may2007 Sponsored by InfoWorld Return to special report DOWNLOAD PDF Click here to download InfoWorld's special report Managing data

That data architecture will typically include multiple data models, each oriented to a specific type of subject or process, notes Paul Patrick, chief architect at BEA Systems. That actually helps IT by allowing the data architecture to be developed in stages, plus it limits the mapping required between data models. (A unified data model must account for all possible mappings, whereas a federated model does not.)

Furthermore, IT should concentrate on dealing with exceptions, says ZapThink’s Schmelzer. For example, IT should develop services that check for data that are out of normal bounds, rather than try to develop an enterprisewide ontology that maps out every possible state or relationship, he says.

Ultimately, the enterprise should build up layers of data services in which master data is distributed, says William McKnight, senior vice president of data warehousing at consultancy Conversion Services International, although the infrastructure and tools to deliver on this goal aren’t yet mature.

Roll up your sleeves
Provisioning data sources as services across an organization is a monster undertaking. For a traditional integration effort, it means understanding the context within each application and how data is transformed for delivery to other apps. For an SOA, it requires understanding the multiple relationships and dependencies data can have with various business processes. “There are so many variables here,” notes Common Sense’s DePalma.

Analysts and consultants agree that this complexity requires both an upfront investment in modeling data architecture and an ongoing effort to systematically think through data dependencies and context. Discovering the data models and relationships among your systems to create the mappings is about 70 percent of the effort in an SOA’s data architecture, says IDC’s Morris. At GGB, IT director Kenngott said the modeling and discovery effort was about 30 percent of the data-integration effort within its ERP consolidation project.

That initial push is well worth it, argues Starwood’s Park. “Otherwise, you can get pretty far along with your project and discover that you have 10 fields that you don’t need, 10 that you do but didn’t know when you designed the service, and five that are different than you thought. When you have a complex system with hundreds of services, these interfaces have to be nailed down.”

In most organizations, the tough slog of codifying interfaces and reconciling distributed data models is long overdue. But today, with the majority of large organizations pushing ahead with some sort of SOA initiative, the natural inclination to avoid this ugliest of hairballs can no longer be sustained. “The problem is too big to sweep under the rug any more,” Conversion Service’s McKnight says.