I'm going to tackle SOA and data integration this week due to the number of e-mails I've been receiving on the topic. I did indeed address it in my Podcast this week, but I would like to hit this topic in my blog as well, perhaps provide some more light.

First, the history. Data integration is the name the vendors have adopted to replace ETL (Extract Translate Load), Data Cleansing, and Data Warehousing tools of days gone by. These tools actually pre-date the notion of EAI, and were really the first sets of technology designed to deal with data and the usage of that data for decision support (business intelligence now). They would extract large amount of data from a single or several operational data stores, clean the data, roll up the data, and place it in another data store, typically the data mart or data warehouse for analysis. From there somebody would "mine" the data to extract relevant information, such as productivity over time, sales effectiveness over years, profit by division, you get the idea. Very powerful notion for its time, and very powerful technology today.

When I first was working on the notion of EAI, I did indeed use these tools, and thought they had value. However, I also understood that the value of integration was really about movement of information in real time, system to system, in patterns that resemble actually business processing (sales to inventory to accounting, etc.), and while there was some business intelligence there it was really in the domain of real time monitoring (BAM). Thus, you really had two different threads of technology that was born...ETL (now data integration) and EAI (now integration or SOA).

Enter SOA, and the hype around it, and everyone looking to link to it. All data integration vendors, and EAI vendors for that matter, are repositioning, retooling, and remarketing their technology as a "SOA solution." So, where is the actual fit for data integration?

I'll hit that tomorrow, or Wednesday.

Posted by Dave Linthicum on July 10, 2006 05:20 AM