Informatica Stays with Its Strengths in Data Integration

At Informatica’s recent industry analyst summit, Chris Boorman, the company’s chief marketing officer, opened the event by describing Informatica as expanding beyond its core offering in data integration in a broader sense. He compared this growth to Amazon expanding from being an online bookseller to offering computing resources via Amazon Web Services. I see it almost the opposite way. Informatica has always been in the data integration business. It has excelled at making this area of IT more relevant and more applicable to broader audiences. My colleague described their latest efforts to focus on line of business users in a recent post. My purpose here is to review some of the highlights of the company’s latest product releases.

The focus of the soon to be released version of its flagship product, Informatica 9.1, is to consolidate a range of offerings into a single platform. This consolidation includes the master data management (MDM) products acquired from Siperian during 2010, which we commented on. We’ve seen similar efforts from other data integration vendors, for example, the consolidation of the IBM InfoSphere platform.

Informatica 9.1 includes a laundry list of information management capabilities, starting with data integration and continuing with quality, profiling, governance, MDM, federation and virtualization, life-cycle management, event processing and low-latency messaging. So Informatica can be a one-stop shop for a complete portfolio of data integration tools. It also has aggressively embraced cloud computing, providing access to cloud–based data sources and offering its products as services in the cloud. As more enterprise data resides in the cloud, a hybrid configuration likely will become more appealing. We’ll have more data on this subject soon as we are beginning benchmark research on Business Data and the Cloud benchmark research program.

It’s impossible to cover a product line as broad as Informatica’s in a single blog post. For example, this vendor not only provides core data integration, data quality and data profiling capabilities but also has a dedicated product for information life-cycle management that includes data archiving and test-data management. It offers data virtualization and federation capabilities via what it calls data services. And the list goes on. For the moment then, I’ll point out what caught my attention, setting aside Informatica’s impressive ability to execute and continue to expand its presence in the market.

Informatica has invested heavily in the technology acquired from Siperian. As a result, MDM is now a key part of Informatica’s platform – in fact MDM seems to be everywhere in it. While many vendors have limited or specialized their MDM offerings to a customer-related data domains, Informatica’s works well across multiple domains. At the analyst conference presenters shared MDM examples ranging from traditional customer data to aircraft engine specifications to seed catalogs. (However, despite the support for many domains, Informatica does not provide a prebuilt product information management (PIM) application such as those covered in our PIM Value Index for 2011 though they support product data integration. Informatica’s MDM approach provides both repository-based and registry-based hub architectures via a single product, which ensures consistency between the two. In version 9 you can also change data models and rules while the system is running to produce an environment with no downtime. The service-oriented architecture (SOA) of Informatica also comes through in a variety of ways. Informatica created data controls to embed data quality and MDM capabilities in other applications, demonstrating at the conference Informatica functionality embedded within Excel and salesforce.com for true self-service capabilities. Presenters also demonstrated packaging of Informatica activities as a Web service – that goes beyond the needs of most end users, but a skilled technical resource could embed Informatica in virtually any application.

On the other side of the equation, Informatica supports a variety of new data sources via its Data Services offering. These data services can be used for virtualization and federation similar to those of other vendors such as Composite Software or Denodo, but Informatica can also apply other features from its technology stack such as data quality to its data sources, in effect, creating “data quality in flight.” If your organization relies on virtualization and federation, you can use these features to ensure that the quality of that data matches the standards you have set for the rest of your data. Data services also provide another powerful capability. When you define a logical data source, it can include a variety of other steps (such as data quality and data governance) in the definition of the data source. So even if you don’t plan to use virtualized or federated data sources, you should still look into data services as a way to associate all the necessary information management processes with the data sources you make available to users in your organization.

In event processing, Informatica has a bit of split personality. It has both a complex event processing (CEP) engine acquired from Agent Logic and a low-latency messaging infrastructure acquired from 29 West. What the company refers to as CEP is in my opinion more similar to a rules engine as opposed to a product like Streambase – in fact the product is called RulePoint. It is designed for complex rule structures but not necessarily very low latency data such as stock quotes. That’s where the 29 West product comes in that we covered at time of acquisition. Informatica Ultra Messaging provides a messaging infrastructure that can deal with very low latency data – measured in microseconds – but does not provide the same robust rule capabilities. I expect we’ll see more convergence between these two products over time if Informatica intends to be competitive in the CEP space and what we call Operational Intelligence. However, for most organizations that don’t need to process ultra low latency data, RulePoint is probably sufficient.

Cloud computing continues to play a big role for Informatica. It bet early and bet big on the cloud, as mentioned in my previous blog post “Clouds are Raining Corporate Data”.

I arrived at the event wondering how Informatica could sustain its growth and remain so strongly competitive. I expected to hear of plans for acquisitions and expansion into other aspects of business intelligence or data warehousing. I left with the impression that Informatica not only has thrived in its core market but also has found ways to expand its product line and broaden its addressable market within the data integration segment.

Regards,

David Menninger – VP & Research Director

Kognitio Pablo Brings MDX to Data and Analytics

Kognitio announced the addition of MultiDimensional eXpressions (MDX) capabilities for its WX2 product line. John Thompson, CEO of U.S. operations, and Sean Jackson, VP of marketing, shared some of the details with me recently. I find the marriage of MDX and large-scale data both technically challenging and potentially valuable to the market.

Kognitio develops and markets a massively parallel processing (MPP) database system targeted at the business intelligence and data warehousing market for use as an analytical database. WX2 is available as on-premises software, an appliance or software as a service (SaaS). The product has a relatively long history, having been developed in the early 1990s as part of Whitecross Systems, which merged with Kognitio in 2005. Headquartered in the U.K. the company is making a push for a bigger presence in the U.S. using funds raised for the purpose in December.

MDX provides more powerful ways than SQL to express relationships between data elements that are organized into cubes for online analytical processing (OLAP) analysis. Although a number of vendors have implemented OLAP capabilities on top of relational databases (referred to as ROLAP), it’s difficult to create meaningful sets of derived data using this approach. MDX makes it easier to express formulaic relationships between different data elements, thus enabling organizations to create relatively sophisticated historical or prospective measures to assess or project the performance of their business. To illustrate the difference, with SQL you could project sales as a percentage of last year’s sales, but with MDX you could project sales based on last year’s sales along with this year’s advertising programs and hiring plans and include a projected ramp-up period for each sales person. Using MDX it also is much easier to create what-if and planning types of analyses. Our benchmark research shows that only 22% of organizations currently can conduct what-if analysis for planning and forecasting, but 84% said it is important or very important to add these capabilities for decision-making and performance management.

Let me note that providing MDX or other what-if analyses over large amounts of data presents some significant technical challenges; these are mostly based around performance issues, although the amount of memory required for the analyses can be another issue. Because MDX makes it easier to express complex formulas, any calculation could reference any other data point in the entire data set. The challenge lies in working through the calculation dependencies and getting the necessary data into memory quickly to perform the calculations and deliver the results. As I pointed out in the MPP blog post referenced above, it becomes even more challenging when the data can reside on another node in the MPP system.

Kognitio will introduce one of the first products to combine MPP and MDX capabilities. SAP also provides MDX capabilities for its high-performance analytic appliance (SAP HANA), which my colleague commented on. As more and more data gets stored and analyzed using MPP systems, the MDX capabilities can help organizations produce more accurate analyses of where their business is headed and thus make better business decisions. The alternative today is to operate on subsets of data, which can potentially reduce the accuracy of the results or can lead to more complicated systems that attempt to combine results from multiple separate analyses. The MDX capabilities of Kognitio’s WX2, referred to as Pablo, are not a separate product but are built into it. The company touts these features as helping with the process of producing and managing analytic cubes. While these capabilities will help with that process by maintaining the cubes directly within WX2, I see as much potential value in being able to do more meaningful analyses over larger amounts of data.

These new MDX capabilities are scheduled for availability in June. Initially at least they come with some caveats. If you are evaluating Kognitio, be aware that the initial target client tool is Excel pivot tables. You’ll have to wait to use another BI front-end tool. I also advise you to explore what subset of MDX is supported to make sure the expressions that are critical to your business can be included in your models. Also, Kognitio provides no inherent mechanism to process updates for purposes of what-if analysis, but the company claims you can update the underlying relational data using other mechanisms and the cubes will be recalculated automatically. Finally the biggest caveat will be performance. In addition to assessing the overall performance of the new system vs. existing OLAP systems, you will want to see how much memory is required for the cubes and what happens when the system exceeds the available amount of physical memory. The litmus test will be to define a cube that is larger than the memory on one of the nodes and see whether it continues to perform adequately.

Regardless of whether Kognitio gets it right in the first release, it’s encouraging to see vendors advancing the types of analytics that can be performed on large data. I expect others to follow suit, and that will be good for business users who need to perform planning and what-if analysis on ever larger amounts of data.

Let me know your thoughts or come and collaborate with me on  Facebook, LinkedIn and  Twitter .

Regards,

David Menninger