Information Management Technology Revolution and Research Agenda for 2012

In our definition, information management encompasses the acquisition, organization, dissemination and use of information by organizations to create and enhance business value. Effective information management ensures optimal access, relevance, timeliness, quality and security of this data with the aim to improve organizational performance. This goal is not easily met, especially as organizations acquire ever more data at an ever faster pace. In our business analytics benchmark research of more than 2,600 organizations, almost half (45%) have to integrate six or more types of data in their analyses. More than two-thirds reported that they spend more time preparing data than analyzing it. To assist in dealing with these sorts of issues and others, we’ve laid out an ambitious information management research agenda for 2012.

In recent years the complexity of information management has risen dramatically. The volume of information being processed has increased exponentially and so have the challenges of ensuring consistency and quality and managing governance and the information life cycle. New data types and sources such as comments on social media have emerged and must be integrated into an organization’s information assets. Moreover, in many cases the boundaries between organizations and the outside world with which they interact have become far less distinct, leading to the need for a more expansive understanding of information management. Our Business Data in the Cloud research shows that data is seldom stored in only one repository; the majority of organizations (86%) need to bring together cloud-based data and on-premises data.

We will provide new insights on the dynamics of the information management market as we complete research on Information Management Trends. This research will illuminate the priorities organizations place on data quality, master data management and data governance. It will also explore ways in which organizations are incorporating virtualization and replication for broader and faster data access. The growing volumes and sources of data will require data integration that can help facilitate better linkages across IT and into business. We will assess the vendors and products in a Value Index for Data Integration that will determine what suppliers can be best fit for your enterprise.

Our research will also help organizations facilitate adoption of and use of big-data technologies. Our recently published Big Data research highlights the role of various technology alternatives for managing data on a large scale. More than 80 percent of organizations utilize more than one technology to tackle their big-data challenges, but organizations lack maturity when incorporating these data sources.  Specifically, our research shows that business have not adapted many of their standard processes to deal with big data. We’ll follow up this research by looking at specific vendor capabilities and how they can help extend information management processes to support big data.

Data is increasing not only in volume but in velocity as well – the speed with which data is generated and communicated. Technological developments such as smart meters, RFID, sensors and embedded computing devices for environmental monitoring, surveillance and other purposes are creating demand for tools that can derive insights from huge, continuous streams of event data coming into systems in real time. Traditional database systems are geared to manage discrete sets of data for standard BI queries, but event streams from sources such as sensing devices typically are continuous and their analysis requires different kinds of tools that enable users to understand causality, patterns, time relationships and other complex factors. These requirements have led to innovations in complex event processing, event stream processing, event modeling, visualization and analytics. We’ll be exploring how organizations can capitalize on real-time data collection and analysis in our benchmark research on operational intelligence and complex event processing. We will also assess vendors and products in a Value Index to determine the value of vendor offerings in Operational Intelligence to harvest the events from these streams of data.

Information management continues to be a strategic business imperative. It can help organizations improve their understanding and use of enterprise information and to establish governance of it. To accomplish these aims they must manage the flow of information throughout the full life cycle of data and provide proper data stewardship to support the business while minimizing risk. We need to better use the information through a simpler means of being able to assemble and deploy it to those in business who might even want to receive it through mobile technologies. This is what we call information applications that can help in timely access to information and should be coupled with an information management discipline. Our research will deliver education and best practices that can help you understand how to reduce the costs, time and risk of delivering these capabilities to your organization.

It will be a big year for information management in the forms of technology but also the methods and processes for which to manage and utilize the full value of it within organizations. I look forward to connecting with all of you on LinkedIn or following me on Twitter.

Regards,

David Menninger – VP & Research Director

The Information Management Technology Revolution in 2011

The information management (IM) technology market is undergoing a revolution similar to the one in the business intelligence (BI) market. We define information management as the acquisition, organization, control and use of information to create and enhance business value. It is a necessary ingredient of successful BI implementations, and while some vendors such as IBM, Information Builders, Pentaho and SAP are in addition integrating their BI and IM offerings, each discipline involves different aspects of the use of information and will require it sometimes integrated and sometimes separate.

Some might consider information management as the “plumbing” behind BI. They take it for granted and only notice when it is missing. We have a more holistic view. Our recent benchmark research on business analytics shows that, for example, organizations struggle to collect all the data they need, with two-thirds of them stating they spend more time in data related activities than analytic ones.

Three key issues are driving our information research agenda in 2011:

1) Combining all the sources and types of data into an integrated information architecture.
2) Enabling organizations to manage and analyze larger volumes of information.
3) Providing accessibility to information throughout the organization.

We’ll be updating our information management benchmark research this year to see how these central issues are impacting IM overall. In addition we will focus on five technology innovations my colleague has identified as the business technology revolution in 2011: cloud computing, mobile technologies, social media, analytics of more types over more data and collaboration. Let me flesh out each of these a bit as they impact the evolution of IM.

As I pointed out in “Clouds Are Raining Corporate Data,” cloud computing is having an increasingly large influence over the IT landscape. It’s likely that, whether you realize it or not, corporate data exists and/or is migrating outside the walls of your organization. Cloud-based applications and services raise information management challenges that don’t necessarily exist in on-premises deployments. We’re investigating these issues now in our Business Data in the Cloud benchmark research program and where many new providers dealing with cloud data like Dell Boomi, Jitterbit and Snapdata play into the existing landscape of IBM, Informatica, iWay Software, Oracle, Pervasive and Syncsort to name just a few.

Mobile technologies are enabling organizations to deliver information to users when and where they need it. They are one of the forces driving cloud adoption as organizations look to make it easier to deliver applications to users regardless of their location. Mobile applications also are consuming and producing more location-related information and creating a need to manage this kind of data.

In the world of information management, social media has created entirely new challenges. Most social media data is unstructured text and is forcing organizations to embrace text analytics to deal with it, in many cases for the first time. The volumes of social media data and the speed with which it should be collected and analyzed also present new challenges. From an IM perspective, organizations must learn how to solve these challenges while enforcing appropriate data quality, data governance and life-cycle management policies.

Analytics present an additional set of IM challenges. The necessity of managing more data and different types of data has led to the adoption of large-scale technologies such as Hadoop. In research that is under way now we are researching the various ways of dealing with these huge data volumes and the role of Hadoop in that process. Predictive analytics also create IM challenges. Sampling, which is a key to producing unbiased predictive models, may or may not become less critical as database analytics grow in popularity. The models and the scores that such analytics produce are another form of data that must be managed and retained, often for compliance and auditing purposes. We’ll be studying these issues as part of predictive analytics benchmark research that will commence in the first part of 2011.

Collaboration provides a means to communicate and extend the processes of IM. It creates a new channel not only for delivery of information but also for input into the delivery process. Using collaboration tools such as Twitter, Chatter  or Tibbr can help organizations use data and related information by involving more people. This wider audience collectively contains more knowledge about the underlying data and can also comment on its quality, which ultimately will lead to better data and more trust in it. Collaboration tools also provide a mechanism to link the workflows of information management with the constituents involved in the process.

Information management continues to evolve and grow, somewhat to my surprise as indicated in my recent assessment of Informatica. These changes present challenges for IT groups and lines of business alike. With our IM research agenda, we hope to provide useful information to both functions and help you navigate together through this changing landscape and achieve the goal of creating and enhancing business value.

Regards,

David Menninger – VP & Research Director