"A strategic Business Intelligence platform puts the right information in the right hands at the right time, and gives managers and executives the ability to test various scenarios for business spending and investments while monitoring important operational drivers of company performance."
The Importance of Business Intellience
This article provides a primer on business intelligence (BI) and serves as an introduction to the concepts used throughout the rest of the articles in this issue of the OSBR. We define BI, discuss the components of a BI solution, explain the types of BI tools and provide a brief overview of the evolution of BI.
What is Business Intelligence?
Hans Peter Luhn first defined BI in his article "A Business Intelligence System", published in the October 1958 IBM Journal. More recently, Howard Dresner of the Gartner Group popularized the following definition for BI: An umbrella term to describe "concepts and methods to improve business decision making by using fact-based support systems."
BI, as a practice, has really come to the forefront in the last fifteen years. It is primarily used as a way for organizations to make sense of the mountains of data that have become available to them due to the proliferation of inexpensive storage and the ability to collect large amounts of data via multiple input sites.
What Makes Up a Business Intelligence Solution?
BI solutions grew out of a combination of increased globalization, competition, and pervasiveness of information systems.
The original BI systems were little more than copies of transactional databases. These allowed reports, developed by system programmers, to be run against the data without impacting the performance of production systems.
However, any report changes or new reports required intervention by the information technology group, since the data models used by transactional systems were highly complex and often cryptic. They often utilized space saving techniques to reduce the data storage. On top of that, it was pretty much impossible to combine information from separate systems.
The need for current and complete data to facilitate business decisions, the proliferation of systems and data, and a desire to drive the data to the business user has resulted in the explosion of BI over the past decade.
So, what components are required to build a BI solution? A traditional approach to BI is usually comprised of three major components as shown in Figure 1.
Figure 1: Components of a a Business Intelligence Solution
1. Data Storage
Organizations typically store the extracted and transformed data in a separate database distinct from the production systems. This insulates the production systems from any ‘run away’ queries that could be generated by users. It also allows the organization to control updates to the data since there is often a desire to ensure that everyone reports from the same time series.
2. Extract/Transform/Load
Extract/Transform/Load (ETL) describes the process and tools whereby data is extracted from source or transactional systems. Typically, data is extracted and changed or transformed prior to loading into the reporting database which is more commonly referred to as a Data Warehouse or Data Mart. The transformation is meant to accomplish the following:
Provide data in a consistent format. The same data may be represented differently depending upon the source system. Gender may be represented as Male/Female in one system, M/F in another, and even 1/2 in a third system. The ETL process supports the transformation of data to a consistent format.
Transform data to a format easily understood by the business. Transactional systems are designed to process information as quickly as possible. As a result, the relational data models employed are often very complex and difficult to understand, even for seasoned database programmers. A Dimensional Data Model or Star Schema Model is typically employed in order to represent data in a way that is both easily understood by the business and that expedites database access.
Data may be enriched or corrected when transformed. For example, integration of multiple customer records to a single customer record may be undertaken, and rules may be applied to ensure that the data is correct. Customer records may be segregated by region or area based on the zip or postal code. This correction step is now being eschewed in favour of correcting data at the source level, as the corrective activities themselves can build in incorrect assumptions regarding the data.
3. Reporting and Analysis Tools
There is a plethora of BI tools available for the surfacing and analysis of data. Most organizations will utilize multiple tools as there isn’t a ‘one size fits all’ BI tool. To describe the tool, we segregate their users into categories:
1. Consumers: typically use delivered reports or dashboards that provide summarized information presented on a web page. If further analysis is required, another member of the organization undertakes the work. Consumers are often senior executives within an organization who don't have the cycles to do the analysis and, more importantly, have individuals who can undertake the analysis. Open source software (OSS) offerings include BIRT, Pentaho and Jaspersoft. MicroStrategy, Business Objects, and Cognos provide commercial BI software.
2. Analysts: often utilize tools that allow a structured review of the data. These include configurable reports and queries or the use of data cubes, often referred to as Online Analytical Processing (OLAP), to slice and dice the data and to drill-through to details. In addition to the commercial leaders listed above, Wabit, Pentaho, and Jaspersoft offer OSS products..
3. Explorers: undertake detailed analysis of data. They often use ad-hoc reporting tools or program their own queries using Structured Query Language (SQL). Additionally, some users may utilize data mining tools to undertake advanced statistical analysis of data to drive out patterns and trends. OSS data mining tools include Pentaho's Weka and R products. Base SAS and Enterprise Mine from SAS, Clementine from SPSS, and IBM are among the commercial leaders in this area.
The Evolution of BI Solutions
Originally, specialized offerings were developed by organizations to provide point specific BI functionality. Over time, larger commercial software vendors acquired these offerings in order to provide fully integrated ‘across the board’ solutions. Examples include SAP’s acquisition of Business Objects, IBM’s acquisition of Cognos, and Oracle's acquisition of BRIO and Hyperion. Each of these organizations in turn swallowed up other best of breed solutions prior to being acquired.
This consolidation has meant that organizations looking to implement BI solutions are faced with the choice of purchasing a very expensive consolidated solution from a large vendor rather than acquiring best of breed tools. Open source solutions offer organizations the opportunity to implement best of breed BI Solutions that offer everything the commercial products have in a cost effective and as-needed manner. Why? Because the freely available nature of open source code means that the community develops the interoperability to published standards, and the OSS model eliminates lock-in from one vendor.
Recommended Resources