Vendor Accent

Open Source Business Intelligence

Satish Joshi talks about how Open Source BI can help companies cut costs

Like most of you, I have invested some of my savings in several investment products managed by a particular asset management company. They regularly send me a ton of promotional materials on their various schemes and new products from time to time. More than 90% of what they send me is of no interest to me. In other words, a bulk of the money these companies spend on their marketing campaigns simply goes down the drain. However, it does not have to be this way.

Companies such as these already have in their possession all the relevant data needed to create a unique ‘buyer profile’ answering questions such as ‘what is my risk appetite’, ‘what kinds of investment products do I prefer buying’, ‘what my annual expenditure is’, ‘what months of the year am I most likely to have surplus funds to invest’ and so on. All of this should allow them to send focused information to all of their customers/prospects and cut down on a considerable amount of wasteful expenditure.

This is but a small example that highlights two important issues that I am sure are endemic to most enterprises:

• Easily available data/information is rarely put to any use by companies
• In the event that the information is in fact used, it is wrongly done, which throws up no real benefits

Enterprises survive on their ability to make informed decisions quickly and to do so making use of every grain of available information is as crucial as weeding out the chaff from the grain! In addition, there is technology available to do so quite effectively.

Yet, compared to the investments in technology that fuels the engines of an enterprise, investments in Business Intelligence are disproportionately small.

From one point of view, this is understandable. The return on a technology investment that helps drive the efficiency of a production line up by say 10%, is more directly and immediately measureable. On the other hand, measuring the impact of an investment in Business Intelligence that forecasts market trends, competitor behavior or customer satisfaction etc. can be more nebulous and difficult to quantify and has a long period of gestation before the returns are realized.

Moreover, the commercially available technology and tools for building Business Intelligence applications are anything but cheap. Licenses alone come in the range of hundreds of thousands of dollars and similar investments may be needed in building applications using these tools making the need to prove good returns on the investment all the more critical.

However, what if there was a way to dip your toes to test the waters before plunging in, to reduce the upfront investment in BI and scale them up in tune with demonstrated benefits?

Enter Open Source BI

BI is not a single, monolithic application, or for that matter, not even a single suite of tightly integrated applications like an ERP platform such as SAP. Tools and development platforms for building BI applications belong to many different categories including Data extraction, transformation and loading (ETL tools), Data warehousing platforms, Data cleansing and quality assurance, Data Analytics, Data mining and discovery, Reporting and presentation etc.

These tools are used to extract, reorganize and analyze massive transactional data generated and captured by various enterprise applications and then to glean insights from that analysis to fuel decision-making, identify trends, and predict future problems so that an organization can respond with agility and foresight.

Recently, many products have emerged in the open source world spanning most of the categories mentioned above that can be deployed to create robust and functionally rich BI applications. For example, the Pentaho Open BI suite that provides tools for ETL (e.g. Kettle), OLAP (e.g. Mondrian, Jpivot), Mining, Reporting, Dashboard etc. These also integrate well with open source databases such MySQL or Hypersonic and open source Web Server/application server platforms such as JBoss or Apache. There are data quality tools like OpenDQ from Infosolve, ETL and OLAP tools like JasperETL and JasperAnalysis, Reporting tools like BIRT, data warehousing products like PostgreSQL. Ingres one of the original Relational database servers that dominated the landscape decades ago, has also become open source. The problem is not of finding the right tool to address a specific need in BI application development. Rather it lies in making the right choice from amongst a plethora of tools available across the entire spectrum of the BI technology stack!

While Open Source Business Intelligence (OSBI) tools come at a fraction of the price (sometimes as low as 15%) of commercially available standard BI software, what is more important is the flexibility many of them have started offering. For example by providing a subscription model which allows companies to start small and then scale up the investments by building on the small and early, but successful use of BI techniques. These pay-by-the-drink models are especially important in the economic climate we are in where capital available for upfront investment is scarce and every CIO’s budget across the globe is being slashed to bare bones.

However, two other considerations are as important as the financial aspects:

• The maturity and richness of features that Open source BI tools have reached in the last couple of years – making it possible and safe to take the chance of deploying them for enterprise class applications and
• The rapid evolution of these tools because of the community collaboration effort that surround Open Source development in general

To be sure, the community collaboration can be a double-edged sword at times leading to problems in quality and stability of the product, addition of meaningless features that serve no particular purpose etc. Therefore, one needs to be extra careful and cautious when selecting open source products for building enterprise class applications. In addition, since most open source software is the outcome of the combined (many times accidental) efforts of many different people essentially working without a single coherent and synchronized plan it is difficult to forecast how an open source product will evolve in the future. Therefore for areas like BI where many different tools obtained from multiple sources need to be used in tandem to build a well-integrated application that serves a defined business goal, using diverse open source tools can be challenging.

Given the dire need of enterprises to utilize all their information assets most productively to survive and grow, and to do so especially in the current challenging environment, without large capital investments of questionably ROI, Open source BI alternatives provide an attractive and low risk option.

The author is EVP, Patni