Comparing hype with reality is a new mini-series that will be featured within Tech Forum from now on. This topic has been on my mind for a long time—more than two years. Finally, I thought the time is now ripe to talk about it. As all of you know, the information technology industry is a fast-paced industry. It has its advantages and disadvantages. There are too many technologies being introduced/upgraded at too fast a pace.

This pace is not by design. It has reached this level because all vendors are trying to catch up with their own customers’ wish lists as well as with competitors’ product features. The base thought behind it is to make a given product or technology the best when compared with competition.

The problem is, being the best is just a state for a particular short period. This does not last long. Someone overtakes the vendor and the rat race continues. While all this is happening, what’s required is customer mindshare about adoption of a particular technology or tool, for it to succeed commercially. To achieve this, media, events, Web, books and all possible methods of promotion are used incessantly by vendors. The scale and visibility of such promotional activities depends upon the promotional expenses available with the vendor. Bigger the money, more the visibility. Due to this, only a few products or concepts end up being more attractive and visible than others. These excessively media savvy products or concepts eventually turn into HYPE.

Hype is required for vendors to sell products. But what happens to us IT professionals? What about end users? What about non-technical decision-makers? How does this hype affect them? I have been keenly observing and analysing these areas. Before we go into details and examples, here is the conclusion:

There is a huge gap between what the HYPE PROMISED and what was actually achieved in REALITY.

This is probably known to many of us, either consciously or subconsciously. But this is not all. The conclusion mentioned above may be drastic, but the bigger issue is that none of us are actively concerned about this gap. The purpose of this mini-series is to highlight these gaps and find solutions to address them. Sounds like too much philosophy? Not at all! Here are some live examples.

Business Intelligence Hype
The hype surrounding Business Intelligence (BI) is great. It aims to provide ad-hoc reporting, drill-downs and drill-throughs, user-defined pivot tables, data warehouse integration, filtering, heterogeneous source data repository, centralised security and so on. The typical cost could be anything from a few lakh rupees to a few crores, and really depends on who is selling it and to whom. So much for the hype.

Almost all established database vendors are into this hype as of now, including Oracle, Microsoft and IBM. There are a variety of third party tools including Business Objects, Cognos and MicroStrategy. In addition there are existing reporting tools and of course the ubiquitous Excel!

Business Intelligence is supposed to revolutionise decision-making by executives—incidentally, they are called Knowledge Workers (please note they are NOT called Knowledgeable Workers!) It is supposed to unearth yet unknown trends and hidden information from all the gigabytes that are currently wasting disk space in your enterprise. Great promise, sounds good, feels good, users feel empowered to have all the tools and tricks in their hand. So much for the hype. Now let us look at the reality.

Reality
I had to describe the hype because it does not possess an objective definition or manifestation. Hype is a state of wellbeing by virtue of thinking that we are using some great technology. Reality, however, is easily stated in a much more definite manner.

Note:

• The reality may be shocking. Please read it at your own risk. While reading it, be honest to yourself and assess whether you are facing the problems listed below.
• The purpose of the list given below is NOT to discourage everyone and generally moan about the state of affairs. There is a very positive thought behind it. But that will come later.

Here is reality as a bulleted list.

1. Few people at the end-user level—oops, ‘knowledge worker’ level—understand the difference between reporting and Business Intelligence. Believe me, there is a vast difference.

2. Most databases are designed in an ad-hoc, shabby, patched up, need-based manner. Therefore, the act of trying to make sense requires a substantial amount of ETL (another bit of jargon - Extract, Transform and Load) activities. Most of these are time- and resource-intensive. Moreover, these ETL activities are used to patch up the lacunae in existing design. Therefore, the real value of well-designed base data never gets unearthed.

3. Most users of traditional reports do not utilise all the data that is available in the reports. They simply look at the reports as on-screen versions of their earlier paper-based/Excel-based reports and look for familiar data at familiar places and reach routine conclusions.

4. Availability of more toolbars, drilldowns, hierarchies, timelines, unnecessarily graphical representations of data, ad-hoc reporting and so on inundates rather than empowers people.

5. Only a small fraction of available reports from traditional applications (custom built or packaged products) are ever looked at by relevant personnel on a regular basis, especially to provide proactive decision-making for strategic planning. This problem is further compounded with packaged products, especially ERP systems where literally thousands of reports are available. Beyond a point, nobody has the time or energy to look at all available reports and map these (probably with some customisation) to individual users of the application. The result is that only a few critical ones are used. All the others just remain as menu options.

6. Most reports are replicas of existing manually made reports. No thought is given while automating these reports, to how these could be enhanced, either in terms of content quality, summarisation, ease of interpretation or effective decision making.

7. Most reports are generated for statutory needs or for satisfying those higher-up. As long as that is done, any other purpose that the report could serve is simply ignored.

8. Most often, complex reports are so slow to generate that users are reluctant to use them. This happens especially when the data size grows over time and the system is (needless to say) neither designed nor tested to withstand the real-life load.

9. If a user requests an enhancement to a report or reporting mechanism, or asks for a completely new report, the response time from the IT team or vendor is so sluggish that most users stop asking for changes, unless there is a real fire. This situation creates a general inundation that stunts effective utilisation of information available.

10. Very often information is scattered geographically, and therefore, getting a complete picture is simply not possible at the time it may be most needed.

11. For decentralised data, the frequency of transfer to central sites and consolidation is typically 15 days to one month. In such cases, the data is already very old and all the effects (good or bad) of whatever trends were there within the data, are already history. It is of no use for proactive decision making.

12. Data across applications needs to be put together manually. Within an application there may be many reports available. But the moment you need another column from another application (may be at a different location or running on a different platform or database), the situation becomes unmanageable.

13. The Business Intelligence initiative typically starts with a department and never progresses beyond that. This happens because users in the pilot department did not see enough business value in the so-called ‘Business Intelligence’ project. This, most often, is NOT due to the fact that there is no business value to be accrued. But rather, the methodology, technology, user expectations, implementation or all of the above were not managed in an appropriate manner.

14. Hardly any company can genuinely claim that all the internal users and decision makers who could potentially benefit from better utilisation of captured information are doing so in a consistent manner and further that the information is being exploited to the fullest extent.

15. Many BI projects stop at being mere replicas of standard reports with a few graphs and more toolbar buttons. The utilisation of these additional features is not very well taught to end-users. Therefore, they continue to depend upon the IT team or vendor for further changes.

16. Moreover, I have observed the utility of the information actually reducing after introduction of BI tool. Why? Because people typically find their own, inefficient workarounds and cumbersome manual ways of handling complex reporting on their own. When the environment changes from familiar Excel to some high-tech, inundating tool, their productivity goes down. Again the tool is not to be blamed. It is the problem with the amount and quality of effort that is put in to ensure full utilisation of the technology being deployed.

Solution
As promised earlier, this is not just an open criticism session about BI. If you remember, we started off by taking Business Intelligence as an example to illustrate the gap between hype and reality. But do the hype makers know the reality? Very often not. Even if they know, many of them are only interested in selling their tools. So, as long as the current hype has made many hopeful organisations buy their products, they are happy. They will wake up again only when the newer version is to be sold. How customer centric!

The solution to the BI reality will be given next time. If the problems take one article, the solution will be of equal—if not bigger—size, isn’t it! Anyway, I will not let all my readers suffer with the negative side of reality. There is a positive side also. Most of us use Excel. Within Excel, one of the most popular tools is the pivot table. I will give you some immediate tips in Excel pivot tables, which you can use now to improve your analytical capabilities.

Here is some base sample data for this example. This data shows customer name, sales person name, type of service and amount. Please note that the data is fictitious and provided for illustration only. Only the company names are real! With this base data, let us see what we can analyse. This data is simplistic, but yet shows the possibilities.

Based upon this data, you can make a simple pivot table, which shows you sales person based revenue, potential.

I am sure all of you know how to get to this point. If you don’t, read the Excel help file! But I also know that very few people know much beyond this (No doubt there are exceptions, so please do not feel offended—I am talking in general here).

Let us say, to find out percent contribution of each of these persons to the total revenue, how would most of us go about it? Simple, just add another column next to the totals and put appropriate % formula based upon the total. This will work. But there is a simpler and better way.

What we are trying to do is to change the data from SUM to something else - in this case percentage. So double click on the “Sum of Amount” header button. A dialog appears. In this dialog, notice that there is an Options button. Many of us do not notice buttons and features with titles like “More...”, “Advanced...”, “Options...”, “Customize...”, and so on. This is one of such ignored buttons. Now click on the Options button. The button will feel happy that someone has noticed it.

Now the dialog expands to show more options. There is a combo box called Show Data As. The default is ‘Normal’. Funny. It should have been labelled as Default or Standard. Anyway, open that popup and there you have a treasure of useful stuff. For starters, just choose the second last option % of total. And choose OK. Now your pivot table gives you percentage exactly as you wanted it. Without any manual formulas, formatting or any effort whatsoever.

There are more options in the combo box. It’s all yours to explore and find out how you could analyse your data more effectively. Remember, all it needed was a click on a mundane-looking button called Options. This feature works from Office 97 onwards. You always had the option to click on Options!

Feedback
Your feedback, suggestions, requests for covering specific topics or issues are welcome. Please send feedback to techforum@mediline.co.in

About the Author Dr Nitin Paranjape is the Chairman and MD of Maestros (Mediline). He is a consultant with many organisations, covering appropriate technology utilisation, business application of relevant technology, application architecture and audit as well as knowledge transfer. He has authored more than 650 articles on various technology-related subjects. He can be contacted at nitin@mediline.co.in