Real-time Business Intelligence How Real Should We Get

Updated: Nov 4 2003, 05:30am hrs
Real-time Business Intelligence (BI) is the new buzzword in the data warehousing industry since the past few months. More so, because it gives a sense of power to the user in taking split-second decisions and a feeling of being in complete control of the situation. This, perhaps, is the psychology that makes anything real-time somewhat easier to sell.

This does not mean that we do not need to make decisions in real-time, but to take a call on how real the real-time should be. This is simply because the cost of implementing a BI solution is somewhat directly proportional to the real-time nature of information.

In fact, justification of investment in BI and data warehousing itself is a complex exercise because the ROI (Return On Investment) depends not on the intelligence generated by the solution but on the action the user takes on the information provided by the system. Hence, it is not enough to just produce intelligence, it is also essential to ensure action, which can make the BI solution a success or a failure. Hence, any addition to the already complex set of justifying parameters for a BI solution must precede with caution.

In a traditional BI system, frequency of extracting information would be anywhere from weekly to daily. And if the the frequency of extraction can be reduced from daily/weekly to hourly/every minute or second, the system can be stated to be functioning in real-time. However, this is easier said then done. There are various methodologies suggesting how this can be done and one such methodology suggests that we should create one big data warehouse which will fulfill all the requirements of tactical queries, historical analysis as well as get populated from the OLTP (Online Transaction Processing) system instantly.

There is no doubt that current technology, such as Enterprise Application Integration (EAI) tools, can support pulling data from the OLTP systems in real-time and pump it in large data warehouses. However, the problems associated with such real-time implementations are complex and the solution can be frightfully expensive.

Financials apart, the technical difficulties in building and maintaining real-time BI solutions can be far more challenging then we anticipate. Users find it difficult to maintain terabytes of data which are stored in different databases separately. Imagine the problems we will face if we combine all of them in one massive data store. The OLTP systems are expected to perform at very high performance to ensure that operational efficiency can be maintained. If we burden the system to update the ODS on real-time, performance degradation will occur which may affect the business so badly that the need for BI itself may become irrelevant, let alone the real-time part of it. Also, the concept where vertical-centric data marts enable departmental decision-making on a subset of data, making it extremely efficient to operate and easy to scale, may have to be abandoned in favour of real-time information availability.

The argument advanced in favour of real-time BI could be that in tough market conditions, the need for organisations to be aware of, and have the ability to react to, changing business conditions by examining all sources of information as soon as possible is ever so high; hence, the necessity of instant gratification by way of real-time BI. However, what is necessary is to have the ability to react correctly, rather than instantaneously, in most of the situations. And organisations that falter in the correctness of decision-making may be forced into a false sense of illusion that instantaneous leads to efficient decision-making.

This is not to state that real-time BI is a term coined by vendors to make the offering more attractive, but to emphasise the need to be cautious before one embarks on the real-time journey towards decision-making. It will be prudent her to examine what are the decisions that will impact the business.

It is true that the business owners are alerted when an event occurs which requires their attention but the critical issue is what should be the spontaneity We will observe that that there are not too many situations where the decision will need to be taken on a real-time basis.

For example, if your supplier logs in orders every evening in his system, it does not make sense for you to monitor the inventory level on a real-time basis and generate the Purchase Order as soon as the re-order level has been reached. Hence, a pragmatic approach towards real-time BI will be to isolate KPIs, which warrants immediate attention, and build the real-time solution only for those, rather than jumping on to the generic real-time BI bandwagon.

The author is general manager, products and technology group, Patni Computers