Cover Story

Send, receive, share and now… archive

Experts believe that the global market for e-mail archiving solutions will grow at CAGR of 50 percent until 2008. Big things are expected for this technology in India as well. By Faiz Askari

Communications are an indispensable aspect of modern business. This is reflected by the fact that legislation defines e-mail as a form of business communication that has contractual legitimacy, which can be invoked in the course of forensic investigations. Then there are the everyday business applications where you have to find and retrieve past communications. Businesses today need to keep track of enormous volumes of e-mail which can become a logistical quagmire if you don’t have the right management systems in place.

As enterprises today have access to a greater swath of information, the need for management software is a prime concern. Data is exploding requiring significant investments in storage infrastructure. At the same time it must be retained for longer periods of time for compliance or legal discovery.

Many large enterprises are considering deployment of data retention solutions to avoid potential legal liability resulting from employee misconduct, loss of confidential data or inconsistent customer service. Soumitra Agarwal - Director of Marketing, NetApp India says, “Solutions designed to capture, preserve and manage relevant e-mail and attachments are being explored primarily by organisations in financial services, banking, pharmaceuticals, BPO and government. More than regulatory compliance, as is prevalent in North America and Europe, business risk mitigation by avoiding potential legal liability is driving the need to deploy archival solutions in India.”

In terms of e-mail growth, data says that global e-mail volume has increased substantially in the past few years. The number of e-mail messages sent has risen from nine billion per day in 1999 to 56.3 billion per day and it is going to reach 163.4 billion per day in 2007. Even instant messaging has crossed 12 billion per day mark. E-mail is the most critical business communication tool in 93 percent of organisations world-wide. This kind of growth necessitates proper e-mail management strategy.

The growing dependence on e-mail as a mission-critical business tool and the impact of this dependence on a business’ operational efficiency is a sign of the times. Manish Bapat, National Manager, NAS & CAS, EMC India & SAARC, says, “Osterman Research in association with EMC/Legato revealed that 38 percent of organisations have experienced over 50 percent growth in e-mail volume. There has also been a consistent and substantial growth in e-mail storage requirements. 55 percent of organisations have experienced e-mail storage growth of greater than 50 percent during the past two years, while 10 percent of organisations have experienced e-mail storage growth of over 200 percent during this period.”

For the typical enterprise, e-mail has become a critical service that affects nearly every aspect of its operations. Rajendra Dhavale, Consulting Director, CA says, “The challenge has shifted from adoption and implementation to management. The question has changed from “How do you use your e-mail system?” to “How do you keep track of the information in your e-mail system?”

Market on the move

The e-mail archival segment is growing at multiples of the larger aggregate market of content management. This category is starting to mature with consolidation and is being driven by 70 to 80 percent compound growth rate based on the demand for storage management, compliance, and discovery.

In addition to this, Dhavale adds, “Recent research from the Radicati Group, Inc. has shown that a typical corporate e-mail account sends and receives about 10 MB of data per day. Assume a company with a thousand users, that’s an average of 300 GB per month. Corporate e-mail servers are simply not designed to store such massive amounts of data for long periods of time. By 2008, usage is expected to rise to 15.8 MB per user, per day. Up to 70 percent of an organisation’s intellectual property resides within an enterprise messaging system and with increased regulations and tighter enforcement, managing this data is more important than ever.”

According to IDC the worldwide information management for compliance market will cross the $20 billion mark in 2009 growing at a compound annual growth rate of 22 percent through 2005-2009. IDC also expects that e-mail archiving software market will to continue to grow at CAGR of 50 percent up to 2008.

Although Frost & Sullivan has not conducted any market research on the Indian market for e-mail and data archival, Alok Shende, Director, Information and Communication, Technology Practice, Frost & Sullivan, India has observed this market from the analyst’s perspective. Shende says, “E-mail technology has grown manifold in India. It is now been considered as collaborative tool of communications for business. E-mail is no longer considered just as a communication tool. Organisations have acknowledged e-mail as an integral part of knowledge management repository of their business.”

In the case of e-mail and data archival there are some key drivers, Shende says, “There are several issues involved. E-mail traffic has gone up. At the same time it has become important to store e-mail. Among larger enterprises the archival of e-mail is visible because of regulatory compliance. The medium and small business market segments are yet to adopt e-mail archival because they are yet to understand the importance of the growing storage requirements for e-mail.”

Explaining the challenges of this market in India, Shende says, “In terms of enterprise, there is an urgent need for a storage strategy in the enterprise segment. Strategic investment in storage infrastructure leads to better and optimised output.”

Compliance as a Driver

E-mail is one of the acceptable legal documents in a court of law, even a summons from a court could be sent through e-mail. So, any kind of business communication through e-mail has to be stored for its future legal and regulatory impact. For using e-mail as a legal document, one must ensure that its format and content remains unchanged. There should not be any room for changing the content after opening it.

Dhavale says, “Storing, managing, indexing and retrieving intelligently are becoming extremely important. As Indian financial, infrastructure, health and telecom sectors are coming under more regulatory supervision, the need is to invest resources in solutions, independent of storage technology. Indian regulatory laws are increasing the scope of e-mail being used as a form of evidence. Clause 49 of the listing agreement in Indian stock exchanges is one such example.”

Organisations today, in some way or the other, do business with countries like the US. Dhavale adds, “Hence the compliance regulations in the US are applicable. Under the Securities & Exchange Commission (SEC) rules in the US, banks, brokers and dealers have to keep their e-mail and instant messaging documents for three years. Under Sarbanes-Oxley corporate reform measures, all listed firms from July this year will have to follow the same rules.” National Association of Securities Dealers (NASD) is also a key regulatory body.

When it comes to Indian banks, the RBI has indicated a requirement for record retention so that messages, required for business and regulatory reasons, are safely stored and easily retrievable. For government departments or agencies coming under the ambit of the Right To Information Act, 2005, the concerned organisations need to maintain their records, duly catalogued and indexed in a manner and the form which facilitates the right to information under this Act and ensure that all records that are appropriate to be computerised are, within a reasonable time and subject to availability of resources, computerised and connected through a network all over the country on different systems so that access to such records is facilitated.

Regulatory compliance is now one of the biggest business drivers in deploying e-mail management solutions. R Dhamodaran Director, IBM Software Group, IBM India/ South Asia says, “Failing to comply with regulations can have severe, negative impacts on an organisation, costing them thousands if not millions of dollars in fines, worst of all, loss of reputation for noncompliance is driving companies to take compliance seriously. Organisations that adopt and embrace e-mail management solutions will have a greater chance of avoiding legal or regulatory liabilities, enabling them to focus on driving their business goals, rather than putting out potential fires.”

Emphasising on the importance of operational efficiency, Soumitra Agarwal says, “By relieving primary storage systems of data that is not vital or used regularly, archival solutions can help reduce the cost of primary storage, increasing application performance and in some cases availability, and relieving the burden on disaster recovery and backup systems and processes. Certainly, the optimisation of storage investments thanks to data being divided into tiers results in a better ROI, an important metric for any Indian enterprise.”

Status of data retention

Records are one of the most critical elements of any organisation. Therefore, many organisations would need to develop a set of policies governing the care and management of records. These policies would typically dictate the format in which the records are stored, the media on which they are stored, how long the data needs to be retained, and when and how the data should be deleted permanently. While many organisations have adopted enterprise content management (ECM) applications that often help with records management, the vast majority of enterprise data Agarwal acknowledged the role of long-term data retention as a driving element of this market in India. In the past, archives had short shelf lives, but now, retention periods are going up. “For instance, in the insurance industry in India, policy data is required to be retained for up to 10 years after the policy is settled or terminated. Or in the telecom industry, TRAI is mandating archival of CDRs for a number of years. For the engineering services/ product development businesses, design documents are required to be retained more from the purpose of re-use of standard components in new products. The pharmaceutical industry needs to store documents on clinical trials for various regulatory filings in India and outside. In the print media business, digitisation and long term archival of old issues could lead to a new revenue stream, with researchers wanting access to old and historical data.” Agarwal stated.

Indian enterprises have realised the fact of retaining e-mail and quickly retaining them as and when required by the user or for audit by regulatory bodies. Sai Gundavelli CEO of Solix Technologies says, “E-mail is a critical communication medium. There are increasing demands to keep it available and retrieve e-mail quickly. Archiving helps both performance and reliability by trimming the size of the message database. Like a loaded cargo ship, an e-mail application becomes slower and less responsive as the database increases. Moving the data to an archive helps the application stay nimble and effectively handle more users and message traffic. Moreover, this lets enterprises defer server upgrades even as e-mail requirements increase.” However, reliability comes from improved recovery times and shortened backup windows. With less data to restore, recovery from a disaster or logical fault is faster, whether using disk/tape backups, snapshots, or mirrors. Backup windows are shorter, helping to complete them in a timely and scheduled manner. It also saves on backup media, including transportation and storage costs.

Organisations believe that minimising the retention time of historical e-mail reduces their business risk. They have to make it a routine practice, and in certain instances attempt to set company-wide policies, to delete old electronic communications, assuming that the messages will not be needed nor recoverable in other places. Dhamodaran of IBM further emphasised on the role of retention policy and said, “There is strong need for retention policy at every enterprise. Sooner or later this will be visible in the industry.”

Public companies need to preserve information. Any destruction of potential evidence, including deleting e-mail, can be construed as obstruction of justice. Companies have tried to work around mandatory retention with stated corporate policies, only to find these policies null and void because an employee has copied or moved their e-mail to a location where it can be retrieved after it has supposedly been deleted. Highlighting the need of a clear retention policy, Sharad Rege, Senior Vice President, Paradyne Infotech Limited says, “If an organisation is asked to produce e-mail in discovery for a lawsuit or as part of a federal audit, it must assess all locations where messages might exist; often, leaving a firm to sift through backup tapes, personal storage folders, file cabinets, desktop and laptop computers and other forms of offline media such as CD-ROM and USB keys. The task can be enormously expensive and time consuming. Many companies have chosen to settle lawsuits because they could not provide adequate defence information in a reasonable time frame, even though the information resided somewhere within their organisation.”

A prime concern

Businesses have increasingly become dependent on e-mail and everyone has acknowledged it as a mission-critical tool and a lot of critical information resides in e-mail stores.

Given the above, e-mail management, its security and its management has become an issue of prime concern to protect critical information and comply with regulations. Bapat of EMC says, “Today, most company information and record systems are born digital and dwell in digital form on at least one computer system (and probably multiple systems, including online and offline storage). Section 404 of SOX (Sarbanes Oxley Act)– that is likely to have a greatest long term impact on IT’s compliance role—requires an unprecedented level of alignment between IT and business practices; between technology management and financial management; and between record management practices (retention, storage, access and disposition) and compliance requirements.”

E-mail retention is adopted more from the perspective of avoiding potential legal risks by being able to produce correspondence on demand. While explaining the customer’s needs Agarwal of NetApp says, “Security is a requirement for any kind of data that is deemed confidential or mission critical; including e-mail. Security infrastructure decisions complement e-mail retention infrastructure decisions and one does not follow the other.”

Many organisations are struggling with the challenge of managing the wealth of data in their enterprise e-mail systems. Although managing that information has always been important, recent concerns have brought management to the forefront.

Adding his views on this, Dhavale of CA says, “Employees spend significant amounts of time managing their respective mailboxes. Organisations today, require automated Mailbox Management wherein the e-mail administrator is able to automatically archive e-mail, instant messages, and files based on policies and indexing capabilities. A policy can be set up to stub e-mail after 60 days and store e-mail for six years. Employees are no longer required to archive messages locally or delete messages to stay within a mailbox quota.”

Taking an example of emerging trend in this domain, Dhavale adds, “Instant messaging (IM) is fast becoming a critical tool for time-sensitive business communication. Chances are the organisation is employing IM as a real-time collaboration tool with partners, customers and employees.” These conversations often include business-critical information that must be centrally managed, and archiving IM is essential in the financial services industry, where the NASD ruled that instant messages must be retained for at least three years and that communication through this channel must not violate NASD rules governing sales literature and correspondence. Hence IM is being archived as well.

Four elements are vital when it comes to managing data in your messaging system (as opposed to managing the messaging system itself): discovery, compliance, archival, and retention.

• Discovery: High-profile court cases have used the discovery of a particular e-mail message or e-mail “conversation” to establish facts.
• Regulatory compliance: A growing number of laws address several areas of messaging operation, including e-mail message and transport security, backup and recovery, administrator permissions, auditing, and privacy.
• Archival: IT departments struggle to manage the disk storage, backup and recovery windows, and data indexing that the growing volume of e-mail requires. You might need to store much of your messaging data for a period of years, but keeping it in user mailboxes can degrade the performance and reliability of the entire messaging system.
• Message retention: Even after you winnow the chaff from the wheat, storing the e-mail messages that you want or need to keep takes up resources and presents technical obstacles such as maintaining reasonable backup and recovery times. In the interest of reducing costs, many companies institute retention policies that dictate how long users can keep e-mail messages before they must be purged from the system.

With these four aspects for managing e-mail, driving much of the discussion and marketing of messaging products today because they involve practical concerns that affect almost every organisation.