India Inc. takes to ILM for data management

Information Lifecycle Management (ILM) is not a technology. It is a combination of data management processes and technologies that determine how data flows through an environment. By doing so, it helps end-users manage data from the moment it is created to the time it is no longer needed. Many companies are implementing a combination of these technologies and processes to enable a successful ILM practice, says Gaurav Patra

In order to realise ILM’s benefits, organisations must continuously review the usage patterns of storage resources and ensure adherence to policies and procedures, says P K Gupta

After a lot of storage consolidation bids through hardware and software implementations, it’s time for some policy-based synchronisation of data in the enterprise. ILM will be the most convenient way of doing this without wasting money. According to a recent study done by the University of California, Berkeley, new information generation has increased by 30 percent each year from 1999 to 2002. Almost every aspect of life around the world is being recorded and stored in some format. Some researchers say the amount of new information stored on paper, film, and optical and magnetic media has doubled in the last three years. That is why it’s no surprise that the development of effective, reliable and cost-efficient strategies to store data is of increasing importance.

Challenge

The amount of information that organisations need to manage and use is immense. So the challenge is how to manage all the information according to its value at every stage of its life—from creation and protection to archiving and disposal. It calls for application-level integration directly into the storage infrastructure to allow for policy-based, proactive management of the relentless growth of both structured and unstructured data.

Information Management (IM) is a model for managing an organisation’s information based on business requirements related to effectiveness and efficiency. New business drivers and regulations have added new requirements to different types of information that must be managed; these include e-mail, CRM data and all types of long-lived customer records and business documents. IT departments of companies face major challenges in meeting information protection, availability and accessibility requirements of these business-critical systems in the face of rapidly expanding user loads and exponential data growth. With their comprehensive solution suites, storage vendors have developed a unique ability to help customers maximise business continuity while minimising total cost of ownership via ILM strategies.

Storage management capabilities are not keeping pace with data growth despite declining storage infrastructure costs. Given the increasing amount and value of data, there is an information management gap. This will be the key driver for ILM. “ILM—or what we at HDS call Data Lifecycle Management—will be fuelled by the need to move data from one form of storage to another. This will help control costs when data need not be kept in expensive 24x7 infrastructures,” says P P Subramanian, country manager, Hitachi Data Systems. Manoj Chugh, president, India & SAARC, EMC, feels that ILM is a process that enables enterprises to manage their critical business information more productively through its entire lifecycle. “ILM is a term that has been coined to describe the management of data that needs to be stored for a set period of time,” he explains. Adds Avijit Basu, marketing manager, NSSO, business customer sales organisation, HP, “ILM will actively manage information throughout its life based on its changing business relevance and usage over time. This will be done by utilising policy-based storage services to automate the management of information based on service level objectives.” Shailesh Agarwal, country manager, storage solutions, IBM India, has an interesting comment to make: “On the solid foundation of automated networked storage and by incorporating the added functionality of integrated information management, we are moving to the next stage called ILM.”

ILM will actively manage information throughout its life based on its changing business relevance and usage over time, says Avijit Basu

Market moves

According to industry gurus, ILM is going to be part of all layers, starting from network storage to compute to network access to personal devices. Information spends most of its life in storage, so the growth is expected to be unimaginable. “ILM is an excellent opportunity for those who provide cross-disciplinary services that can understand diverse issues to ensure that all their needs are met,” says Agendra Kumar, country manager, Veritas Software.

Despite the IT slowdown, storage had been witnessing tremendous growth. This is because businesses are seeing an explosive growth in digital information and accessing; managing, storing, protecting and retrieving this information has also become critical. The growth in ILM in India will be triggered by the sheer increase in data created by enterprises, which, according to estimates, is growing at 50-70 percent annually. Apart from this, new business drivers and regulations have also added new requirements to the different types of information that must be managed within an enterprise, including e-mail, CRM data and all types of customer records and business documents.

With the back-end of the data lifecycle growing, and retention policies now being based on data value and legality issues, reference activity is also driving ILM. “Indian enterprises, before investing in a storage solution, emphasise low total cost of ownership (TCO) and faster return on investment (ROI). ILM helps achieve the right balance between the cost of managing a piece of information and the value of that information,” says Chugh.

Getting started

Before undertaking an ILM project, an organisation must understand how data is used and also understand its value to the organisation at any point of time based on the business requirements for a specific application. This helps an organisation uncover the business value related to the combination of data and applications—it is really this combination that defines information. “The starting point should be to understand the types of data associated with each application, whether it is structured, semi-structured or unstructured. In addition to this, the organisation needs to determine whether the data is transactional or referential. It is also important to recognise that data may change from transactional to referential during its useful life,” says P K Gupta, director, strategic development, intercontinental operations, Legato Systems.

To have an effective ILM set up, the first step is to move away from direct attached storage (DAS) and adopt automated networked storage. “This is what gives you the critical tools to consolidate and control, and ensures continuity and compliance,” says Chugh. Agrees Agarwal, “For a successful ILM, organisations should look at implementing automated network storage.”

Another important aspect is that companies should look at applying ILM practices and policies to specific applications. To achieve this, IT managers should also look for software that can provide scalability and performance, leverage existing products/environments, and support heterogeneous environments, ease of management, etc. They should not be locked into a single vendor. Apart from this, organisations need to layer their applications and storage, and should set policies that will dictate the different levels of performance, availability, functionality, and economics. This will move information to an appropriate service-level layer as requirements change. They should also define business policies for various information types. In addition, one should target a number of key applications and start applying the ILM approach to them.

Initially the approach should be towards creating an ILM infrastructure across all applications. This can be achieved by automating ILM—matching the right service level to the right application at the right cost. However, industry experts opine that most organisations will go for a three-step approach:

ILM is an excellent opportunity for those who provide cross-disciplinary services that can understand diverse issues to ensure that all their needs are met, says Agendra Kumar

• Implementing automated networked storage.
• Applying ILM practices and policies to specific applications.
• Creating an ILM infrastructure across all applications.

These steps will allow IT staffs to develop their skills and methodologies to understand their information requirements and advance to increasing levels of automation as their practical experience deepens.

Strategy

Experts say ILM is as significant a business process as CRM or ERP. An effective ILM implementation can streamline costs and management efficiencies. Organisations can actually use ILM processes to more effectively implement CRM and ERP solutions, ensuring that critical data is given top-priority storage resources and is always available. “ILM is an ever-growing process. In order to realise its benefits organisations must continuously review the usage patterns of storage resources and ensure adherence to policies and procedures,” says Gupta.

It is expected that new advances in ATA and SATA will play an important role in helping IT administrators with ILM, giving them the ability to stage back-ups and snapshots inexpensively. Storage software innovations have also increased the ability to identify, classify and then move data to the proper location over time. Once the IT department can begin showing executive management exactly how IT assets are being used, it will be in a position to properly assign charge-backs to the various groups in an effort to turn itself into a profit (or at least break-even) centre. Now is the time to start planning.

To have an effective ILM infrastructure, it is important for users to identify key capabilities or components of a comprehensive information management solution that need to be evaluated when managing information through its lifecycle. Says Subramanian, “The ideal solution is a standards-based archival platform that can satisfy the requirements of e-mail and a variety of other document types in both traditional corporate environments and specific vertical industries.” In order to implement an ILM solution, strategies are needed for network and storage consolidation, migration to automated networked storage, and alignment of service levels with data value. Keeping this in mind, the first step in implementing ILM is thoroughly understanding business needs. Requirements that should be defined include budget, business-process and access requirements; regulatory and risk management requirements, and service-level objectives related to availability, recoverability and protection of information.

Kumar of Veritas points out that once the business needs of a particular information set are defined, one can then determine requirements. While formulating the strategy users should keep in mind functionalities like access/share, monitor/remedy, replication/mirroring, protection/recovery, and migration /archiving/destroying. Storage software solutions play an important role here.

Says Agarwal of IBM, “The customer should review the complete set of products that the vendor offers and then understand the roadmap for each so that a building block approach may be used for preparing ILM infrastructure.” This means that the components used to make the ILM solution should not only be open but also support open standards so that management is simpler. “The open, flexible policy engine allows users to manage their data according to a variety of rules to address regulatory compliance, storage efficiency, data retention, data organisation, control of storage growth and a whole range of other data management issues,” says Kumar.

Again, users must define the value of each set of data at various points in the lifecycle from creation to deletion. “The IS organisation should begin implementing a lifecycle plan for select sets of data based on the available tools and the benefit to the enterprise with respect to meeting performance metrics and cost savings,” says Subramanian. Adds Vijay Pradhan, general manager of StorageTek India, “ILM itself is based on simple principles that form the basic strategy behind selection and implementation. Two critical factors should be kept in mind—the nature of data and the frequency of access, and the latency of access that the business can tolerate.”

According to P P Subramanian, ILM will be fuelled by the need to move data from one form of storage to another. This will help control costs when data need not be kept in expensive 24x7 infrastructures

E-mail

According to the new Indian IT policy of 2002, one has to preserve e-mail data for a specific period of time. E-mail is the most-used application today. It has become a major means of business communication. IDC has forecast that the number of e-mails sent daily will grow from 9.7 billion in 2000 to over 35 billion in 2005. “As e-mail has become a standard means of communication, the need to manage this data and keep it available for reference has also increased,” states Gupta. He adds that a company must maintain solid storage management, especially with respect to electronic records. Keeping this trend in mind, one will need ILM tools to manage this unstructured e-mail data. Various storage vendors are offering specific tools/products for this purpose. For instance, Legato offers Networker for backing up e-mail data, CSBS–AA, AAM for automated availability and monitoring of data, and the EX family of products for e-mail management. To manage burgeoning e-mail storage requirements, as well as to maintain access to corporate knowledge within their e-mail repositories, Legato also offers products like EmailXtender.

Hitachi Data Systems has adopted ISO’s Reference Model for Open Archival Information Systems (OAIS) to define the base set of common archival functions needed to provide archival solutions tailored to the needs of each customer. The first solutions based on this general purpose archival platform are Message Archive for E-mail, Message Archive for Compliance, and an Archival Policy Design Service to walk organisations through a review of their e-mail and storage environments and recommend specific archival policies.

Veritas too is also focusing on this space. Integration and heterogeneity is Veritas’ offering in this area. StorageTek’s solutions address the e-mail data management problem; they ensure that access to e-mail remains while you store archived e-mail and attachments on low-cost storage devices such as automated tape libraries. IBM offers a complete solution for Hierarchical Storage Management (HSM) through its range of Tivoli HSM products, IBM LTO tape libraries and eServers.

Road ahead

ILM is quickly becoming popular. As new regulatory rules are implemented and the number of disaster recovery implementations increase, ILM will play a pivotal role in helping IT professionals adhere to new standards while incurring minimum management headaches. “As soon as Indian companies start looking at their mid-term to long-term business requirements and storage needs, they will realise the need for an ILM strategy,” says Gupta. It is certain that ILM will assume greater significance as companies increasingly look at rationalising and consolidating storage resources. The advantages of cost savings and faster access to critical data will see organisations having a favourable view of ILM. Indeed, it will become a business necessity with the massive growth of data in enterprises.

The key factors for implementation of ILM are the nature of data and the frequency of access, and the latency of access that the business can tolerate, says Vijay Pradhan

Vendors Legato

Legato aims at delivering ILM solutions for the enterprise. Legato and EMC’s alliance has led to ILM solutions that will reduce the omplexity of storage management by consolidating storage management, integrating functionality and automating administration. The company offers information protection solutions that boost IT productivity combined with automated solutions to improve data access and reduce downtime. Legato’s solutions work with both Windows and Unix systems. Some of the more popular solutions that it offers are Networker (DiskBackup Option), Networker Client Connection, and DiskXtender. Along with EMC it also offers Automated Availability Manager for EMC SEDF.

Hitachi

Hitachi has infrastructure for archival solutions which embraces not only e-mail but also a variety of other document and data types, including structured and unstructured data. This infrastructure is based on an open, ISO-compliant architecture that implements Data Lifecycle Management or OLM as a complement to mainstream storage and business continuity practices. The component functions have been enhanced to reflect new business realities such as the need for regulatory compliance. This openness allows customers to participate in an interoperable environment. The company also announced the Archival Policy Design Service to help customers institute policies that remove the challenge of administering burgeoning e-mail infrastructures and satisfy e-mail-specific regulations.

ILM helps achieve the right balance between the cost of managing a piece of information and the value of that information, says Manoj Chugh

EMC

EMC’s vision is to help customers get maximum value from their information at the lowest total cost. That means building an ILM infrastructure across all applications that is policy-driven, business- centric, heterogeneous and application-independent. With a range of storage platforms, management software and consulting services, no other company has all the three components to fulfill a complete ILM solution such as EMC.

The company recently acquired two software companies, Legato and Documentum, which are into storage and information management, and give EMC added capability to rapidly implement an ILM solution for customers. EMC is also in the process of taking ILM to the next level, with an initiative called Automated ILM.

EMC’s ILM strategy is to completely manage its customers’ data flow, a method EMC believes will help carry its software sales to new heights, as well as give it the upper hand over competition. The ability of its software to support third-party products is the clear differentiator. This helps protect the customer’s investment in legacy infrastructure and at the same time upgrade its storage systems, thereby improving overall performance.

Veritas

Veritas sees information lifecycle management and compliance as a subset of the larger enterprise data management problems customers are facing. The company is focusing on enabling customers to retain, search, retrieve and report on their records regardless of storage medium, retention timeframe, retrieval requirements, etc. Veritas’ Integrated Compliance Solution approach allows for tremendous flexibility in the customer environment; it is based on business needs and keeping costs down.

Veritas offerings include OpForce for server management, Cluster Server to ensure application availability, i3 to manage application performance, SANPoint Control and Storage Central to deliver storage and data profiling, Data Lifecycle Manager to implement and enforce data management policies, and NetBackup and BackupExec to protect data.

StorageTek

StorageTek offers a range of solutions targeting the ILM space. It is also an independent storage vendor that offers all storage technologies—online, inline and nearline. In addition, the company has some specific solutions that address the e-mail data management problem. StorageTek has practical options for managing information throughout its lifecycle.

IBM

IBM believes that enterprises can effectively manage data and storage resources with HSM, and offers a complete solution through its range of Tivoli HSM products, IBM LTO tape libraries and eServers. In addition, IBM also offers the SAN File System which can be used in conjunction with other IBM virtualisation products such as the storage volume controller to effectively manage data placement and migration throughout the lifecycle of the data.

The company is either announcing or previewing enhancements to many of its existing products, bringing together a whole host of IBM services that provide customers a comprehensive, open set of solutions that can be tailored to help address their particular needs.

For successful ILM, organisations should look at implementing automated network storage, opines Shailesh Agarwal

Hewlett-Packard

HP’s ILM strategy leverages its deep, rich history in delivering technology innovation and its strategy for the Adaptive Enterprise. It builds on the company’s unique approach designed to help the customer buy less storage less often and provides the roadmap needed to address the three major information management challenges: retention, data management and reference information management. HP addresses ILM the way customers manage data, step-wise through stages in the lifecycle where the data resides.

HP offers products for replication and distribution, archiving and recalling, protection and recovery. In terms of services, HP can offer back-up and recovery services, data protection services, archiving services, data replication and data sanitisation.

ILM growth drivers

The amount of digital information created because of the information explosion, and which needs to be stored. The need for proper alignment between the cost of managing a piece of information and the value that information has to the business. Business continuity. The real-time capture and archiving of e-mail, instant messaging, and other forms of communication have triggered the need for information management in an enterprise. The cost of storage hardware is falling but has not kept pace with the rate of growth of data volumes being generated. IT budgets are either flat or declining, so storage acquires strategic relevance in reducing overall IT costs. Regulatory and legal compliance issues have assumed vital importance in the global business climate, requiring organisations to preserve data for longer periods of time.

The step-by-step approach

Business-centric approach: Derived from the key processes, competencies and initiatives of the business. Policy-based: Anchored in enterprise-wide information management policies that span all processes, applications and resources. Centrally-managed: Providing a single view at all information assets of the business. Heterogeneous: Encompassing all types of platforms and operating systems. Aligned with value of the data: Matching storage resources to the data’s value to the business at any given point in time. Need for an application-focused ILM approach: Provides retention, access, recoverability, searchability and value.

gaurav@expresscomputeronline.com