Peer-to-Peer

How Rediff.com got its groove back

Plagued by data overflow and having no way to keep its active and inactive data segregated, Rediff went in for ILM. By Tanu Talwar

With online portals growing at a stupendous pace, the sustainability of any portal is determined by the user experience that it guarantees for which enhanced application performance becomes a necessity. In order to ensure that a user gets the best online experience, it is imperative that a site offers information and services at the click of a button. However, to bring about efficiency in its performance its imperative that the site is also equipped with an efficient IT infrastructure.

Rediff.com, is a leading provider of Internet news, information, communication, entertainment and e-commerce services. As its user base grew, the company faced the problem of managing enormous data volumes and the user experience at Rediff began to deteriorate. Venki Nishtala, CTO, Rediff, says, “Being a consumer portal we have to deal with large volumes of data that grow every minute. These records are of immense importance and need to be continuously stored and updated for future reference.”

Besides managing and storing entertainment and other information, the portal’s e-commerce services required the site to handle confidential customer and seller information that had to be vetted and made available to clients every now and then.

Performance degradation

Rediff had implemented several data intensive custom applications with multiple instances, for various online functions. These applications were designed in-house and they did not offer features such as archival, purging or sub-setting resulting in huge storage and data management problems. Being a B2C site, the portal had to deal with numerous transactions captured from multiple applications and distributed across instances. Lacking a structured storage process, locating these transactions was a major hurdle. Then again, with no differentiation between operational and production data, separating active from inactive data was tedious, time consuming and a direct drain on performance. Nishtala explains, “Over the years, huge volumes of data stored with us had become inactive due to infrequent usage. However, as there was no systematic separation between operational and unused data a lot of valuable time and storage space was spent each day in taking backups of both used and unused data.”

Moreover, with the database burgeoning the performance of in-house designed custom application was adversely affected. The performance of functions within this application began to deteriorate causing it to run slow. This application degradation in turn increased the backup time as well as the storage costs. Nishtala asserts, “Even though we had large volumes of records and data that were no longer in use they could not be deleted. These records contained valuable information that might be needed in the future”. In order to avoid any contingencies arising in the future, the portal was not only spending resources in storing unneeded information but it was also incurring millions of rupees of expenditure to pay off mounting IT maintenance bills.

However as the performance degradation started to hamper user experience resulting in the bottom line suffering, Rediff decided to take quick action. Nishtala states, “Our commitment to an excellent user experience requires that all data captured be available for online access 24x7 and for 365 days. However the tremendous increase in the volume of data had become a major concern for us as it was acting as a hindrance in meeting our standards and performance expectations.” In order to transform its overall operations, the company decided to go in for a well-structured Information lifecycle management (ILM) strategy besides, reducing the load on its production database by relocating e-commerce transactions. “The performance issue forced us to evaluate the option of moving the historical data from the production environment,” adds Nishtala.

Choosing Solix

Having decided to do something about its IT woes, Rediff went searching for a solution to upgrade its deteriorating applications. The company considered a number of ILM vendors before deciding to go with Solix ARCHIVEjinni. Sai Gundavelli, CEO, SOLIX Technologies explains, “Rediff required an organized ILM function that would help it manage and serve information by assigning a type or category based on access rules, retention requirements and business practices.” The key was finding a solution that could help Rediff automate the process of selection, archive, purge and focused extraction of data. Moreover, it was essential to ensure data integrity as well as online access to archived data and non-production instances to make the project a success.

From Rediff’s perspective there were a number of factors in favour of Solix’s solution. Its patent pending interoperability layer technology, enables companies to maintain transparent “read-only” access to archived data while ensuring data integrity remains relationally intact, even after upgrades. The result is a secure production environment and streamlined performance. The solution also offered to address the storage cost by eliminating unnecessary data duplication with powerful instance subsetting capabilities. It ensures enhanced application performance by relocating inactive data from the production database to a secure online data archive.

The implementation cycle

The solution was implemented by Solix over eight weeks. A two-member team from Solix’s professional services group closely studied the portal’s requirements and demonstrated the most effective way to tackle the situation. The team along with the Rediff’s engineering team planned out a roadmap to discover the metadata of the custom application that had been built in-house and based on that built the configurations for data archiving. Challenges faced by the team included finding a way to relocate the bulk of database transactions without compromising on online data access. It was determined that such a complex task, taking into account the intricacies of data models stored in multiple custom applications deployed at Rediff, would take considerable time and there was always the danger of loss of data integrity and access.

Taking these aspects into consideration, Solix presented a strategy for archive and purge as well as instance subsetting for entire online applications including the order management system (OMS). The application consultants and the users at Rediff finalised the details of data retention, extraction and data storage. The project was divided into two phases. The first phase involved archive & purge and the second phase was about instance subsetting. Solix further implemented its configurable metadata driven ILM tool ARCHIVEjinni at Rediff using its in-built features such as the configurator and MiniMe. The team from Solix also trained the personnel at Rediff in the use of Solix ARCHIVEjinni.

Enhanced application performance

Post-implementation Rediff got its return on investment. The solution provided Rediff the capability to design configurations for seven custom applications, which enabled automatic archival and purging of transactional data into separate instances. With the solution in place, Rediff is now capable of providing data subsets for testing purposes, instead of backing up all its data on a daily basis. “We have achieved around 75 percent reduction in storage requirements. Now we keep a few months data on the production server and the rest in online archiving storage. We are experiencing more than a 50 percent Performance mprovement in our online services after implementing ARCHIVEjinni,” adds Nishtala.

The deployment has empowered Rediff to accommodate any changes in its applications and data retention needs even with minimal support from the solution provider. Besides saving on huge storage and maintenance expenses, the organisation has been able to minimise its storage, backup and recovery time.