Backup system is lifeline of NSE

The National Stock Exchange handles more data in a day than many organisations do in a year. Here's how the NSE does it

G M SHENOY says the most critical asset of the National Stock Exchange is its transaction data

The National Stock Exchange (NSE) has a 60-65 percent share of the equity market. Its average daily turnover is Rs 8,000-9,000 crore. According to G M Shenoy, vice-president of information technology at NSE, the most critical asset of the exchange is its transaction data.
The total volume of data residing in NSE's systems runs into 2-3 terabytes, and is expected to grow to 7.5 terabytes over the next four years. Any downtime is unacceptable at NSE since the business lost due to data loss can be tremendous.
Till about three years ago, NSE had a distributed back-up system in place using direct-attached tapes. The processes required 400 GB of data to be backed up daily and 900 GB at the weekend over multiple servers and across different platforms such as Unix, NT and Linux. However, the changing external market dynamics warranted a centralised back-up solution. Data volumes within the exchange were also growing.
The time required for back-up activity also kept increasing as data increased, so there was concern that the NSE would not able to meet its deadlines in the future. The back-up time window available to NSE is between 9 pm and 7 am. Shenoy points out that considering the rate at which the volume of data was growing, the exchange would not have been able to meet the 7 am deadline if it had continued with its earlier system.

Out of the box

NSE had two options. First, to go for back-up on the network, that is, LAN. Second, back-up using SCSI. Since it had a back-up window of just a few hours, neither of these two could meet NSE's requirements. A LAN-based back-up would lead to network congestion, while connecting the tape libraries to the server using a SCSI cable would mean that storage resources could not be shared. In order to enable sharing of resources, and efficient transfers within the back-up window, NSE therefore opted for a SAN-based solution. With a SAN-based LAN-free solution, the SAN functions on its own communication path independent of the LAN, and the main network is also relieved of data transfer loads and congestion. "A separate network for back-up means that there are no chokes in the network; the back-up, being on a Fibre Channel, is also faster. This was required for meeting the back-up time window, which remained constant," explains Shenoy.
NSE considered various competitive options before zeroing in on StorageTek's SAN-based LAN-Free solution. The evaluation before arriving at the decision was made on the basis of factors like speed, connectivity, back-up software, recovery and support.

SAN-based backup

The solution implemented at NSE comprised a StorageTek L700 tape library (400 slots) with four DLT tape drives, Brocade switches, Storage Net Fibre Channel routers/bridges and back-up software. NSE's servers are now connected to the Brocade switches using Fibre Channel. The tape drives are connected to the Brocade switches via bridges to implement a heterogeneous SAN. The SAN is specific only to the centralised tape solution for back-up purposes. Veritas NetBackup Data Centre has been installed on the servers. The tape library is located at the data centre in Mumbai. At the end of the day, the back-up is taken on the tape library. The solution allows the pool of tape drives to be shared among multiple connected servers, hence there is no need to back-up the servers individually.
NSE did face some teething problems with its new back-up system during the implementation stage as well during as the early stages of post-implementation. A few of the servers gave problems, and some software patching had to be done. During the early stages a lot of testing of procedures was carried out. It took almost six months for the system to stabilise. However, once the back-up system steadied, NSE was comfortable with it.

Meeting deadlines

SAN-based direct connect technology and the use of automated tape back-up has resulted in a more efficient, consistent, disciplined and reliable back-up process for NSE even while its data size is growing and the back-up window shrinking. Today, even with 600 GB of data back-up per day, NSE is easily able to meet the 7 am deadline. Since the entire process is now automated with no manual intervention required, NSE can back-up in half the time, and the manpower requirement has not increased though data volumes and applications have.
Centralisation of the process has made control and monitoring tasks easier so that back-ups made are more reliable. Today the entire back-up happens in the Mumbai office, as opposed to back-ups being taken at multiple locations. Availability of a single management console for heterogeneous platforms provides for seamless and more efficient management. Restoration is easier, more systematic and faster. It also provides the scalability needed to increase storage capacity as NSE's requirements grow.