Linux in the Enterprise: High Performance Computing

Linux clusters go upwardly mobile

The fastest supercomputer in India runs Linux and the open source operating system has a near monopoly on high performance computing in R&D and chip design. However, it still plays second fiddle to Unix in the commercial world, says Prashant L Rao

High performance computing (HPC) is gaining ground in India across verticals. While commercial HPC clusters still tend to run on commercial Unix or even on Windows, Linux-based clusters are popular in R&D circles and among chip design firms for Electronic Design Automation (EDA). Linux-based HPC clusters often run on top of Intel’s Xeon or Itanium processors. In fact, India’s biggest HPC cluster resides at Intel India’s Airport Road office in Bangalore. This cluster is ranked 125 on the global Top500 list of supercomputers and consists of IBM xSeries machines powered by 574 Xeon 2.4 GHz processors tied together by Gigabit Ethernet. Its theoretical peak performance rating is 2755.20 Gflops. Intel uses this machine for (among other activities) running EDA tools as it designs next-generation processors.

The SGI Altix 3000 is the most powerful Linux-based SMP HPC box in the world

A nascent market

Vijay Keshav, Industry Solutions manager-Asia Pacific (High Performance Computing & Life Sciences) at Intel says, “HPC in India is pretty nascent. India is not a mature market.”

More than 25 clusters have been deployed in the country during the last ten months. The biggest of these include those at the Indian Institute of Science (IISc), Bangalore, C-DAC, the Ministry of Defence and the Bhabha Atomic Research Centre (BARC) in Mumbai.

Traditionally, HPC starts in R&D, moves into engineering and finally becomes a tool of commercial production. India is still moving into the engineering phase.

“Virtually all clusters in R&D and in engineering run Linux,” says Keshav. “Some commercial clusters doing CFD (computational fluid dynamics) run Windows NT/2000 as well.”

Why clusters?

While commercially available SMP (symmetric multiprocessing) systems top out at 128 processors, clusters run into thousands of processors. The world’s largest Linux-based HPC cluster is the MCR Cluster at Lawrence Livermore National Laboratory, USA, which has 1,152 nodes with 2,304 processors. “Weta [Digital, New Zealand] uses 3,000 Xeons in a cluster for movie animation. There have been a few deals in the 4,000- to 5,000-node range,” adds Keshav. India’s biggest cluster, as we mentioned earlier, has close to 600 processors.

That said, there’s still a place for SMP boxes in HPC. Take the example of IISc’s SERC (Supercomputing Education & Research Centre), which has deployed a large scale SGI SMP HPC solution.

“Certain applications run best on an SMP box,” says Puneet Gupta, country manager, pSeries, IBM India. He should know; IBM has sold lots of 32-way p690 machines to HPC users in oil & gas and higher education.

Vijay Keshav of Intel estimates that 75 to 80 clusters will come up in India in the next 12 months

Where Linux fits in

“EDA is typically done on Linux. In the old days it was done on UNIX. Porting applications from Unix to Linux is easy. Migration is more from Unix to Linux rather than from Windows,” says Keshav.

Then there are the tools. The OSCAR (Open Source Cluster Application Resources) toolkit lets you install a Linux cluster in 30 minutes. “90 percent of Linux clusters are built using Beowulf & OSCAR,” says Keshav. On the commercial side, people use Alinka for deploying Linux clusters.

Rajesh Saha, country manager, zSeries, IBM India says, “We don’t focus on a particular processor. Intel, AMD or Power4—we offer Linux clusters on all these processors.” IBM even has a cluster offering, the eServer 1350, that’s shipped as a rack filled with servers that have been tested, integrated and pre-installed with Linux. The customer orders X nodes with Y CPUs and Z memory and that configuration is tested and integrated with software and shipped ready to use.

“Code and algorithms required to solve grand challenge problems need constant optimisation. Linux provides HPC with an environment that lets you use the same code from a notebook to a supercomputer,” adds Saha.

IBM’s SP (Scalable Parallel) computer can support as many as 8,000 processors as a single computing environment. That kind of software is getting ported to the e1350 now. SP products are getting reworked as CSM (Cluster Systems Management). IBM is taking the PSSP (Parallel System Support Program) software stack of the SP machines and offering it as CSM for Linux and AIX HPC clusters. “Only IBM and Cray have that kind of software stack,” says Gupta.

Even in SMP-based HPC, Linux has a key role to play. SGI’s Altix 3000 is built around the Intel Itanium 2 processor and it runs Linux. “This is the only system that lets you run Linux on 64 processors,” says Avinash Fotedar, marketing manager, SGI India. Beyond that, you can cluster two 64 CPU boxes using NUMAlink (the same interconnect that is used within the Altix system). “You can build superclusters of up to 2,048 processors, there is no degradation up to that level,” says Fotedar.

Will HPC honours help Linux?

Linux is popular with academics doing R&D and with chip designers who run Linux-based HPC clusters. Intel’s Keshav estimates that 75 to 80 clusters will come up in India in the next 12 months. Oil & gas, life sciences and digital content creation are expected to be commercial markets for HPC in 2004. “We are talking to quite a few companies, this is a huge market. We see a lot of opportunities emerging by mid-2004 for HPC on Linux,” says Saha.

Unlike the R&D and engineering markets, the commercial HPC world has been the preserve of Unix or Windows. However, as the case study of National Stock Exchange (on page 8 of this issue) indicates, Linux is becoming upwardly mobile in the corporate world too.