It's testing time

As software users become “certificate-conscious”, developers are queuing up to use the services of independent software testing (IST) service firms. G Sankaranarayanan reports how the business of finding and fixing bugs is finally taking off

Indian software testing firms are building competencies by focussing on specific domains such as banking, says V N Mahesh

When asked “Why do you need IST firms in the first place?” V N Mahesh, executive director of Maveric Systems, a fast growing IST firm, says: “Because, in the past most IT products failed to deliver.”

IT users are now more demanding and expect that the products that they buy are rigorously tested and certified for deliverables. “The survivors of the IT economy understand that it is imperative to deliver “fully-tested” products,” says Kulasekharan, chief operating officer of the Chennai-based ReadyTestGo (RTG), a pioneer in Indian software testing services.

Today’s comprehensive testing practices go beyond the “final” process of in-house “proofreading” of written codes. It is a critical process and stretches itself through a product lifecycle, from development to implementation to maintenance.

Catch bugs at the beginning

It makes sense to have a project or product tested even at the conceptualisation stage and the pilot thereafter, since it costs “a hundred times more in time, effort and money” to find and fix bugs once the product moves to and comes out of development. “The (testing) processes are geared to shift quality assurance from a tactical operation aimed solely at finding defects, to a strategic activity that aims to deliver superior quality products,” puts a testing company’s brochure succinctly.

Testing is usually associated with performance engineering and its various hues that put a product through functionality, compatibility, security, architecture benchmarking, and installation (hardware & software) hurdles.

Performance testing is something basic that users and producers do now-a-days and it answers questions such as “How much can I get out of a given product?” “How many transactions per second?” “What’s the speed at which it can execute tasks?” or even “How long will the batteries last?” The test could be a simple, manual experiment done with a stopwatch or using a “testbed” - a single PC or hundreds of network clients and servers connected over a gigabit backbone. The testing methodologies and tools are many. Certain guidelines are emerging, however. The Illinois Institute of Technology, for instance, recently developed a Testing Maturity Model, to assist software organisations in assessing the tasks of improving their software testing processes.

Detached testing

Software companies set up detached testing divisions to audit their products. A detached team does not consist of developers who passionately wrote the codes that are to be tested and can therefore take a level headed look at the programme.

Recently, HCL Technologies set up the HCL Testing Centre of Excellence (CoE) in association with Segue Software, “the application reliability expert” to expand its independent testing and quality assurance services for its outsourced IT and software development projects.

However, many companies cannot afford to set up an exclusive testing team and keep up with the competencies required for a state-of-the-art software testing centre. These companies turn to specialists with domain expertise offering independent services.

Like corporates that encourage consulting firms to assess their business plans or management practices, software companies seek the opinion of third-party testers on the quality of their software products. IT users, on the other hand, engage IST firms to ensure that they are buying exactly the product that they need. Users go by certificates to know how effectively new products will integrate into their existing infrastructure or if their applications will perform as expected on a new system. In testing, independent views are much sought after. As in accounting or quality, in testing too, third-party endorsement, benchmarks and certifications, lend credibility. That’s why companies have started to display certification logos on their products.

Outsourced testing takes its first steps

The total outsourced testing services market is still quite small, roughly 5 to 7 percent of a project’s cost flows into testing. The spending on testing entirely depends on the criticality of the applications being developed. Consumers normally don’t go berserk if they find dust in their food. However, if they find the same in a medicine it may be endangering lives (of the drug’s manufacturers). Thus, the attitude of users and producers towards testing, change depending upon criticality. NASA, for instance, could spend upwards of 50 percent of its budgets on testing.

In India, there are roughly ten players of a reasonable size with a combined turnover of around Rs 200 crore. The leaders are RelQ, ThinkSoft, RTG, Maveric, Polaris Application Certification Enterprise (PACE), IQA Tester and Stag Software. US companies that enter India either set up company owned testing centres or outsource. According to reports, Lionbridge Technologies, the parent company of VeriTest, has already set up a centre in India. Companies such as Accenture plan to outsource testing for Indian projects to IST firms.

Regular business for IST firms comes from global IT majors. “MNCs have developed processes to outsource to best-of-breed vendors. Those, who outsource to multiple development vendors in India are involving independent software verification and validation partners to ensure software quality,” Kulasekharan says.

Vertical-focused firms

Without exception, all IST firms are new and small. Perhaps that explains why they are focused on one particular technology or customer segment. This way, they can hope to gain expertise across vendor neutral and standard testing tools for a particular segment such as banking. For instance, RTG claims to have extensive domain expertise in technology and finance. The company could become an offshore quality assurance partner for several software product companies managing the testing process right from the first build onwards. Maveric too focuses on the financial segment.

Indian IST firms are expanding, by year end RTG wants to take its team size from 160 to 250. Maveric, which has a 170-member team, wants to grow to 500 by 2007. The challenge is going to be finding the right people as developers applying for testing jobs consider it as a stop-gap arrangement. This perception is changing slowly as testing experts’ pay packages improve. IST firms, too, charge a few dollars more per man-hour than the software houses. To make things worse, there are no comprehensive courses in software testing offered by Indian training institutions. Hence, IST firms have to depend entirely upon in-house capabilities to train beginners.

Maveric is planning to tie-up with BITS Pilani to jointly offer software testing courses to its employees. Maveric will design the course material and BITS Pilani will manage the examinations and certifications.

Branding testing services

On the future direction of testing services, the pioneering US-based IST firms such as VeriTest, 20/20, BetaLabs and National Software Testing Company are showing the way. Their mantra: Build brands to get ahead in the certification market.

VeriTest sells its benchmark results to a popular IT publication. Niche IT publications naturally become customers of IST as they reach the target audience and push the need to adopt software testing. However certification is different from consulting. “We are not the Gartners of the world. We do not draw up business strategy for organisations to influence their IT spending decisions,” Mahesh opines. “But we can be more than a generic testing company. We can carve a niche for ourselves with certification in specific domains. We are heading there by building our internal competence.”

People in the industry admit that there is only so much they can do. There is a ceiling to growth. Many admit that they do not foresee their expansion exceed the 1,000 client mark. Besides, like customer service, quality assurance is too strategic for anyone to outsource. For instance, an automobile company such as Ford or General Motors cannot afford to hand over quality assurance to a third-party. Even software heavyweights such as IBM or Microsoft have not thought about this yet. Apart from IT users, business for IST firms comes mostly from small and medium software companies. That said, there’s still a large market for existing players.

sankar@expresscomputeronline.com