Why should you test the software process first?

Dr. S Ravichandran Managing Director Trimentus Technologies

Many testing companies tend to underestimate the importance of measuring a software process using metrics analysis. The key challenge lies in identifying the type of metrics that one has to rely on to improve the software development and testing process.

In overall software engineering activities, data collection and analysis of software metrics play a major role in the software process assessment and its management. Data captured from daily activities, experiences and lessons learnt and the routine analysis that provide guidance in improving current and subsequent efforts will lead to process improvement and result in increased software reliability and predicting estimation of a testing process.

Benefits of software process assessment

With software process assessment (SPA), we can increase the software reliability multifold. It also helps in early predictability, reduction in cost of ownership and cycle time, while increasing the productivity.

Further, SPA guides for accurate schedules and risks for future software efforts. Above all, we can hope to achieve high standards in quantitative project management and increased product quality.

SPA provides information at a glance; proactively warns when situations is out of control. Managers can identify the key elements to focus on, understand what is done and know quality and productivity.

On the other hand, the engineers can now better understand product attributes; assess product and process quality and effect continuous improvement.

The metrics cycle

Metrics cycle starts when the projects start. The first step is to determine the list of metrics to be collected then, the methods of collection and analysis throughout the software development life cycle. This helps in tracking the project status, taking corrective and preventive actions based on quantitative facts.

The process metrics

Metrics that are related to the process quality includes 1) testing and review efficiency 2) defect distribution 3) testing planning and 4) testing effort distribution.

The product metrics

Metrics that are directly related to the product quality includes 1) Testability 2) Defect density and 3) Residual defect density.

Metrics demystified:

Let us discuss in detail the advantages of the metrics now.

Effort distribution

Effort distribution indicates effort for each activity in the Software Development Life Cycle (SDLC) and helps in scheduling and work distribution.

Defect density

Defect density is the number of defects in system/ Size that indicates the overall number of defects that may be present in the system (removed through reviews/ testing etc).

Residual defect density

Residual defect density denotes the number of defects after testing/ size. This number indicates the number of defects passed on to the customers

Defect distribution

Defect distribution indicates percentage of defects found during each phase of SDLC. Having arrived at a defect distribution chart, it is easy to know the phases where defect distribution is high. Now to improve the quality we must pay more attention to remove and prevent defects in design and coding. The evaluation of defect areas can also be done based on severity of defects.

Defect age

Defect age refers to the number of defects that had happened during development (phase detected - phase introduced) and the total number of defects in development and post delivery. The defect age indicates the number of phases defect has lived through since introduction.

Defect removal efficiency

We could arrive at the defect removal efficiency by calculating the number of defects detected in system and the total number of defects present. The process indicates the efficiency of the review and testing process.

Metrics collection process

Having identified the type of metrics useful for bettering the software process efficiency, we must evolve various processes to collect the metrics.

I insist that there should be a centralized knowledge master, a repository of metrics from all projects. In our projects, we appoint a quality conscious programmer who is attached to project as Quality Co-ordinator and make him or her responsible for reviewing the project metrics. We institute a Process Metrics Group and assign the responsibility for collating and issuing organizational metrics and suggest process improvements. The Project Leader have to ensure that the project data are entered into the Metrics Database regularly. There should be a Process Metrics Group Coordinator who should be responsible for collating and presenting metrics for discussion during meetings of the Process Metrics Group.

The sources of data could be the various review records, fault reports/ test records, test results report, time sheets, count sheets and estimation records, among others.

Project Metrics Analysis Techniques

I recommend the Pareto Charts and Control Charts for project metrics analysis. The Pareto charts are good for identifying the basic cause and display relative importance of problems.

The control charts on the other hand is widely used to discover the variability of processes and determine whether a process is in statistical control and to differentiate random and normal variation and determine special causes of variance.