QA Metrics: Beyond Discovering Defects

What are the different types of QA Metrics?

The absolute QA metrics are based on the data obtained during the development and execution of test cases. These metrics are tracked during the entire software testing life cycle. These metrics are an indicator of how well the testing process is going along.

The derivative QA metrics can help to assess the effort that the testing team puts in day in and day out to achieve the desired outcome or results. This information is helpful for testing leads, and QA managers who are carefully monitoring the performance of different resources.

Importance of QA Metrics

With QA metrics, your customers are regularly updated about the status of the project and can easily find information if needed.

QA metrics should not just focus on tracking and discovering bugs.QA metrics evaluate the various data trends to control the quality of software systems. You can also rely on past data trends and experiences to bring improvements. This can be a helpful guide in estimating future testing efforts.

Let us try and understand some of the key metrics used by QA companies at the software project level, department level, and company level respectively:

1. Burn Down Chart: This is a pictorial representation of the amount of work completed by the development teams. If you consider testing a separate activity, a burn-down chart can also help visualize the real performance of the in-house and offshore QA testing teams for a complete release.


2. Defect Distribution: The distribution of defects that are discovered in the different parts of the software. This helps to improve specific modules of software that have more defects than expected.


3. Defect Removal Efficiency: This helps to identify the total number of defects identified during the development cycle, and how many of them are being fixed during that release.


4. Cost of bug fixes during the pre-or post-release cycle: This metric helps to evaluate the total effort that was spent to fix issues discovered before the release of a software version versus the afterward release of that version. QA companies can understand how costly it is to fix a bug before or after production.


5. Issues Reported by Customers: The total number of quality-related issues that are reported by external customers. It indicates the software quality and how it impacts the customers that are using it.


6. Defect Severity: This metric is all about detecting how severe a defect can be. The defect can have an overall impact on the business value or system functionality as well. For example, a QA company can use this metric to discover that certain modules need extensive testing because it directly impacts the business value and customer engagement aspects.


7. Defect Cycle Time: The metric allows you to measure the length of time it took when the bug was discovered until the time it was fully resolved.


8. Defect Spillover: Used to measure how effective the teams were when it comes to handling defects. The total number of defects that cannot be resolved in each sprint or iteration cycle.


9. Defect Categories: You can categorize software defects into:
   1. Functional Defects
   2. Performance Defects
   3. Usability Defects
   4. Security Defects
   5. Compatibility Defects