SOFTWARE QUALITY METRICS
Software metrics can be classified into three categories:
❑ Product metrics describe the characteristics of the product such as size, complexity, design
features, performance, and quality level.
❑ Process metrics can be used to improve software development and maintenance. Examples
include the effectiveness of defect removal during development, the pattern of testing
defect arrival, and the response time of the fix process.
❑ Project metrics describe the project characteristics and execution. Examples include the
number of software developers, the staffing pattern over the life cycle of the software, cost,
schedule, and productivity.
• Some metrics belong to multiple categories. For example, the in-process quality metrics of a
project are both process metrics and project metrics.
• Software quality metrics are a subset of software metrics that focus on the quality aspects of
the product, process, and project. These are more closely associated with process and product
metrics than with project metrics.
IEEE Defines a Software Quality Metric as
❑ A quantitative measure of the degree to which an item possesses a given quality attribute.
❑ A function whose inputs are software data and whose output is a single numerical value
that can be interpreted as the degree to which the software possesses a given quality
attribute.
Software quality metrics can be further divided into three categories −
❑ Product quality metrics
❑ In-process quality metrics
❑ Maintenance quality metrics
1. Product Quality Metrics
These measure the quality of the software itself i.e the final product. They include
❑ Mean Time to Failure
❑ Defect Density
❑ Customer Problems
❑ Customer Satisfaction
i. Mean Time to Failure
MTTF is the average time that a software system operates before it fails. It reflects the reliability
of the software. It is measured in time units such as hours or days.
MTTF = Total Operating Time / Number of Failures
A higher MTTF signifies a more reliable system, with longer intervals between failures. A lower
MTTF warns of potential flaws or increased risk of breakdowns.
The MTTF metric is most often used with safety critical systems such as the airline traffic control
systems, avionics, and weapons.
ii. Defect Density Metric
• Defect Density is a software quality metric that measures the number of defects found in a
software system relative to its size. It helps determine how error-prone or stable a software
product is after development or testing.
Defect Density= Number of Defects/Size of Software (in KLOC or Function Points)
“You can’t control what you can’t measure” Tom DeMarco (1982)
, • The denominator is the size of the software, usually expressed in thousand lines of code
(KLOC) or in the number of function points. Defect density is therefore expressed as the
number of defects per thousand lines of code (KLOC) or per function points.
• A low defect density indicates high software quality, suggesting the system was well-designed
and thoroughly tested while a high defect density shows poor quality, meaning more errors
exist per unit of software.
iii. Customer Problems
• Another product quality metric used by major developers in the software industry measures
the problems customers encounter when using the product.
• For the defect rate metric, the numerator is the number of valid defects. However, from the
customers’ standpoint, all problems they encounter while using the software product, not just
the valid defects, are problems with the software.
• Problems that are not valid defects may be
❑ Usability problems
❑ Unclear documentation or information,
❑ Duplicates of valid defects (defects that were reported by other customers and
fixes were available but the current customers did not know of them),
❑ or even user errors.
• These so-called non-defect-oriented problems, together with the defect problems, constitute
the total problem space of the software from the customers’ perspective.
• The problems metric is usually expressed in terms of problems per user month (PUM):
PUM = Total problems that customers reported for a time period ÷ Total number of license-
months of the software during the period
Where
❑ Total problems reported include both true software defects and non-defect-related
issues such as usability difficulties or documentation errors
❑ License-months represent the total number of active users (licenses) multiplied by
the number of months they used the software during that period.
iv. Customer Satisfaction Metric
• The Customer satisfaction metric measures how satisfied users are with a software product
based on their experience using it.
• This metric helps software developers and organizations understand how well the software
meets user needs and expectations after deployment.
• Customer satisfaction is often measured by customer survey data via the five-point scale:
❑ Very satisfied
❑ Satisfied
❑ Neutral
❑ Dissatisfied
❑ Very dissatisfied
Based on the five-point-scale data, several metrics with slight variations can be constructed and
used, depending on the purpose of analysis. For example:
❑ Percent of completely satisfied customers
❑ Percent of satisfied customers (satisfied and completely satisfied)
❑ Percent of dissatisfied customers (dissatisfied and completely dissatisfied)
❑ Percent of nonsatisfied (neutral, dissatisfied, and completely dissatisfied)
“You can’t control what you can’t measure” Tom DeMarco (1982)
Software metrics can be classified into three categories:
❑ Product metrics describe the characteristics of the product such as size, complexity, design
features, performance, and quality level.
❑ Process metrics can be used to improve software development and maintenance. Examples
include the effectiveness of defect removal during development, the pattern of testing
defect arrival, and the response time of the fix process.
❑ Project metrics describe the project characteristics and execution. Examples include the
number of software developers, the staffing pattern over the life cycle of the software, cost,
schedule, and productivity.
• Some metrics belong to multiple categories. For example, the in-process quality metrics of a
project are both process metrics and project metrics.
• Software quality metrics are a subset of software metrics that focus on the quality aspects of
the product, process, and project. These are more closely associated with process and product
metrics than with project metrics.
IEEE Defines a Software Quality Metric as
❑ A quantitative measure of the degree to which an item possesses a given quality attribute.
❑ A function whose inputs are software data and whose output is a single numerical value
that can be interpreted as the degree to which the software possesses a given quality
attribute.
Software quality metrics can be further divided into three categories −
❑ Product quality metrics
❑ In-process quality metrics
❑ Maintenance quality metrics
1. Product Quality Metrics
These measure the quality of the software itself i.e the final product. They include
❑ Mean Time to Failure
❑ Defect Density
❑ Customer Problems
❑ Customer Satisfaction
i. Mean Time to Failure
MTTF is the average time that a software system operates before it fails. It reflects the reliability
of the software. It is measured in time units such as hours or days.
MTTF = Total Operating Time / Number of Failures
A higher MTTF signifies a more reliable system, with longer intervals between failures. A lower
MTTF warns of potential flaws or increased risk of breakdowns.
The MTTF metric is most often used with safety critical systems such as the airline traffic control
systems, avionics, and weapons.
ii. Defect Density Metric
• Defect Density is a software quality metric that measures the number of defects found in a
software system relative to its size. It helps determine how error-prone or stable a software
product is after development or testing.
Defect Density= Number of Defects/Size of Software (in KLOC or Function Points)
“You can’t control what you can’t measure” Tom DeMarco (1982)
, • The denominator is the size of the software, usually expressed in thousand lines of code
(KLOC) or in the number of function points. Defect density is therefore expressed as the
number of defects per thousand lines of code (KLOC) or per function points.
• A low defect density indicates high software quality, suggesting the system was well-designed
and thoroughly tested while a high defect density shows poor quality, meaning more errors
exist per unit of software.
iii. Customer Problems
• Another product quality metric used by major developers in the software industry measures
the problems customers encounter when using the product.
• For the defect rate metric, the numerator is the number of valid defects. However, from the
customers’ standpoint, all problems they encounter while using the software product, not just
the valid defects, are problems with the software.
• Problems that are not valid defects may be
❑ Usability problems
❑ Unclear documentation or information,
❑ Duplicates of valid defects (defects that were reported by other customers and
fixes were available but the current customers did not know of them),
❑ or even user errors.
• These so-called non-defect-oriented problems, together with the defect problems, constitute
the total problem space of the software from the customers’ perspective.
• The problems metric is usually expressed in terms of problems per user month (PUM):
PUM = Total problems that customers reported for a time period ÷ Total number of license-
months of the software during the period
Where
❑ Total problems reported include both true software defects and non-defect-related
issues such as usability difficulties or documentation errors
❑ License-months represent the total number of active users (licenses) multiplied by
the number of months they used the software during that period.
iv. Customer Satisfaction Metric
• The Customer satisfaction metric measures how satisfied users are with a software product
based on their experience using it.
• This metric helps software developers and organizations understand how well the software
meets user needs and expectations after deployment.
• Customer satisfaction is often measured by customer survey data via the five-point scale:
❑ Very satisfied
❑ Satisfied
❑ Neutral
❑ Dissatisfied
❑ Very dissatisfied
Based on the five-point-scale data, several metrics with slight variations can be constructed and
used, depending on the purpose of analysis. For example:
❑ Percent of completely satisfied customers
❑ Percent of satisfied customers (satisfied and completely satisfied)
❑ Percent of dissatisfied customers (dissatisfied and completely dissatisfied)
❑ Percent of nonsatisfied (neutral, dissatisfied, and completely dissatisfied)
“You can’t control what you can’t measure” Tom DeMarco (1982)
