Using Measurement to Improve Quality

Originally Published MDDI October 2001

The new ISO 9001:2000 standard puts an unprecedented emphasis on measurement. Although companies may find compliance challenging, they will benefit from improved quality systems.

Daniel P. Olivier and Paschal Dwane

With the official publication in December 2000 of the latest version of the ISO 9001 quality system standard (ISO 9001:2000), an unprecedented number of changes were introduced. Among the most significant of these is the process approach to the definition of quality systems.¹ In this approach, an organization is defined as a system of processes, and quality management consists of identifying, analyzing, and controlling these processes and their interactions. This approach also emphasizes the understanding of requirements, the need to assess the value of processes, and the measurement of the processes to support evaluation and continual improvement.

The concept of processes is familiar to most companies through the use of flow chart techniques, among other things. The concept of measuring processes, however, gives rise to questions that are new to many. What processes should be measured? What measurements should be collected? Are there industry standards for measurement? To what extent should the collection of these measurements be automated?

Analysis of quality processes is critical for process optimization and determining the effectiveness of process changes. In the absence of appropriate measurement, attempts at process improvement often fail. Effective measurement is essential to ensuring that any process changes introduced realize the desired quality and productivity benefits.

WHY MEASUREMENT?

Measurement was not an essential component of the ISO 9001:1994 standard. Why has it become one in ISO 9001:2000? The answer involves the process approach to quality systems. ISO 9001:1994 was somewhat prescriptive regarding the definition of processes, specifying 20 subelements. But in ISO 9001:2000, manufacturers are responsible for defining their own processes. Therefore they must also define their own standards for assuring that those processes generate quality products efficiently.

Along with this responsibility to define internal processes comes a need to define measurements that ensure the effectiveness of those processes. Measurements of processes are essential to ensure the final quality of the product. In other words, to the extent that design and manufacturing processes have a low incidence of errors, the finished product is likely to be of higher quality. Measurements can also identify process inefficiencies such as scrap rate, effort devoted to rework, and excessive cycle times. Each of these measurements also provides insight into business efficiency and profitability.

CATEGORIES OF MEASUREMENT

The science of measurement is not well established for most organizations. Figure 1 outlines a hierarchy of measurements that corresponds to the evolution most companies go through as they implement and refine a measurement system. This evolution reflects that of the ISO 9001 standards themselves. Before the introduction of the first ISO 9001 standard in 1988, industry focused primarily on quantity measurements. The 1994 and 1998 standards gave rise to optimization measurements, which are aimed at corrective and preventive actions. The ISO 9001:2000 standard requires a third generation of measurements to establish processes that strive for continual improvement.

Quantity Measurements. As companies implement a quality system, they focus first on measurements of quantity. Such measurements reflect a single quantitative dimension. Examples include the volume of products manufactured, the total revenue generated, or the number of new product designs released. These measurements have typically been used for production forecasts and inventory management.

Optimization Measurements. In contrast to quantity measurements, optimization measurements provide more detail. While a quantity measurement may include the volume of products produced, an optimization measurement would address factors such as products produced per hour of labor, cycle time required for each product, or external resources consumed per product. Optimization measurements can be used to evaluate efficiency and, therefore, to assess the benefits of process improvement efforts.

Process Improvement Measurements. Although optimization measurements help show how to optimize a given process, they do not show how to redesign a process to achieve new levels of productivity. A process improvement measurement can lead to a root-cause assessment that can identify the source of recurring quality problems. This can lead in turn to identification of new product and process designs, such as the redesign of a printed circuit board to consolidate components and increase reliability or of an automated production line to reduce manufacturing process steps.

WHAT TO MEASURE

Even when the benefits of measurement are understood, the question remains which measurements should be taken. A list of candidate measurements used by many organizations is provided in Table I. These are based on recommendations provided in ISO 9004:2000 and on the best practices of industry leaders.²

Table 1. Candidate measurements and metric types.

Manufacturing. The most common measurements used for the manufacturing process are related to production planning and productivity evaluation. Other measurements include evaluation of process optimization factors such as yield (yield data include the number of failures found in each step of the manufacturing process). Even though production capacity may be acceptable, a low yield points to process inefficiencies that may cause a high scrap level and require significant rework. Mikel Harry describes these inefficiencies as a "hidden factory," representing the wasted capacity spent producing bad product.³ Process measurements identify inefficient processes. In addition, analysis of defects (nonconforming products) found during manufacturing can reveal opportunities for process improvement. Tracking manufacturing trends such as process capability values over time also can be helpful in evaluating the long-term effectiveness of process improvement efforts.

Engineering. Engineering is often neglected when process measurements are discussed. The engineering process, the argument goes, is a creative one and should not be subject to the types of measurements used in manufacturing. But while it may be true that manufacturing measurements are not appropriate for engineering, it does not follow that no measurements should be performed.

For most companies, the biggest demand placed on engineering is to reduce time to market for new products. This can be accelerated by selecting the appropriate measurements. Just like the "hidden factory" in manufacturing that wastes resources, engineering resources are often wasted because the customer requirements are not well defined before the design process begins, or because the requirements change after significant design progress has been made. Using measurements to recognize the factors that contribute to delays is the first step in improving the engineering process. In addition to a time-to-market measurement for each project, many organizations also use productivity measurements for engineering teams (such as engineering change orders completed per month or tested software releases per month).

The most beneficial measurement to help improve the engineering process comes from an analysis of design defects found during the engineering process and by customers after release. Another key process improvement measurement involves assessing the product development life cycle to identify the phase where the majority of time is spent. Improvements in this phase offer the greatest potential for shortening the overall development schedule.

Suppliers. With the increasing complexity of today's products, companies are becoming more and more reliant on suppliers. With this increasing reliance on suppliers comes heightened risk that inferior products or services from suppliers may hurt product quality and profits. One appliance manufacturer, for example, traced 75% of warranty failures to the inferior quality of components from suppliers.⁴ Effective measurement of suppliers is essential not only to ensure quality of products but also to improve productivity. Traditional measurements for suppliers include the number of suppliers that are certified, the number of components that can be accepted without inspection (dock-to-stock), and the number of defects related to supplied components. In addition, analysis of the cause of defects that occur in supplied parts can be shared with suppliers as the basis for implementing a process improvement program. Supplier trend data also helps in selecting suppliers for long-term contracts and preferred contract awards.

Customer Satisfaction. Customer satisfaction is specifically identified as requiring measurement in ISO 9001:2000 section 8.2.1. Although such measurement is widely used as a gauge of product quality, it is also an excellent measurement of profitability. Customer satisfaction studies have shown that a 5% increase in customer retention can translate into a 20% improvement in profits.⁵ For this reason, trending of customer satisfaction and analysis of customer complaints to identify the root cause of problems have become widespread performance indicators.

MEASUREMENTS FOR PROCESS IMPROVEMENT

This discussion of measurements has focused primarily on evaluating conformity to established quality and productivity goals. These measurement systems can also provide the basis for defining new, enhanced processes. Process measurement systems that address general causes of defects can highlight the vital few defects that are the source of the majority of quality problems.⁶ (This strategy of ranking defects in order to decide which quality improvement projects to pursue is also known as the Pareto principle, see Figure 2.) For most companies, this is one of the most significant benefits of a measurement process. More detailed knowledge of the cause of quality problems provides a better understanding of the best strategies for correction. As a process matures, the attributes that identify failures can be expanded so that the remaining problems and their causes may be addressed.

Another method for conducting a more detailed analysis of measurements to better define quality improvement programs is a cost-of-quality calculation. The cost of quality represents the difference between the actual cost of a product or service and what the cost would be without product failures, manufacturing defects, or substandard service.⁷ The cost of quality is calculated as the sum of the cost of external failures (customer complaints), internal failures (failures in manufacturing and engineering), appraisal costs (costs for reviews and testing), and prevention costs (costs for process improvements and training). These costs can be calculated by assessing the average cost for each category and multiplying that cost by the number of instances of the failures or prevention activities reported (cost of appraisal is normally a fixed cost based on the resources dedicated to the activity). Measurement of the cost of quality is an excellent way to gain understanding of the factors that contribute to product quality and production costs. Calculation of the cost of quality frequently clarifies the significant benefits that can be realized from investment in preventive action programs.

PRINCIPLES OF MEASUREMENT

With the emphasis on the measurement of processes there is a risk that many ineffective types of measurements may be created. The result can be processes that become more complicated instead of more efficient. The following are some guidelines on how to define and use measurements in a way that minimizes risk and maximizes benefits.

CONCLUSION

There are no established standards for acceptable measurement levels for the majority of quality system processes. Although some candidate measurements have been published for productivity and quality levels, these measurements must be considered in light of the target industry and manufacturer. The best answer to measurement standards is benchmarking other companies, preferably the leaders in the industry. Benchmarking is an excellent way not only to establish measurements for excellence but also to learn new process improvement techniques.⁸

ISO 9001:2000 puts the onus on each manufacturer to define its internal processes and to develop measurements that ensure that these processes are effective. While adjusting to the new standard will be challenging for many manufacturers, the end results will pay them back many times over.

REFERENCES

1. ISO 9001:2000, Quality management systems—requirements (Geneva: International Organization for Standardization, December 13, 2000), v.
2.ISO 9004:2000, Quality management systems—Guidelines for performance improvements, (Geneva: International Organization for Standardization, December 13, 2000), 34–45.
3.Mikel Harry and Richard Schroeder, Six Sigma: The Breakthrough Management Strategy Revolutionizing the World's Top Corporations (New York: Doubleday, 2000), 79.
4.JA Donovan and FP Maresca, "Supplier Relations," in Juran's Quality Handbook, 5th ed. (New York: McGraw-Hill, 1999), 21.5.
5.Frederick F Reichheld, The Loyalty Effect, The Hidden Force Behind Growth, Profits, and Lasting Value (Boston: Bain, 1996), 13.
6.JM Juran and FM Gryna, Quality Planning and Analysis, 3rd ed. (New York: McGraw-Hill, 1993), 47.
7.Jack Campanella, Principles of Quality Costs, 3rd ed. (Milwaukee: ASQC Quality Press, 1999), 5.
8.Robert C. Camp, ed., Global Cases in Benchmarking. (Milwaukee: ASQ Quality Press, 1998), 7.

Copyright ©2001 Medical Device & Diagnostic Industry