Beyond the Gantt Chart: New Methods for Measuring Product Development

Originally Published MDDI January 2003

PRODUCT DEVELOPMENT INSIGHT

Deliverable-based metrics offer a product development team objective measures of project progress—and the focus needed to get the job done.

David Warburton

Many new product development programs fail to finish on time. That observation is not a surprise to anyone. What might be a little more surprising, however, is that the project manager often does not recognize (or admit) that the schedule isn't being met until very late in the project. And what is very surprising is that the project manager's subsequent estimates of completion may be no more accurate than the initial estimate.

Projects fall behind schedule for countless reasons, many of them unavoidable. But the reasons that a project manager doesn't foresee the schedule slips—and then cannot predict when that same project will finish—are usually the same. In these cases, the Gantt chart, that ubiquitous tool of the project manager, is not providing accurate enough information about a project's progress against schedule.

Gantt's Limitations

Henry Gantt developed his chart in the beginning of the twentieth century to schedule textile workers more efficiently, and it has become an almost universal tool for displaying the schedules of engineering projects. For all its utility, however, a Gantt chart—and the commercial scheduling software packages that create it—does not easily allow a project manager to measure progress.

There are two reasons for this. First, it can be difficult to accurately measure the progress of even a single task using the chart; second, the methods used by commercial packages to calculate and report progress are not always transparent to the user.

The reason that it is difficult to measure the progress of even a single task lies in the nature of the product development process itself. For a medical device, the typical product development cycle looks like Figure 1. The transition from one phase of the project to the next is gradual, and there is typically considerable overlap between the efforts performed by each functional group. The same holds true for individual tasks. Unlike a task performed by an individual textile worker in a turn-of-the-century factory, tasks in a new product development cycle usually show a gradual increase in activity leading to a period of maximum effort; this is followed by a gradual, hyperbolic transition to the next task.

Such a task cycle can create difficulties in determining progress. For example, when an electrical engineering manager confidently states that a particular design task is "80% done," the last 20% of the effort may well take as much time as did the first 80%. Thus, from a timeline perspective, this means that the task is actually only 50% done. Given that the manager's estimate is probably only accurate to ±10%—not to mention that the engineering manager and project manager may be defining the word done in two completely different ways—the difficulty in accurately assessing progress on even a single task becomes obvious.

Now, consider that a typical new product development schedule may have more than 100 tasks. In order to determine progress against the overall schedule, the progress estimates for each task must be summed to give an overall progress estimate. Commercial scheduling products do allow the manager to "roll up" the progress on groups of tasks to generate a summary progress estimate; however, the methods used to generate that number are not necessarily transparent, and they cannot be easily modified by the user.

A Deliverable-Based Approach

Frustrated with the limited ability of commercial scheduling packages to help me measure progress against schedule, I began to look for other methods. Ultimately, the approach I took focused on a program's deliverables rather than its tasks. Deliverables are easy to count and are either complete or incomplete, eliminating the inherent subjectivity of the progress estimate. Furthermore, progress metrics based on deliverables can be compiled and calculated in a transparent, easy-to-understand manner.

While all phases of the development program can benefit from deliverable-based methods of progress measurement, this article focuses on two methods I have used successfully to measure the design engineering effort. For most projects, the engineering design phase requires the largest amount of time and resources.

The most basic deliverable metric is to count the number of parts released to the manufacturing resource planning bill of materials (BOM). Ultimately, in order to release a product to manufacturing (or to begin validation), all the product's parts must be released to a BOM through an engineering change order (ECO). I plot both the numbers of parts released each week and the cumulative numbers of all parts released. Then, based on data for similar products, I estimate total parts count and use that to forecast when the engineering effort will be complete (see Figure 2).

The method of counting the released parts works well; the progressive addition of parts to the BOM creates a large number of distinct, easy-to-measure events. Most manufacturing resource planning or product data management systems provide a query that will report the number of parts in an indented BOM, so calculating the number is simple and objective—there is no room for creative interpretation.

This metric's most powerful advantage is that it graphically illustrates the effects of resource limitations. It also provides some indication of one of two things: how many additional resources are required, or when the project will finish with the existing resources.

The metric also has two major limitations. First, it is only effective on products with high part counts. For example, I did not find it useful when developing implants, which often consist of only a single part. The other limitation is that the metric is only useful in the latter stages of the product development cycle. It provides no information during the (often long) period before the first part is released.

So I began to use a more sophisticated metric.

A New Metric

This method involves identifying between 20 and 50 key components or assemblies in the product. This is possible in all but the simplest projects. The information can usually be obtained from a product's engineering family tree. Next, the project leader identifies the stages that each of those components must pass through before it is complete. Typically, each component will go through these phases:

Again, every organization approaches product development differently, but it should be possible to identify four to five distinct phases that each component of the product passes through.

To determine progress, the project leader can determine the number of components that have completed each phase, then plot the progress of completion week by week (see Figure 3).

There are many advantages to using this metric over the simple part count. Even on relatively simple projects with less than 100 parts, there is enough resolution in the method to measure progress as frequently as every week. Progress can be measured very early in the project, long before the first part is released to a BOM. Although the method is more complicated to set up and maintain, it does provide a useful discipline. In order to complete the weekly tally, the project manager is forced to check the progress of each of the key components. This ensures that no deliverable is overlooked.

Other Metrics

Once project managers embrace the concept of tracking tangible deliverables, they will quickly find other deliverables to measure as well. The measurement of success in the one-part implant, for example, was completing the process validation on time. The same techniques can be applied. First, identify the tangible deliverable: in this case, the completed, signed-off validation report. Then identify the phases each validation report must pass through before completion. Third, chart weekly (see Figure 4).

Conclusion

The Gantt chart and the critical path method of scheduling have proven to be useful tools to the project manager. Like all tools, however, they have limitations. Deliverable-based metrics complement the Gantt chart because they provide objective measures of progress that are based on tangible items, such as documents and drawings. When deliverable-based metrics are used, people have greater confidence in the schedule forecasts because they are based on objective data, not opinion. The data, because they are easily normalized, can also be used to compare the performance of different projects, and can be used as a scheduling aid for new projects, which has typically been a difficult thing to do.

Finally, and most importantly, the metrics help the development team focus on the most fundamental objective of the project—completing the hundreds of necessary documents required to release a new product on time.

Copyright ©2003 Medical Device & Diagnostic Industry