Is Your Current Benchmarking Credible?
Benchmarks were used for hundreds of years by carpenters and metalworkers as a means of ensuring that a notch carved on the bench by the master would act as a guide for the apprentice in cutting the optimum length of a piece of wood or metal. Thus, marking the bench became benchmarking.

A benchmark is a point of reference or a criterion of quality. It is intended to serve the user as a guide for measuring optimum performance or to suggest solutions to problems or deficiencies. A benchmark per se implies the best practice.

Internal and comparative benchmarking are important activities in laboratories in that they:

	provide the means for longitudinal assessment of key performance parameters within a laboratory during times of change and
	give a manager a perspective on performance in comparison with cognate laboratories.

Internal and comparative benchmarks are useful in identifying specific deficits in performance that may require correction. They can be used also for baseline assessments prior to initiating planned changes so that their impacts and outcomes can be assessed and their specific effectiveness measured. The most valuable benchmarking tools are those that can be used to compare laboratories in reference to a national database as well as to measure internal performance within a region or a system of institutions. However, benchmarking can be effective only if the results are truly credible.

The benchmarking process in laboratory practice is used to identify, by measurable indices, the best performance to serve as models for others to emulate. The benchmark also indicates that a certain performance is achievable in given, defined circumstances.

The comparative method generally uses a selection of matched laboratory peers that represent truly comparable environments and conditions and then presents their benchmark for a particular performance index as the model or "stretch target."

Thus, two issues that are critical to a successful benchmarking process are:

	the accuracy of the performance measures themselves and
	the comparability of the selected peers.

Most benchmarking services meet the first of these criteria. The second is far more problematic.

An extreme example illustrates the point: Two labs match each other with respect to volume of tests performed, urban location, nonteaching status and size of institution. Both are nonunionized and have similar compensation policies; both have a comparable degree of automation. Yet, their mean cost per test is at opposite extremes of the spectrum. The self-evident conclusion might be that the low-cost institution is very efficient, and the high cost one is not. The critical differentiating factor may be, in fact, the test menu. The more complex or esoteric the menu, the higher the cost is likely to be. Therefore, it becomes critical to ask how a benchmarking source deals with the criterion of complexity (esotericism) as part of peer selection. There are two prevalent approaches:

	The first is an approximation that says, for example: A lab with flow cytometry is more complex than one without it, or one with both flow cytometry and molecular diagnostics is still more complex.
	The second aggregates all CPT codes of all performed tests, applies a relative value (RVU) to each test and calculates the total units performed in a given period, leading up to the assignment of a complexity index for the aggregate complexity.

By using the second approach, any guessing about the level of complexity is eliminated, and the peer comparison can be conducted on an "even playing field."

Ideally, peer comparison is truly accurate only if the two processes are absolutely identical in the peer and subject settings. This is extremely difficult to achieve when dealing with an entire laboratory or even with a section of the laboratory, such as chemistry or microbiology, or a stat lab and a core lab. In the final analysis, the comparison becomes almost perfect at the level of an individual workstation where the comparability can be truly unequivocal (i.e., the equipment and staffing at a workstation are identical, all other factors being equal).

Thus, in evaluating the credibility of a benchmarking process, it is imperative that the method of peer selection and data comparability is clearly understood by the manager of a laboratory before placing any reliance on the benchmark as a target for attaining a best practice performance.

The comparison based on workstations must be left to the future when more consistent and reliable information on the cost of supplies and technology will be available.

Jan W. Steiner, MD, FCAP, FRCP(C), Senior Vice President, Chi Laboratory Systems (CLS), is a pathologist with a long record of consulting and management in research, planning, and operations of laboratories and academic departments of pathology.

CLS has experience in measuring laboratory performance in comparison with CLS benchmarks for matched peer institutions. CLS has over 400 hospital, clinic, and independent laboratories in this database, which represents a broad cross section of laboratory sizes and types throughout the United States and Canada.