How to evaluate your metric

By Duke Rohe

Performance Improvement Specialist

M.D. Anderson Cancer Center

Houston

- Can it answer the question "How do we know we got there?"

- Can the measure be traced to a strategic goal?

- Is it representative of the entire population?

- Is it statistically sound?

- Can it be electronically collected?

- Is the sampling a representative alternative to 100% collection?

- How often does it need to be resampled or rerun?

- Does it measure the entire process?

- Are there any inferences in the measure that need explanation?

- If there is not a direct measure, is there an indirect correlative alternate?

- If it’s a cycle time, are the start and the stop points clear?

- Who commits to collect and report the measure, and at what frequency?

- Who needs to receive the measure?

- How "fresh" does it need to be?

- Does it require interpretation to have meaning? If so, who will do it?

- Is it a process component piece of a larger measure?

- How can you ensure integrity of the measure over time?

- Can the measure be more useful as a flag in an exception report?

- Are special causes (out of limit points) fully explained?

- Is confidentiality appropriately secured?

- Can masking the data be more fitting, more acceptable?

- Does it cause action or provide assurance?

- Are the parameters known for when to stop the measure?

- Is the cost of producing the measure justified?

- Is the measure clear to all, or does it need an explanation in the graph?

- Is the data source(s) listed on the graph?

- Could other people/departments gain from the measure?