National Quality Measures Clearinghouse | Expert Commentaries: Making Clinical Diagnoses: How Measureable Is the Process?

National Quality Measures Clearinghouse | Expert Commentaries: Making Clinical Diagnoses: How Measureable Is the Process?

May 5, 2014

New This Week

Making Clinical Diagnoses: How Measureable Is the Process?

By: Robert El-Kareh, MD, MPH

A timely, safe and reliable diagnostic process is at the heart of high-quality medical care. Providers and health systems have worked to reduce errors in treatment; however, if the diagnosis is incorrect, even "appropriate" treatment may not help (and may even harm) the patient. Diagnostic errors are common, with an estimated prevalence of ~10% to 15% (1), and remain at or near the top of the list of contributors to patient harm and medical litigation. (2) These factors create compelling motivation to establish reliable quality metrics for diagnosis in order to facilitate improvement.

When considering measures for the diagnostic process, it proves helpful to consider what an ideal process would look like. The goal of the diagnostic process is not necessarily to establish a definitive diagnosis at any cost. Instead, like all of medicine, it requires a balance between benefits versus risk and cost. In essence, the goal of a more definitive diagnosis should be pursued when the risks and costs of an inaccurate or delayed diagnosis (e.g., poor outcome because of failure to provide timely, effective treatment) outweigh the risks and costs of further investigation (e.g., medical complications of diagnostic procedures, incidental findings that lead to needless procedures, treatments, side effects, and anxiety). (3)

For example, all physicians would agree with the need to evaluate a patient for acute coronary syndrome if he or she presents with acute chest pain and has risk factors for coronary artery disease. However, if high-risk causes of the chest pain (e.g., acute coronary syndrome, aortic dissection, pulmonary embolism, pneumothorax) are effectively ruled out, the remaining potential causes of the chest pain may include low-risk diagnoses that we need not distinguish among, such as muscle strain or costochondritis. In this scenario, many would consider actively pursuing a more definitive diagnosis to be over-testing and instead recommend empiric symptomatic therapy. (4)

A National Quality Forum (NQF)-endorsed measure of the proportion of patients with low back pain without "red flag" features who did not undergo imaging in the first 28 days of symptoms is an example of how this risk-benefit balance can be represented in a measure. For this population, the risks and costs of imaging low back pain in the first 28 days would very likely outweigh the benefits gained from these tests. Therefore, given the known natural history of low back pain without high-risk features, aggressive diagnostic workup in the first several weeks is considered inappropriate. (5)

Currently, there is a relatively small number of quality measures specifically aimed at the diagnostic process. Within the current Centers for Medicare & Medicaid Services (CMS) inventory (6), one main category of these measures is geared toward diagnostic screening of asymptomatic patients (e.g., colonoscopy, mammogram, fasting blood glucose, diabetic complications, etc.) and follow-up of related test results. In these scenarios, the denominators (number of people in the population needing testing) and numerators (number of people who actually underwent the test) for the measures are relatively straightforward to calculate. While generally feasible, calculation of numerators of this type using electronic data can sometimes be limited by incomplete capture of patient risk factors and screening tests performed at outside facilities. Another main category of the CMS measures focuses on test utilization in specific diagnoses, such as use of a group A streptococcal culture when prescribing antibiotics for pharyngitis in children or the use of imaging in low back pain as mentioned above. Outside of these categories, a large part of the clinical diagnostic process remains unmeasured.

Despite these examples, a significant challenge to creating quality measures for diagnosis for many symptoms and conditions lies in the difficulty in recreating the diagnostic risk-benefit analysis specific to a particular patient at a given time. This information is necessary to establish the "gold standard" against which a clinician or health system's performance will be measured. For most symptoms, we lack a clear set of "red flag" findings to separate high- and low-risk etiologies, and clinical judgment is used instead. For patients presenting with these symptoms, the factors affecting the clinician's risk-benefit calculation often involve several steps of the diagnostic process. These include eliciting the patient history, performing the physical examination, analyzing prior test results, and constructing a set of potential diagnoses for consideration. In addition, the thresholds to employ "watchful waiting" vs. further testing vs. empiric treatment may be affected by the patient's and clinician's levels of comfort (or discomfort) with diagnostic uncertainty. Therefore, the most suitable diagnostic process becomes quite individualized. To retrospectively evaluate its appropriateness in specific cases, we depend, in large part, on the documentation of the different steps, especially the clinician's thought process. This documentation is often sparse and inadequate to reliably perform this assessment. Even when a diagnosis was first missed or delayed, and the correct one was eventually established, it can be very difficult to retrospectively determine whether the misdiagnosis or delay represented an avoidable error. (7)

The remedies for this challenge are not simple ones. At the very least, they would require substantially more complete and accurate documentation of patient's historical details, risk factors, diagnoses considered, and clinician diagnostic reasoning. Improving documentation of the various steps of the diagnostic process would also enable improved diagnostic decision support to help avoid errors in real time. However, prior to demonstration of a clear benefit, getting clinicians to devote time to enhance documentation will be difficult. Another remedy could be to improve our ability to properly assess the clinician's determination of the patient-specific risks and benefits of a diagnostic workup. Except in fairly constrained scenarios, this undertaking would be even more ambitious and complex, likely requiring novel research approaches on very large data sets to generate adequate "gold standard" risk and benefit assessments.

Since it does not currently appear feasible to create reliable quality measures assessing the appropriateness of someone else's diagnostic decisions in many cases, what can we do in the meantime? Providing outcome data directly to the people involved in the diagnostic decision-making seems a reasonable intermediate step. Currently, any type of feedback of diagnostic performance is quite limited, therefore preventing clinicians from effectively calibrating their decision-making. (8, 9) Our team within the Division of Biomedical Informatics at University of California (UC), San Diego has taken the approach to pursue a system that will provide clinicians with easy access to the diagnostic outcomes of their own patients using an electronic health record-based system. The clinicians themselves are familiar with the specific contexts of their diagnostic decisions, and, once the system is operational, the clinicians will be able to reflect on the appropriateness of those decisions given the diagnostic outcomes. While not as powerful as a completely independent assessment, this approach appears considerably more feasible in the short term.

In summary, our current ability to create diagnostic quality measures is quite limited. Scenarios that seem best suited for such measures include diagnostic screening in asymptomatic patients, test result follow-up, and symptoms and conditions with well-established and easily captured sets of criteria to distinguish high- and low-risk etiologies. Improved documentation may enable measures of various steps of the diagnostic process (e.g., whether the appropriate diagnoses were considered), but the current state of documentation would not make this reliable. As a next step, providing feedback from the outcomes of diagnostic workups directly to clinicians can help them calibrate their decision-making and assist in highlighting fruitful areas on which to focus diagnostic quality improvement efforts.

---

Authors

Robert El-Kareh, MD, MPH
Divisions of Biomedical Informatics and Hospital Medicine, University of California, San Diego, CA; La Jolla, CA

Disclaimer

The views and opinions expressed are those of the author and do not necessarily state or reflect those of the National Quality Measures Clearinghouse™ (NQMC), the Agency for Healthcare Research and Quality (AHRQ), or its contractor ECRI Institute.

Potential Conflicts of Interest

Dr. El-Kareh declared no conflicts of interest with respect to this expert commentary.

References

1. Graber ML. The incidence of diagnostic error in medicine. BMJ Qual Saf. 2013;22(Suppl 2):ii21-ii27.
2. Saber Tehrani AS, Lee H, Mathews SC, Shore A, Makary MA, Pronovost PJ, Newman-Toker DE. 25-Year summary of US malpractice claims for diagnostic errors 1986-2010: an analysis from the National Practitioner Data Bank. BMJ Qual Saf. 2013;22(8):672-80.
3. Almond SC, Summerton N. Diagnosis in general practice. Test of time. BMJ. 2009;338:b1878.
4. Mold JW, Stein HF. The cascade effect in the clinical care of patients. N Engl J Med. 1986 Feb 20;314(8):512-4.
5. Choosing Wisely. American Academy of Family Physicians: fifteen things physicians and patients should question. [internet]. Philadelphia (PA): ABIM Foundation; [accessed 2013 Dec 1]. Available: http://www.choosingwisely.org/doctor-patient-lists/american-academy-of-family-physicians/ .
6. Centers for Medicare & Medicaid Services (CMS). CMS measures inventory. [internet]. Baltimore (MD): Centers for Medicare & Medicaid Services; [accessed 2013 Dec 1]. Available: http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityMeasures/CMS-Measures-Inventory.html .
7. Schiff GD, Kim S, Abrams R, et al. Diagnosing diagnosis errors: lessons from a multi-institutional collaborative project. In: Henriksen K, Battles JB, Marks ES, et al., editors. Advances in patient safety: from research to implementation (volume 2: concepts and methodology). Rockville (MD): Agency for Healthcare Research and Quality; 2005 Feb.
8. Croskerry P. The feedback sanction. Acad Emerg Med. 2000 Nov;7(11):1232-8.
9. Schiff GD. Minimizing diagnostic error: the importance of follow-up and feedback. Am J Med. 2008 May;121(5 Suppl):S38-42.