April 30, 2012
The maturation of quality measurement has progressed to a varying degree across medical and surgical specialties. Surgical treatment of pancreatic cancer is complex, and quality measurement is particularly needed given the considerable variability in hospital performance and outcomes after pancreatic cancer surgery. Only recently, few opportunities existed for providers to assess the quality of their pancreatic cancer care, and there was no ability for hospitals to compare their processes and outcomes to other hospitals. A number of national initiatives now provide a variety of opportunities for clinicians/physicians and hospitals to examine the quality of pancreatic cancer care and identify areas for improvement.
Pancreatic cancer generally carries a dismal prognosis. Candidates for surgery comprise a small fraction of patients, as most present with unresectable locally advanced or metastatic disease. Surgery for pancreatic cancer, particularly the Whipple procedure for malignancies in the head of the pancreas, is complex, with high postoperative complication rates. Even when patients have a margin-negative resection and no lymph node involvement, long-term survival is poor. Numerous studies demonstrate considerable variation across hospitals in short-term outcomes, long-term survival, cost, process metrics, quality of life, and patient satisfaction after pancreatic surgery. (1-6) Quality assessment for pancreatic cancer focuses on those patients who undergo an operation, as these patients' outcomes are potentially modifiable.
Many have suggested that only allowing expert centers to perform pancreatic cancer surgery, so-called regionalization, presents the only solution to the variability in outcomes. Bilimoria and colleagues' (i.e., the research team's) recent work suggests that some patients (i.e., young and no comorbidities) may be able to safely undergo certain complex cancer surgeries (e.g., gastrectomy) at community hospitals with comparable mortality to that of high-volume, experienced centers; however, irrespective of age and comorbidities, pancreatic or esophageal cancer surgery comprise the only operations in which all patients had lower mortality rates if treated at high-volume, expert centers. (7) Recent evidence suggests that some regionalization occurs based on initiatives from payers, government oversight agencies, and private entities. However, considerable hospital-to-hospital variability in outcomes exists even among National Cancer Institute (NCI)-designated cancer centers treating pancreatic cancer. (8)
Assessing hospital quality of care using valid quality metrics has been suggested as a mechanism to reduce variability in performance, but no such metrics were available for pancreatic cancer surgery, despite the tremendous variability in outcomes for this complex operation. In 2007, the research team developed and tested quality indicators for pancreatic cancer using a formal methodology. (9) They identified candidate quality indicators from the literature, guidelines, and expert interviews. A multidisciplinary panel of pancreatic cancer experts ranked the potential indicators in two rounds with a detailed discussion of each indicator between rounds. Analysis of rankings facilitated the development of a set of variables with high and moderate validity. Indicators were combined to create a composite measure. Indicators encompassed all domains of care (Table 1 and Table 2). (9)
The research team assessed adherence with the composite measure at 1134 hospitals and found that more than half of hospitals were not adherent with 50% of the measures. The results demonstrated considerable variability in the quality of pancreatic cancer care in the United States, clearly showing significant room for improvement. Hospitals use these indicators to evaluate the quality of pancreatic cancer care they provide and identify opportunities for improvement.
Quality measures for pancreatic care are being added to a national program of measurement, benchmarking, and feedback. The American College of Surgeons (ACS) Commission on Cancer (CoC) is a multidisciplinary organization consisting of 49 member organizations. Each CoC hospital reports all of its cancer cases to the National Cancer Data Base (NCDB), and the NCDB provides hospitals with reports regarding their performance on various process measures. (10) The database will soon add outcome measures for 30-day mortality and five-year survival for pancreatic cancer. Thus, hospitals will be able to evaluate themselves in comparison to the other approximately 1,000 hospitals which perform pancreatic surgery.
While the CoC offers some information on pancreatic surgery process of care, no data are available on postoperative complications. A new initiative from ACS will allow hospitals to compare their pancreatic cancer surgery risk-adjusted short-term outcomes to other hospitals, including mortality, serious morbidity, surgical site infections, venous thromboembolism, pneumonia, length of stay, and readmission, as well as procedure-specific complications such as pancreatic anastomotic leak rate. (8) Based on standardized, validated clinical data through the American College of Surgeons National Surgical Quality Improvement Program (NSQIP©), a pilot program examining surgical cancer care has been initiated, ACS NSQIP Oncology. (11, 12) The ACS NSQIP (site.acsnsip.org ) started in 2001 and is now the largest and most prominent surgical outcomes program. (13, 14) ACS NSQIP, a unique quality improvement program, has had a long history of helping hospitals improve care and outcomes. (13, 15-17) The structure of ACS NSQIP, including sampling strategy, data abstraction procedures, variables collected, risk-adjustment methodology, and outcomes collected, has an extensive description. (13, 14, 16, 18-29) Briefly, the program prospectively collects detailed data regarding patient demographics, preoperative comorbidities and other risk factors, laboratory values prior to the surgical procedure, and certain operative variables. (13) Surgical certified reviewers (SCRs) abstract data at each site using standardized data definitions. Clinicians follow patients for complications for 30 days after the index operation. (30) ACS NSQIP then provides participating hospitals with risk-adjusted outcomes (morbidity and mortality) in comparison to the other participating hospitals. The results provided through ACS NSQIP focus on 24 postoperative events. Risk-adjusted surgical outcomes provide the basis for hospitals' rankings. When a hospital identifies that its outcomes are suboptimal compared with other hospitals in the country, that hospital can enact a targeted quality improvement initiative to achieve better outcomes. ACS NSQIP demonstrates reduced morbidity and mortality while reducing costs. (13)
Through the ACS CoC NCDB and ACS NSQIP Oncology, hospitals will be able to measure a broad range of metrics related to pancreatic cancer surgery and fundamentally improve the care of pancreatic cancer patients.
Karl Y. Bilimoria, MD, MS
Feinberg School of Medicine, Northwestern University, Chicago, IL
The views and opinions expressed are those of the authors and do not necessarily state or reflect those of the National Quality Measures Clearinghouse™ (NQMC), the Agency for Healthcare Research and Quality (AHRQ), or its contractor, ECRI Institute.
Potential Conflicts of Interest
Dr. Bilimoria reports the following business/professional interest: unpaid consultant to the American College of Surgeons.
- Bilimoria KY, Bentrem DJ, Feinglass JM, et al. Directing surgical quality improvement initiatives: Comparison of perioperative mortality and long-term survival for cancer surgery. J Clin Oncol. 2008 Jun 23.
- Birkmeyer JD, Gust C, Dimick JB, Birkmeyer NJ, Skinner JS. Hospital quality and the cost of inpatient surgery in the United States. Ann Surg. 2012 Jan;255(1):1-5.
- Birkmeyer JD, Siewers AE, Finlayson EV, et al. Hospital volume and surgical mortality in the United States. N Engl J Med. 2002 Apr 11;346(15):1128-1137.
- Bentrem DJ, Brennan MF. Outcomes in oncologic surgery: does volume make a difference? World J Surg. 2005 Oct;29(10):1210-1216.
- Gordon TA, Burleyson GP, Tielsch JM, Cameron JL. The effects of regionalization on cost and outcome for one general high-risk surgical procedure. Ann Surg. 1995 Jan;221(1):43-49.
- Rosemurgy AS, Bloomston M, Serafini FM, Coon B, Murr MM, Carey LC. Frequency with which surgeons undertake pancreaticoduodenectomy determines length of stay, hospital charges, and in-hospital mortality. J Gastrointest Surg. 2001 Jan-Feb;5(1):21-26.
- Bilimoria KY, Bentrem DJ, Talamonti MS, Stewart AK, Winchester DP, Ko CY. Risk-based selective referral for cancer surgery: A potential strategy to improve perioperative outcomes. Ann Surg. 2009 Nov 5.
- Bilimoria K, Wang X, Cohen ME, et al. Development of ACS NSQIP Oncology: A Surgical Quality Measurement System for cancer centers paper presented at: Society of Surgical Oncology Annual Cancer Symposium 2011; San Antonio, TX.
- Bilimoria KY, Bentrem DJ, Lillemoe KD, Talamonti MS, Ko CY. Assessment of pancreatic cancer care in the United States based on formally developed quality indicators. J Natl Cancer Inst. 2009 Jun 16;101(12):848-859.
- Bilimoria KY, Stewart AK, Winchester DP, Ko CY. The National Cancer Data Base: a powerful initiative to improve cancer care in the United States. Ann Surg Oncol. 2008 Mar;15(3):683-690.
- Merkow R, Bentrem DJ, Chow W, et al. Effect of cancer-specific variables on risk-adjusted hospital surgical quality rankings. Paper presented at: Society of Surgical Oncology 2012; Orlando, FL.
- ACS NSQIP: American College of Surgeons - National Surgical Quality Improvement Program. Available at: www.ACSNSQIP.org . (Accessed February 1, 2012).
- Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY. Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg. 2009 Sep;250(3):363-376.
- Hall BL, Richards K, Ingraham A, Ko CY. New approaches to the National Surgical Quality Improvement Program: the American College of Surgeons experience. Am J Surg. 2009 Nov;198(5 Suppl):S56-62.
- Khuri SF. The NSQIP: a new frontier in surgery. Surgery. 2005 Nov;138(5):837-843.
- Khuri SF, Daley J, Henderson W, et al. The Department of Veterans Affairs' NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program. Ann Surg. 1998 Oct;228(4):491-507.
- Khuri SF, Daley J, Henderson WG. The comparative assessment and improvement of quality of surgical care in the Department of Veterans Affairs. Arch Surg. 2002 Jan;137(1):20-27.
- Werner RM, Bradlow ET. Relationship between Medicare's hospital compare performance measures and mortality rates. JAMA. 2006 Dec 13;296(22):2694-2702.
- Daley J, Khuri SF, Henderson W, et al. Risk adjustment of the postoperative morbidity rate for the comparative assessment of the quality of surgical care: results of the National Veterans Affairs Surgical Risk Study. J Am Coll Surg. 1997 Oct;185(4):328-340.
- Khuri SF, Henderson WG, Daley J, et al. The patient safety in surgery study: background, study design, and patient populations. J Am Coll Surg. 2007 Jun;204(6):1089-1102.
- Davis CL, Pierce JR, Henderson W, et al. Assessment of the reliability of data collected for the Department of Veterans Affairs national surgical quality improvement program. J Am Coll Surg. 2007 Apr;204(4):550-560.
- Khuri SF, Henderson WG, Daley J, et al. Successful implementation of the Department of Veterans Affairs' National Surgical Quality Improvement Program in the private sector: the Patient Safety in Surgery study. Ann Surg. 2008 Aug;248(2):329-336.
- Dimick JB, Ghaferi AA, Osborne NH, Ko CY, Hall BL. Reliability adjustment for reporting hospital outcomes with surgery. Ann Surg. 2012 Mar 1.
- Raval MV, Cohen ME, Ingraham AM, et al. Improving American College of Surgeons National Surgical Quality Improvement Program risk adjustment: incorporation of a novel procedure risk score. J Am Coll Surg. 2010 Dec;211(6):715-723.
- Shiloach M, Frencher SK, Jr., Steeger JE, et al. Toward robust information: data quality and inter-rater reliability in the American College of Surgeons National Surgical Quality Improvement Program. J Am Coll Surg. 2010 Jan;210(1):6-16.
- Birkmeyer JD, Shahian DM, Dimick JB, et al. Blueprint for a new American College of Surgeons: National Surgical Quality Improvement Program. J Am Coll Surg. 2008 Nov;207(5):777-782.
- Cohen ME, Dimick JB, Bilimoria KY, Ko CY, Richards K, Hall BL. Risk adjustment in the American College of Surgeons National Surgical Quality Improvement Program: a comparison of logistic versus hierarchical modeling. J Am Coll Surg. 2009 Dec;209(6):687-693.
- Hamilton BH, Ko CY, Richards K, Hall BL. Missing data in the American College of Surgeons National Surgical Quality Improvement Program are not missing at random: implications and potential impact on quality assessments. J Am Coll Surg. 2010 Feb;210(2):125-139, e122.
- Ingraham AM, Richards KE, Hall BL, Ko CY. Quality improvement in surgery: the American College of Surgeons National Surgical Quality Improvement Program approach. Adv Surg. 2010;44:251-267.
- Bilimoria KY, Cohen ME, Ingraham AM, et al. Effect of postdischarge morbidity and mortality on comparisons of hospital surgical quality. Ann Surg. 2010 Jul;252(1):183-190.
|1||IF an institution performs pancreatic cancer surgery, THEN the institution should monitor their average annual case volume|
|2||IF an institution performs pancreatic cancer surgery, THEN the institution should monitor their surgeons' annual case volume|
|3||IF a patient undergoes resection for pancreatic cancer, THEN the patient should be treated in a multidisciplinary effort with a surgeon, medical oncologist, and a radiation oncologist|
|4||IF a patient undergoes resection, THEN the hospital must ensure that the surgeon is certified by the American Board of Surgery or equivalent international organization|
|5||IF an institution performs pancreatic cancer surgery, THEN the hospital should have interventional radiology services available on site|
|6||IF an institution performs pancreatic cancer surgery, THEN the hospital should have an intensive care unit staffed by critical care specialists|
|7||IF a patient undergoes resection, THEN a history and physical with thorough preoperative risk assessment should be performed|
|8||IF a patient is diagnosed with pancreatic cancer, THEN a stage-specific treatment plan should be documented|
|9||IF a patient is being considered for resection, THEN a triple-phase, multi-slice CT or MRI scan should be obtained|
|10||IF a patient undergoes cancer-directed resection, THEN clinical and pathologic stage should be recorded|
|11||IF a patient undergoes cancer-directed resection, THEN the tumor histology should be recorded|
|12||IF a patient undergoes cancer-directed resection, THEN the tumor size should be recorded|
|13||IF a patient undergoes cancer-directed resection, THEN the tumor grade should be recorded|
|14||IF a patient undergoes cancer-directed resection, THEN the margin status should be recorded|
|15||IF a patient undergoes cancer-directed resection, THEN the number of lymph nodes examined should be recorded|
|16||IF a patient undergoes cancer-directed resection, THEN the number of lymph nodes positive should be recorded|
|17||IF patient undergoes resection of a pancreatic head lesion, THEN in the operative note, the surgeon should document complete removal of all pancreatic tissue, lymph nodes, and connective tissue between the edge of the uncinate process and the right lateral wall of the superior mesenteric artery|
|18||IF a patient undergoes resection, THEN suspicious adenopathy outside the scope of planned resection should be evaluated by frozen section|
|19||IF a patient undergoes adjuvant therapy, THEN the timing relative to resection (before, after, both) should be recorded|
|20||IF a patient undergoes resection, THEN the College of American Pathologists checklist or equivalent reporting system should be followed and fully documented|
|21||IF a patient does not undergo resection, THEN a TNM clinical stage should be recorded|
|22||IF a patient has clinical stage I or II disease, THEN the patient should undergo resection or have a valid reason documented for not undergoing resection|
|23||IF a patient undergoes cancer-directed resection, THEN adjuvant chemotherapy with or without radiation should be considered or administered, or a valid reason should be documented for not receiving adjuvant therapy|
|24||IF a patient has clinical stage IV disease, THEN cancer-directed surgery should not be done|
|25||IF a patient does not undergo resection, THEN chemotherapy or chemoradiation should be considered or administered or a valid reason should be documented for not receiving non-surgical therapy|
|26||IF a patient is to receive treatment, THEN the time from diagnosis to surgery or first treatment should be less than two months|
|27||IF an institution performs pancreatic cancer surgery, THEN the institution should monitor their margin-negative resection rate|
|28||IF an institution performs pancreatic cancer surgery, THEN the hospital should monitor their pancreatic cancer resection risk-adjusted perioperative mortality|
|29||IF an institution performs pancreatic cancer surgery, THEN the hospital risk-adjusted perioperative mortality should be less than 5%|
*Based on strict validity criteria (≥ 90% of expert panel rankings in the 7-9 range). CT = computed tomography; MRI = magnetic resonance imaging; TNM = tumor-node-metastasis.Table 2. Moderate validity pancreatic cancer quality measures*
|30||IF an institution treats pancreatic cancer, THEN the institution should participate in clinical trials|
|31||IF an institution performs pancreatic cancer surgery, THEN the institution should perform ≥ 12 cases per year|
|32||IF an institution performs pancreatic cancer surgery, THEN the hospital should have endoscopic ultrasonography services available on site|
|33||IF an institution treats pancreatic cancer, THEN the institution should have radiation therapy and chemotherapy services available within their institution|
|34||IF an institution performs pancreatic cancer surgery, THEN the hospital should have ERCP services available on site|
|35||If a patient is to undergo resection, THEN on the basis of the CT or MRI scan, the surgeon should preoperatively document 1) no metastatic disease, 2) patent superior mesenteric vein and portal vein, and 3) a definable tissue plane between the tumor and regional arterial structures|
|36||IF a patient undergoes cancer-directed resection, THEN the margins should be macroscopically clear|
|37||IF a patient undergoes resection, THEN in the operative note, the surgeon should document intraoperative findings including the absence of 1) regional arterial involvement, 2) metastatic disease (liver, peritoneal, omental), and 3) distant adenopathy|
|38||IF a patient undergoes cancer-directed resection, THEN ≥ 10 regional lymph nodes should be resected and pathologically evaluated†|
|39||IF an institution performs pancreatic cancer surgery, THEN the institution should monitor their median estimated blood loss|
|40||IF an institution performs pancreatic cancer surgery, THEN the institution should monitor the median operative time for resections|
|41||IF an institution performs pancreatic cancer surgery, THEN the hospital should monitor their readmission-within-30-days rate|
|42||IF a patient undergoes resection, THEN the operative time should be less than 10 hours‡|
|43||IF an institution performs pancreatic cancer surgery, THEN the hospital should monitor the stage-specific 2-year and 5-year survival rates for their patients who underwent pancreatectomy|
*Based on relaxed validity criteria ≥ 95% of expert panel rankings in 4-9 range). ERCP = endoscopic retrograde cholangiopancreatography; CT = computed tomography; MRI = magnetic resonance imaging.
†The expert panel extensively discussed measures with multiple nodal count thresholds, but the only measure retained was for the resection and examination of ≥ 10 nodes. The measure for ≥ 12 nodes was also moderately valid but had lower median score; thus, the measure for ≥10 nodes was retained. The measure for ≥ 15 nodes was ranked as not valid.
‡The expert panel discussed a variety of time thresholds and thought that 8 hours would be a reasonable maximum, but they settled on 10 hours because many panel members believed that operative times greater than 8 hours would not be excessive.