Large-scale empirical optimisation of statistical control charts to detect clinically relevant increases in surgical site infection rates - PubMed

Large-scale empirical optimisation of statistical control charts to detect clinically relevant increases in surgical site infection rates - PubMed

Large-scale empirical optimisation of statistical control charts to detect clinically relevant increases in surgical site infection rates

Iulian Ilieş 1, Deverick J Anderson 2 3, Joseph Salem 1, Arthur W Baker 2 3, Margo Jacobsen 1, James C Benneyan 4 5

Affiliations 

• PMID: 31704893
 
• DOI: 10.1136/bmjqs-2018-008976

Abstract

Objective: Surgical site infections (SSIs) are common costly hospital-acquired conditions. While statistical process control (SPC) use in healthcare has increased, limited rigorous empirical research compares and optimises these methods for SSI surveillance. We sought to determine which SPC chart types and design parameters maximise the detection of clinically relevant SSI rate increases while minimising false alarms.

Design: Systematic retrospective data analysis and empirical optimisation.

Methods: We analysed 12 years of data on 13 surgical procedures from a network of 58 community hospitals. Statistically significant SSI rate increases (signals) at individual hospitals initially were identified using 50 different SPC chart variations (Shewhart or exponentially weighted moving average, 5 baseline periods, 5 baseline types). Blinded epidemiologists evaluated the clinical significance of 2709 representative signals of potential outbreaks (out of 5536 generated), rating them as requiring 'action' or 'no action'. These ratings were used to identify which SPC approaches maximised sensitivity and specificity within a broader set of 3600 individual chart variations (additional baseline variations and chart types, including moving average (MA), and five control limit widths) and over 32 million dual-chart combinations based on different baseline periods, reference data (network-wide vs local hospital SSI rates), control limit widths and other calculation considerations. Results were validated with an additional year of data from the same hospital cohort.

Results: The optimal SPC approach to detect clinically important SSI rate increases used two simultaneous MA charts calculated using lagged rolling baseline windows and 1 SD limits. The first chart used 12-month MAs with 18-month baselines and best identified small sustained increases above network-wide SSI rates. The second chart used 6-month MAs with 3-month baselines and best detected large short-term increases above individual hospital SSI rates. This combination outperformed more commonly used charts, with high sensitivity (0.90; positive predictive value=0.56) and practical specificity (0.67; negative predictive value=0.94).

Conclusions: An optimised combination of two MA charts had the best performance for identifying clinically relevant small but sustained above-network SSI rates and large short-term individual hospital increases.

Keywords: Infection outbreaks; statistical process control; surgical site infections; surveillance.

© Author(s) (or their employer(s)) 2020. No commercial re-use. See rights and permissions. Published by BMJ.

Conflict of interest statement

Competing interests: None declared.

Comment in

• Applying rigour to the interpretation of surgical site infection rates.

  Williams V, Leis JA.BMJ Qual Saf. 2020 Jun;29(6):446-448. doi: 10.1136/bmjqs-2019-009964. Epub 2019 Dec 13.PMID: 31836626 No abstract available.

Similar articles

• Performance of statistical process control methods for regional surgical site infection surveillance: a 10-year multicentre pilot study.

  Baker AW, Haridy S, Salem J, Ilieş I, Ergai AO, Samareh A, Andrianas N, Benneyan JC, Sexton DJ, Anderson DJ.BMJ Qual Saf. 2018 Aug;27(8):600-610. doi: 10.1136/bmjqs-2017-006474. Epub 2017 Nov 24.PMID: 29175853 Free PMC article.
• Postoperative infection in spine surgery: does the month matter?

  Durkin MJ, Dicks KV, Baker AW, Moehring RW, Chen LF, Sexton DJ, Lewis SS, Anderson DJ.J Neurosurg Spine. 2015 Jul;23(1):128-34. doi: 10.3171/2014.10.SPINE14559. Epub 2015 Apr 10.PMID: 25860519 Free PMC article. Clinical Trial.
• Improving Surveillance for Surgical Site Infections Following Total Hip and Knee Arthroplasty Using Diagnosis and Procedure Codes in a Provincial Surveillance Network.

  Rusk A, Bush K, Brandt M, Smith C, Howatt A, Chow B, Henderson E.Infect Control Hosp Epidemiol. 2016 Jun;37(6):699-703. doi: 10.1017/ice.2016.53. Epub 2016 Mar 28.PMID: 27018968
• Intracavity lavage and wound irrigation for prevention of surgical site infection.

  Norman G, Atkinson RA, Smith TA, Rowlands C, Rithalia AD, Crosbie EJ, Dumville JC.Cochrane Database Syst Rev. 2017 Oct 30;10(10):CD012234. doi: 10.1002/14651858.CD012234.pub2.PMID: 29083473 Free PMC article. Review.
• Comparison of control charts for monitoring clinical performance using binary data.

  Neuburger J, Walker K, Sherlaw-Johnson C, van der Meulen J, Cromwell DA.BMJ Qual Saf. 2017 Nov;26(11):919-928. doi: 10.1136/bmjqs-2016-005526. Epub 2017 Sep 25.PMID: 28947635 Free PMC article. Review.