Development and validation of a pragmatic natural language processing approach to identifying falls in older adults in the emergency department.

Patterson BW1,2, Jacobsohn GC3, Shah MN3,4, Song Y5,6, Maru A3, Venkatesh AK7, Zhong M3, Taylor K3, Hamedani AG3, Mendonça EA5,6,8.

Author information

1: BerbeeWalsh Department of Emergency Medicine, University of Wisconsin School of Medicine and Public Health, Madison, WI, USA. bpatter@medicine.wisc.edu.
2: Health Innovation Program, University of Wisconsin-Madison, Madison, WI, 53705, USA. bpatter@medicine.wisc.edu.
3: BerbeeWalsh Department of Emergency Medicine, University of Wisconsin School of Medicine and Public Health, Madison, WI, USA.
4: Department of Medicine, Division of Geriatrics, University of Wisconsin School of Medicine and Public Health, Madison, WI, United States.
5: Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, WI, USA.
6: Department of Pediatrics and Department of Biostatistics, Indiana University School of Medicine, Indianapolis, IN, USA.
7: Department of Emergency Medicine and Center for Outcomes Research and Evaluation, Yale University School of Medicine, New Haven, CT, USA.
8: Regenstrief Institute, Indianapolis, IN, USA.

Abstract

BACKGROUND:

Falls among older adults are both a common reason for presentation to the emergency department, and a major source of morbidity and mortality. It is critical to identify fall patients quickly and reliably during, and immediately after, emergency department encounters in order to deliver appropriate care and referrals. Unfortunately, falls are difficult to identify without manual chart review, a time intensive process infeasible for many applications including surveillance and quality reporting. Here we describe a pragmatic NLP approach to automating fall identification.

METHODS:

In this single center retrospective review, 500 emergency department provider notes from older adult patients (age 65 and older) were randomly selected for analysis. A simple, rules-based NLP algorithm for fall identification was developed and evaluated on a development set of 1084 notes, then compared with identification by consensus of trained abstractors blinded to NLP results.

RESULTS:

The NLP pipeline demonstrated a recall (sensitivity) of 95.8%, specificity of 97.4%, precision of 92.0%, and F1 score of 0.939 for identifying fall events within emergency physician visit notes, as compared to gold standard manual abstraction by human coders.

CONCLUSIONS:

Our pragmatic NLP algorithm was able to identify falls in ED notes with excellent precision and recall, comparable to that of more labor-intensive manual abstraction. This finding offers promise not just for improving research methods, but as a potential for identifying patients for targeted interventions, quality measure development and epidemiologic surveillance.