An evaluation framework for ambient digital scribing tools in clinical applications

https://www.nature.com/articles/s41746-025-01622-1 AI-powered ambient digital scribing (ADS) tools can reduce documentation burdens and give clinicians more time to focus on patient care. Two recent AHRQ-funded studies offer practical methods for evaluating these tools to ensure they’re integrated safely and effectively into clinical workflows. A study in the Journal of the American Medical Informatics Association demonstrated the use of simulation testing to compare two ADS tools. Researchers analyzed transcripts from 11 real patient encounters and found that notes generated by one model took significantly longer to edit and contained more errors in text placement and additions—though it had fewer omissions. The findings suggest simulation testing can help clinicians identify which ADS tools best meet their needs. Access the abstract. https://pubmed.ncbi.nlm.nih.gov/40116912/ A study in NPJ Digital Medicine introduced a structured framework—SCRIBE (simulation, computational metrics, reviewer assessment, and intelligence evaluation)—for assessing ADS tools. Researchers developed their own ADS tool and tested it on recordings from 40 clinical visits. The framework identified strengths such as clarity and completeness, and flagged challenges including transcription errors, unlikely case scenarios, and possible demographic bias. The authors note that SCRIBE may offer a scalable, resource-conscious way to evaluate ADS tools. Access the article.