Linking Evidence Reviews to Organizational Guideline Planning: A Pilot Test of an Interactive, Web-Based Presentation and Discussion of Evidence

Related Files

These items are available in PDF only (Full Report [4.6 MB]; Tool [4.2 MB]). People using assistive technology may not be able to fully access information in these files. For additional assistance, please contact us.

This report is from AHRQ's series on Health Systems Partnership Pilot Project Reports. These reports describe the efforts of Evidence-based Practice Centers (EPCs) to work with health care decisionmakers and facilitate the use of information from AHRQ EPC evidence reports.

Purpose of Review

We pilot-tested the effectiveness of a cyberseminar (an interactive, web-based presentation and discussion of evidence) in integrating systematic review findings into organization-specific clinical guidelines within a large, integrated health system. This project was undertaken in collaboration with key partners at the Kaiser Permanente Care Management Institute. Our core learnings include:

Key Messages

Dynamic presentation formats can simultaneously address the evidence needs of high-level health system decision-makers and those working to implement clinical guidelines.
Timing and organizational readiness are critical to the adoption of evidence.
There is a need for evidence to be placed into a larger decisional context for optimal use by health systems.

Structured Abstract

Objectives. The goal of the Clinical Operations Evidence Review cyberseminar pilot project was to develop and test a method for facilitating the adoption and implementation of an EPC report's findings into a health system's clinical guideline.

Methods. This project was a collaboration with key partners at the Kaiser Permanente Care Management Institute (CMI). Over several months, we developed, implemented, and evaluated a 1-hour interactive, web-based presentation and discussion of evidence on interventions to prevent (or delay) the onset of diabetes. Through phone interviews and an online survey, we evaluated the content and usefulness of the EPC report to inform a clinical operations guideline and implementation process, the utility of the cyberseminar itself, and the extent to which this process was likely to inform decision-making at Kaiser Permanente.

Results. The cyberseminar: (1) targeted multiple disciplines and levels of leadership in the decision-making process, (2) engaged participants using an interactive rather than didactic (static) format, and (3) delivered the evidence in a context relevant to stakeholders. Stakeholders included members of Kaiser Permanente's diabetes guideline development team and national and regional implementation leaders for diabetes prevention efforts within Kaiser Permanente.

The cyberseminar was well received and served the needs of the guideline development team. The presentation focused on a high-level summary of the systematic review evidence; comparison of review findings with other systematic reviews; a description of implementation issues for included lifestyle interventions; a review of CMS reimbursement for lifestyle interventions; and a discussion about pre-identified considerations. Our key partners at CMI agreed that hearing from people working on implementation of diabetes prevention interventions in different regions was helpful, as was being able to query evidence reviewers during and after the cyberseminar. Guideline developers reported that the cyberseminar would change aspects of guideline and process.

Participants identified several areas EPC reports could address beyond effectiveness and harms that would be particularly helpful to health care organizations, including: (1) information about implementation and monitoring considerations for included interventions, (2) information on important subgroups, (3) (if applicable) information on how reports have been used to inform national guidelines, and (4) consistency between report findings and other existing systematic reviews.

Conclusions. Our pilot cyberseminar shows promise as a dynamic format to link evidence and evidence reviewers to organization-specific guideline development, and to integrate key stakeholders into the early guideline development process. The success of this effort required both the readiness of the health system and a partnership between evidence reviewers and the health system.

Citation

Suggested citation: Lin JS, Rubenstein LV, Beil TL, Hempel, S. Linking Evidence Reviews to Organizational Guideline Planning: A Pilot Test of an Interactive, Web-Based Presentation and Discussion of Evidence. Methods Research Report. (Prepared by the Kaiser Permanente Research Affiliates Evidence-based Practice Center under Contract No. 290-2015-00007-I and the Southern California Evidence-based Practice Center-RAND Corporation under Contract No. 290-2015-00010-I.) AHRQ Publication No. 18(19)-EHC025-EF. Rockville, MD: Agency for Healthcare Research and Quality. October 2018. Posted final reports are located on the Effective Health Care Program search page.
DOI: https://doi.org/10.23970/AHRQEPCMETHENGAGELINKING