In Conversation With… Amy J. Starmer, MD, MPH

Interview

Download: (01_April_2016_Starmer.mp3 | 9.12 MB | 6 minutes, 38 seconds )

Editor's note: Dr. Starmer is Director of Primary Care Quality Improvement and Assistant Professor of Pediatrics at Boston Children's Hospital and Harvard Medical School. She served as Project Leader for the I-PASS Handoff Study and currently leads an AHRQ-funded effort to disseminate I-PASS in 32 additional institutions. We spoke with her about handoffs and the implementation and findings of the landmark I-PASS study.

Dr. Robert M. Wachter: What got you interested in the topic of handoffs?

Dr. Amy J. Starmer: During my residency, I became involved in a series of national initiatives that were trying to improve residency training and graduate medical education systems. These experiences led me to pursue fellowship training, where I learned the importance of tying the assessment of educational interventions to meaningful patient outcomes. Patient handoffs ended up being a perfect topic to learn firsthand how to design and implement a curricular intervention for residents, how to develop improved care delivery systems, and how to rigorously assess the impact of these changes on patient outcomes and safety. I connected with Chris Landrigan and Ted Sectish, and we conducted a single-institution study aiming to improve handoffs of care for residents at Boston Children's Hospital. The success of that single-center study ultimately led to the multicenter I-PASS study.

RW: What did you learn from the single-center study that influenced what you did with the bigger study?

AS: Having that study as the foundation and an opportunity for testing the ideas was critically important to the ultimate success of the multicenter intervention. In the original study, we developed a bundle of multiple interventions that included training and new verbal and written handoff processes. The hypothesis was that each element of the bundle would work synergistically with each other to improve communication and reduce error rates. We ensured we had institutional support and involved the residents in the design of the intervention components. Medical errors dropped significantly after putting the bundle in place, and process measures improved as well. However, toward the end of the study, we started to notice that there was a trailing off of how much the residents were actually using the new handoff techniques. They also started to complain that the mnemonic used to standardize the verbal handoff process was really long, that they couldn't remember what all the letters were supposed to mean, and that the verbal handoff didn't mirror the new written computerized handoff tool.

RW: What was the mnemonic at the time?

AS: We used the SIGNOUT mnemonic, which had a reasonable evidence base behind it. But it was tricky. Even I had trouble remembering what all the letters stood for, which was not a good sign. Key lessons learned were that in order to ensure transformational change of something as culturally ingrained as handoff communication, there must be systems in place that will continue to drive ongoing improvement and sustainability after the initial intervention period. We came to learn that having enough faculty and frontline providers on board to engage in that process was going to be really important.

RW: You mentioned that after the pilot intervention was done—and I assume this was before the multicenter study started—you saw some backsliding. How did you address that at the original site?

AS: It worked out well because the original site ended up being somewhat of a testing ground for the I-PASS handoff intervention, which was essentially an enhanced version of the original bundle. Things that had worked well in the pilot study were further refined and incorporated into the I-PASS handoff bundle. We were able to work out the kinks in the new I-PASS curriculum prior to implementation across the research sites by leveraging the participation of Boston Children's Hospital as a non-data collection site. Following the success of the I-PASS research study, the Boston Children's residents were trained each year in I-PASS and now use I-PASS as their primary method of handoff communication.

RW: It's one thing to do a study in a single site where the investigators are around to do cheerleading. How did you build in at eight other sites? How did you both ensure quality control and ensure sufficient enthusiasm and buy-in to make sure that it worked?

AS: When we talk about the I-PASS study, one of the key things we often point to was the tremendously collaborative nature of the team that designed the curriculum and simultaneously carried out the research aims. Individuals from each site were designated to play key roles in developing the intervention and were assigned specific leadership roles to oversee various aspects of the project. Therefore, the sites were really invested throughout the entire phase of curriculum development through implementation and assessment. But even before this degree of involvement—and even before we knew that we were going to have the federal grant support to carry out the study—we asked that each site participating in the project obtain a letter of support from someone high up in the administration to say that they would provide $50,000 toward the project even if the grant funding didn't come through. We were really lucky that the grant funding did come through, and we were able to carry out the study with all the robust outcome measures and data collection that we ended up achieving. However, having support and buy-in from the outset was really important.

RW: Can you give us a little bit of the CliffsNotes of what I-PASS is about?

AS: From the single-site study, we knew it was important to consider what might be more effective in terms of a mnemonic to standardize the handoff communication process. Through an iterative process, we carefully itemized the key elements of the handoff process and reconsidered how these data elements could be communicated in a standard way for verbal and written handoff communication. We ultimately landed on I-PASS, which stands for Illness severity, Patient summary, Action list, Situation awareness and contingency planning, and then Synthesis by receiver. I-PASS is a standard structure embedded into all components of the intervention that serves as an organizing framework for the communication process. However, it's important to note that when I reference the I-PASS handoff intervention, I don't mean just the mnemonic. It is referring to the I-PASS handoff bundle, which is an entire program of several interventions packaged together, which we hoped would work synergistically with each other to make a significant dent in the problem of handoff miscommunications.

The I-PASS handoff bundle includes additional components including interactive training sessions for residents where simulations and role-play experiences afforded an opportunity to practice these new handoff skills and techniques. Each site also worked with IT leadership to develop a written handoff tool that was structured in an I-PASS format and integrated into the electronic medical record when possible. We had a robust faculty development campaign, where faculty engaged in regular observations of the handoff process to provide feedback to the junior doctors to improve their handoff skills. We referred to the final component of the intervention as "the campaign," which was an effort to promote handoff improvement in the everyday lives of the residents by using visual reinforcement strategies such as the development of a logo and a slogan, and putting up posters and screen frames to remind people about the effort.

RW: What did you have to do in terms of logistics, the timing of handoffs and when it happened, the amount of time allocated for it, the settings for them, etc.?

AS: Through the training and the curriculum, we emphasized the importance of having handoffs done in a quiet space with a team-based approach. The historical, traditional approach to handoff communication was to have interns signing out in one room and senior residents signing out in the other room. Despite the fact that providers at different levels own different types of information about those patients—if everybody's not together in the same room there's no opportunity to have information sharing and development of a shared mental model. So we emphasized that a team approach is a more ideal scenario. A big concern residents had was that the structured communication would take extra time. That was a key reason why we collected balancing measure data through a time-motion study where we assessed the impact of the intervention on workflow patterns. Of course, we ultimately found that there was not any significant difference in the amount of time that the handoff communications took before versus after the intervention.

RW: How do you explain that?

AS: The concern was that it was going to take a lot longer. What we found from the recordings of the handoff and all the observations that took place, handoffs typically were a time when there were conversations about the patients, but it was also an opportunity for more social or tangential conversations, especially early in training. The beauty of I-PASS is that it condenses that, gets rid of the nonessential information, but then has everything flow in a logical format. So the hope is that it likely actually ends up saving time overall because they're not having to spend time looking up things later that they wouldn't have been told.

RW: How important were the IT tools? You mentioned that sometimes it was integrated into the EMR, sometimes maybe not. Do people do it in front of a computer, and how important was that in getting this to work?

AS: Only two of the sites weren't able to integrate the handoff tool into their EMR. There's certainly still huge potential and room for improvement and ease in creating handoff tools. But we're moving more and more to a world of electronic documentation. It doesn't make sense to have doctors retyping pieces of demographic and medical information. During my own residency training, I can remember sitting in the workroom room and retyping medication lists, names, and room numbers, and feeling incredibly frustrated when it wasn't clear whether the patient information on the signout sheet was updated or not. There are so many chances for error with a system where patient data is not automatically pulled from the EMR and updated in real time. With the multisite study, we learned that there are still constraints in terms of how these systems are currently set up. Some tools were perhaps a bit more clunky than we would have liked, but EMR companies are starting to see the value and importance of having these types of tools available. I foresee that's going to continue to improve with time. From a learning perspective, so many of us are visual learners that having written information at hand to enhance the process of communication transfer is important to make sure the concepts are getting across, so having high-quality written handoff tools is a really important aspect of the bundle.

RW: Before we turn to outcomes, was this just a handoff done at shift change, morning handoffs of patients admitted at night, or does it also include handoffs someone finishing a rotation at the end of a month? Which was the focus here?

AS: For the multisite study, we focused on the resident physician end-of-shift handoff, which is the most formal handoff in the evening when the day team is going off shift and the night team is coming over to cover. That being said, early on we worked a lot with the leaders from TeamSTEPPS, thinking about ideal team leadership and team communication strategies. We felt it was important to address communication globally and not focus just on the evening handoff aspect. Additionally, although for handoffs we focused primarily on end-of-shift communication, we found that the structure of I-PASS is readily adaptable to other handoff types. For example, people have started using it frequently in unit-to-unit transfers such as the OR-to-ICU handoff, which is a critical area of vulnerability for many institutions.

RW: What outcomes did you find?

AS: With the multicenter study, we found that implementing the bundle of interventions was associated with a 23% reduction in rates of overall medical errors. We also observed a 30% reduction in rates of preventable adverse events, which are errors that result in harm to patients and are the ones we're most interested in being able to improve. We also collected and listened to audio recordings to get a sense of whether critical data elements for the verbal handoff process were present or absent and observed significant improvements in nine out of nine of those elements. Similarly, we looked at copies of the written handoff documents and found significant improvements in five out of five categories of handoff information that were assessed.

For the time-motion study, a research assistant followed individual residents and interns around in real-time, wrote down every aspect of their activity, and recorded what they were doing both before and after the intervention was put in place. Specifically we were interested in how the new handoff processes might affect the amount of time they were spending in front of a computer, the amount of time in contact with patients or families, or the amount of time it took them to pass off patient information. For all of those three medical categories, we didn't see any significant changes comparing before to after the intervention.

RW: The drop in adverse events and in errors almost seems too high to explain based on improved handoffs. I'm wondering whether what you saw in terms of the decrease in errors and in preventable adverse events goes beyond the impact of the handoffs—that focusing on one discrete set of processes with a rigor that people weren't used to may have had some spillover effect on other types of errors that were independent of the handoffs?

AS: I think you could be right. It comes back to why we felt it was important to not focus only on handoff communication but also to address general communication and teamwork training through introduction of TeamSTEPPS techniques. When we started this study 5 years ago or so, the concepts of standardized high reliability communication techniques, huddles, or having a formal debrief to go through things were all relatively poorly applied in health care. However, following I-PASS implementation these things are much more common and the providers seemed more comfortable using a lot of that terminology. So I think we really did do a lot beyond just handoffs.

RW: Do you worry about what has happened in some of the checklist literature, which is people began to think: if we just do a checklist, it will work. The authors of the checklist study have made that point that if it's not embedded in this robust set of cultural changes and some logistical changes, a checklist by itself won't do very much. Do you worry about that with I-PASS?

AS: I think we do. It's related to how I described what we mean by I-PASS. A lot of people hear about the study and they think that, oh, I-PASS is just a mnemonic. We can just send out an email to all our providers describing these five letters and what they stand for and expect that will lead to remarkable reductions in medical errors, and that's just not the case. We emphasize that trying to get people to change their ingrained patterns of communication and how we interact with each other every day as we take care of patients is a huge behavioral change that takes much more than a five-letter mnemonic. Having all of the bundle components work synergistically with each other is what we think is so important to lead to the dramatic improvements in error rates.

RW: Do you have any data or sense of what's happening at the nine places now that the spotlight is off? Is there any recidivism or are things continuing on the same trajectory?

AS: We had a meeting recently where we asked a number of the site leads that same question. One of things we anticipated would be hard to sustain was engaging faculty to observe the handoffs and provide feedback intermittently to the residents. This was one of the most important components of the intervention, but initially required a lot of effort to get off the ground. It turns out that faculty observations in all the sites that were at that meeting were still happening. We found that with time, these new handoff behaviors are truly ingrained into the way residents do their handoffs. Of course, there's a cycle to residency training. In pediatrics, it's a 3-year program. We did notice that it's taken about 3 years to get to the point where all anybody knows is I-PASS. Now, it seems to be part of the normal communication pattern.

RW: What's next for your group and what's next for you?

AS: Currently, we're working on two main initiatives. One project is an effort funded by AHRQ to disseminate the I-PASS handoff bundle to 32 additional institutions, half of which will be adult medicine sites. This project has been a fantastic partnership with the Society of Hospital Medicine, where we're adapting SHM's Mentored Implementation Model, which has been successful in prior multisite quality improvement efforts. The infrastructure of this ongoing project has allowed us to take all of the components of the I-PASS bundle and even further improve the curriculum and implementation materials so that we now have them available for additional provider types beyond just pediatricians.

Our second main focus right now is a project funded by PCORI—an effort to incorporate I-PASS and other structured communication techniques into communication with patients and families. By implementing I-PASS techniques into processes such as family-centered rounds, we hypothesize that we are much more likely to ensure that doctors, nurses, and families are all on the same page with regard to the plan of care.

In terms of my personal next directions, the thrilling part is that this work can lead to so many different opportunities to further disseminate I-PASS and improve communication in other domains. We spoke earlier about the technology piece. I'm really interested in finding ways to leverage technology in order to continue to push boundaries and ensure we have user-friendly systems in place to support verbal and written documentation and communication between providers and patients.

RW: My last question is: Can you envision a world in which the technology is so good that this person-to-person stuff can go away?

AS: I don't think you can ever replace the person-to-person piece. The technology can only enhance that type of communication. The technology affords the opportunity to reference key information throughout the shift and make sure that it's all there and correct and documented. However, two-way person-to-person communication goes back to that final S of I-PASS, synthesis by receiver. This synthesis component was an initially resisted yet ultimately highly valued behavior change that requires the receiving team to listen actively and verbally restate the information that they were now responsible for. We have found this active listening and engagement piece is particularly critical and often triggers additional dialogue and opportunity for asking questions and clarifications that wouldn't have occurred otherwise. That's always going to be an important piece. We can't get rid of people!