Moving Health Services Research Into Policy and Practice: Lessons from Inside and Outside the Health Sector
Moving Health Services Research Into Policy and Practice: Lessons from Inside and Outside the Health Sector
REPORT FROM AN ACADEMYHEALTH WORKSHOP
APRIL 28-29, 2014, WASHINGTON, DC
APRIL 28-29, 2014, WASHINGTON, DC
Executive Summary
Health policymakers, whether in government or delivery system organizations, are confronting critical issues of health costs, access to health care, and the quality and outcomes of health care that require evidence-based solutions. Health researchers are producing studies that address many of these same questions. Yet the longstanding gap between what is known, what is used, and what is implemented continues and the need for effective translation and dissemination endures. As a basis for its future experimentation, skill building, and direct translation and dissemination activities, the AcademyHealth Translation and Dissemination Institute (the Institute) seeks to learn from within health services and policy research as well as experts from other fields. This report summarizes what the Institute explored and learned at a workshop convened on April 28-29, 2014 as part of the Lessons Project.
The Lessons Project sought out experts in a variety of disciplines and commissioned papers to learn from the body of relevant evidence and experience each could offer (See Box 1). AcademyHealth then convened a workshop of 53 experts in April 2014 to discuss the papers and their potential lessons for health services and policy research. Approximately half of the participants came from fields outside the health sector. The remaining participants were members of the health services research, health care delivery, and health policy communities with particular expertise, experience, or interest in how research findings make their way to the decision-makers who need them. The rich discussion over the two days suggested a host of observations and lessons for the producers, funders, translators, and users of health services and policy research. This report groups these points into three categories: (1) issues related to more effectively linking research producing communities with research users in policy and practice communities; (2) issues related to the framing of research and the context for its use; and (3) issues related to communicating research findings.
Linking Research With Policy and Practice
Workshop participants spent considerable time thinking about the most desirable relationship between the producers and consumers of research.
Challenges to Effective Linkages
In any sector, the link between research and its potential uses is less than desired and a robust literature exists as to the reasons for this. For example, linkage can be hampered by an academic reward system that attributes less value to a study that has broad policy impact than to one that has the highest respect of academic peers. Despite this concern, one workshop participant noted that academic rewards for helping practitioners can vary by discipline and department, and that leaders in academic health services research institutions might benefit from considering innovative approaches from other fields and universities. Effective linkages may also be limited because research producers and consumers do not necessarily speak the same language and may even be distrustful or antagonistic to each other. Further, researchers do not always draw out the policy implications of their findings.
In any sector, the link between research and its potential uses is less than desired and a robust literature exists as to the reasons for this. For example, linkage can be hampered by an academic reward system that attributes less value to a study that has broad policy impact than to one that has the highest respect of academic peers. Despite this concern, one workshop participant noted that academic rewards for helping practitioners can vary by discipline and department, and that leaders in academic health services research institutions might benefit from considering innovative approaches from other fields and universities. Effective linkages may also be limited because research producers and consumers do not necessarily speak the same language and may even be distrustful or antagonistic to each other. Further, researchers do not always draw out the policy implications of their findings.
Related questions of interest at the workshop included whether research producers ask the right questions to produce the information research consumers need and whether they are doing so on a timely basis. Participants pointed to a long history of studying new policy initiatives or demonstration projects on a lengthy time frame that makes the findings less useful. Delays in getting articles accepted and published in peer-reviewed journals can exacerbate the problem, especially if researchers forgo a direct means of informing policymakers or delivery system leaders in advance of publication.
A panel at the workshop focused on the safety net delivery system environment for translation and dissemination put forth the idea that innovation can be hard to accomplish. These institutions work with challenging patient populations, operate on narrow margins, and their managers may have limited connections with researchers. In addition, many safety net delivery systems must operate within government hiring and procurement rules, which can limit their ability to introduce or scale innovation or end unsuccessful experiments.
Opportunities for Stronger Personal Relationships
Examples from within and outside of health care demonstrate the value of giving research producers the opportunity to work in research users’ offices and for users to spend time in the research producers’ world. Such cross-fertilization can benefit both the producers and consumers of research and lead to more policy relevant evidence
, although it can also involve a high level of resources to reach a relatively small number of individuals.
Examples from within and outside of health care demonstrate the value of giving research producers the opportunity to work in research users’ offices and for users to spend time in the research producers’ world. Such cross-fertilization can benefit both the producers and consumers of research and lead to more policy relevant evidence
, although it can also involve a high level of resources to reach a relatively small number of individuals.
Opportunities for More Effective Linkages Through the Funding of ResearchWorkshop participants discussed how research funders might be able to create closer links between research and policy or practice. One option might be for funders to be more attentive to how their funding cycles influence the ability of scholars to complete studies in time to make results available to public and private policymakers when they need them. Evaluation research, in particular, can have a greater impact when done on a timely basis. Rather than waiting for the results of fully fleshed out quantitative analysis, another answer to this challenge may be more use of rigorous qualitative research, including case studies. Crowd-sourced funding might provide another approach to generating timely research in a nimble manner. In other areas of science, crowd-sourced funding has provided a faster alternative mechanism to get ideas off the ground quickly (e.g. Cancer Research UK’s My Projects program).
Opportunities for More Effective Linkages Through the Dissemination and Implementation ProcessThe workshop discussion also highlighted a number of ways in which the general approach to translation and dissemination can help assure that research is both useful and used:
- The right messenger is critical to dissemination, and the right messenger is not always the researcher or research organization that produced the study
- Even those researchers who disseminate their findings only through peer-reviewed publications could make the work more valuable by flagging findings of particular relevance to government policy staffs, provider organizations, or others who might use the results to change health care. Although the traditional journal article format includes a discussion section, researchers sometimes view it as an after-thought, rather than an opportunity to make findings relevant to those outside the research community.
- Systematic reviews can provide an efficient way for decision makers to access the current research around particular issues. The availability of collections of systematic reviews such as McMaster University’s Health Systems Evidence database, the Cochrane Collaboration, and the Robert Wood Johnson Foundation’s former Synthesis Project may present opportunities to help policymakers and delivery system leaders in the United States incorporate this evidence into their decision-making.
- Researchers may be able to increase the likelihood that proven delivery system innovations are scaled beyond an initial demonstration and replicated successfully by anticipating the variety of settings where they might be implemented, identifying potential early adopters, and building early support among institutional leadership and other stakeholders.
Framing Research to Increase Its Usefulness for Policy and Practice
Several of the workshop presentations underscored that, despite years of study and many efforts in this country and internationally to document the policymaking process, the research community is not always aware of the political, social, and economic contexts in which policymakers view research findings. Translation and dissemination will fail if the content of research findings does not fit the needs of the policy community.
Any researcher interested in seeing their work used needs to consider content through a lens of context, and then frame it appropriately for the issues of the day. Health services research can remain relevant despite a changing political or policy environment. Gridlock in legislative bodies can shift the policy focus – and thus the need for evidence – from legislation to policy implementation in the executive branch, whether in federal or state governments. Policies also cycle between a focus on federal action and one on state activities as election results modify the environment. The Affordable Care Act provides a good example of these shifts. Action started in Congress with passage of the law, shifted to the executive branch to develop infrastructure and regulations, and then moved on to the states to make and implement policy decisions about the health insurance Marketplaces, Medicaid expansion, insurance rate review, and other aspects of the law. In an era of constrained federal resources, the research community can expect more policy action to take place at the state level.
Health services researchers who work with states should be attuned to the significant political and economic pressures in which they operate as well as states’ varying capacities and interest in making use of research when developing policies. These pressures can make state officials particularly sensitive to some research findings and how they are interpreted in the media, particularly if they could be interpreted as being critical of state actions. One result of this has been a tendency for some state officials to maintain greater control over the analytic process by relying on consultants and contractors rather than independent researchers to answer analytic questions.
One workshop presenter noted that when policy issues become more politically salient, researchers may need to be more attuned to “ordinary knowledge” (a combination of the common-sense perspectives and the biases held by average people). As politics dominates the debate on a policy issue, “ordinary knowledge” can challenge “policy-analytic” information (research evidence that comes from scientific inquiry). Drawing on examples from the debate over climate change, another presenter described how framing can be used by researchers to communicate the relevance of an issue and their findings and to make a case for allowing a discussion of policy solutions to overcome partisan differences. Expert organizations have a potential role as honest brokers of information. However, workshop participants noted that the frame for talking about research findings is only as good as the underlying product.
Communicating Research More Effectively
One major theme of the meeting was whether there are better ways to communicate the results of research to make them more valuable to policy and delivery system leaders. Workshop presenters and participants identified some ideas that, while not new, have received less attention in the health services research and policy community:
- Even in a world driven by electronic communication there is still no substitute for trusted, personal relationships.
- Storytelling elements can be valuable in making research findings accessible and engaging to audiences, especially those without technical training. This method of framing information can help in summarizing research results for a press release, a blog, testimony, or even in constructing a journal article. The context should determine the specific narrative techniques employed.
- In an era characterized by short attention spans and information overload, social media is becoming an increasingly common way to alert research consumers that new studies are available. Some policymakers and delivery system leaders are using Twitter to learn about new research findings. Other forms of communicating research results such as email news digests (Kaiser Health News) or content summarizers (Uptodate.com) can play a similar role for others and can provide more detail than a tweet. There is need for better education in the research community about how to use these tools. In some cases, senior researchers may simply need to look to younger colleagues and students who are more facile with some of the newer modes of dissemination. But in other cases, researchers may benefit from formal training in how to target research results more effectively to research consumers.
In response to a provocative presentation about potential lessons from the experience of science museums, workshop participants also considered whether there was a productive role for health services and policy researchers to communicate directly with the public. Innovative experiences of engaging individuals with researchers around difficult issues such as race or climate change may have relevance to engaging controversial political issues in the health policy arena with average citizens or policymakers. Efforts like those of Consumer Reports to use the findings of effectiveness research together with price information to highlight which drugs are “best buys” for consumers represent another example. An alternative approach for health services and policy research as a field might be to engage indirectly with the public through opinion leaders and public-facing organizations. Directly or indirectly engaging with the public may help build support for the use of research evidence in policy and delivery systems and serve to correct misperceptions about health policy and delivery.
Next Steps
In convening this workshop, AcademyHealth took a first step in helping health services and policy researchers think about how the research enterprise might adapt in order to increase its impact. A natural next step would be to launch pilot projects testing innovative approaches to translation, dissemination, and implementation that incorporate the lessons from this project. One of the most important lessons of the workshop may be it demonstrated there is significant value in bringing together professionals with very different experiences, skills, and perspectives but who share a common interest in effective communication of technical information. Another follow-on activity might be to facilitate a “learning network” in which the producers, funders, translators, and users of research could help build on the enthusiasm, insights, and professional connections coming out of the Lessons Project.
Introduction/Problem Statement
Health policymakers, whether in government or delivery systems, are confronting critical issues of health costs, access to health care, and the quality and outcomes of health care that require evidence-based solutions. Health researchers are producing studies that address many of these questions. Yet the longstanding gap between what is known, what is used, and what is implemented continues and the need for effective translation and dissemination endures. As a basis for its future experimentation, skill building, and direct translation and dissemination activities, the AcademyHealth Translation and Dissemination Institute seeks to learn from leaders within health services research as well as experts from other fields. This report synthesizes what AcademyHealth’s Translation and Dissemination Institute explored and learned at a workshop convened on April 28-29, 2014 as part of the Lessons Project. Box 1 lists the papers and other topics discussed at the workshop.
Appendix A provides additional detail about the Lessons Project, while Appendices B and C present the workshop agenda and the list of participants.
This work builds on a substantial research literature and best practices established over many years both in the United States and elsewhere. Much of the relevant previous work has focused on the translation and dissemination of health services research,1 but evidence-based decision-making in other fields has also received attention.2 This work includes rigorous research studies of the translation, dissemination, and implementation process3 in addition to toolkits and other practice “how to” resources.4 Within the health services field, previous work focuses on translation and dissemination for policy5 as well as dissemination and implementation for the provision of individual and population health services.6 The purpose of the Lessons Project and its workshop was not so much to develop new insights than it was to highlight literature and experience that appears to hold promise for communicating and using health services and policy research but seem not to be part of the conventional wisdom in the field. The report that follows groups insights gleaned from the workshop into three categories: (1) linking research with policy and practice; (2) framing research findings and placing them in context; and (3) communicating research findings.
Work Commissioned as Part of the Lessons Project
AcademyHealth initiated the Lessons Project by commissioning a series of papers during the summer of 2013 that served as the basis for the April 2014 workshop’s agenda. With input from the Translation and Dissemination Institute’s advisory committee, AcademyHealth staff commissioned eight papers in three general categories:
Insights from Health Services Research
Insights from Other Disciplines
Insights from Other Areas of Public Policy
Although the scopes of the papers varied by topic, AcademyHealth asked all of the authors to review relevant literature from their field and/or identify examples of activities or programs that could be adapted for the translation and dissemination of health services research for (1) health policymaking and (2) delivery system innovation, in particular for the provision of safety net care. Because of the nature of the paper topics, some were more relevant to the use of health services research in one of these contexts than the other.
In building the workshop agenda (Appendix B), staff supplemented discussion of the papers with presentations covering four additional topics: (1) the use of systematic reviews to inform the policymaking process; (2) the application of storytelling and dramatic improvisation techniques to communicating research findings; (3) graphic visualization of research findings; and (4) the role of social networks in understanding scientific and health information.
|
Linking Research with Policy and Practice
Workshop participants spent considerable time thinking about the most desirable relationship between the producers of research and consumers of research. Research producers, who conduct research and draw conclusions, may be based in independent organizations such as academia, freestanding research organizations and think tanks, or foundations. In addition, some research producers work in independent government research operations such as the Congressional Budget Office, the Medicare Payment Advisory Commission, or research offices inside various executive agencies (although the latter organizations also serve as policy intermediaries between research producers and research consumers, also referred to as research or knowledge translators). Consumers of research include both government policymakers and decision makers in private health care organizations such as health plans, provider organizations, consumer or patient-based organizations, or health care industry organizations. Some private stakeholder organizations are also producers of research, but with varying degrees of independence from the interests of their sponsors.
Challenges to Effective Linkages
The link between research and its potential uses is not always clear. Workshop participants noted that such links may be hampered by an academic reward system that attributes less value to a study that has broad policy impact than to one that has the highest respect of academic peers. Despite this general concern, one participant noted that academic rewards for helping practitioners understand and apply research findings can vary by discipline and department, and that leaders in academic health services research institutions might benefit from considering innovative approaches from other fields and universities.
The link between research and its potential uses is not always clear. Workshop participants noted that such links may be hampered by an academic reward system that attributes less value to a study that has broad policy impact than to one that has the highest respect of academic peers. Despite this general concern, one participant noted that academic rewards for helping practitioners understand and apply research findings can vary by discipline and department, and that leaders in academic health services research institutions might benefit from considering innovative approaches from other fields and universities.
Effective linkages may also be limited because research producers and consumers do not necessarily speak the same language and may even be distrustful or antagonistic to each other. As workshop moderator Lauren LeRoy observed, motivation is critical, both the producers’ motivation to communicate effectively about their work and the users’ motivation to learn from and use research. But beyond that basic idea, researchers do not always express their findings in language that non-specialists will understand. Further, researchers do not always draw out the policy implications of their findings. The traditional journal article format includes a discussion section. But researchers too often treat this section as an after-thought or even an annoyance, rather than an opportunity to make findings relevant to those outside the research community. For example, there is a rich and extensive body of research on geographic variation in the use of health care services. But one workshop commenter asked whether the research producers who describe and explain these variations are doing enough to offer policymakers potential action steps. In general, workshop participants felt researchers can do more to be ready when policy windows open.
Another question of interest at the workshop was whether research producers ask the right questions or whether they produce the research that consumers need. As noted by one commenter, some researchers appear to generate knowledge for knowledge’s sake without thinking about priorities for what knowledge might be most useful and using those priorities to set research agendas. For example, implementation of the Affordable Care Act is generating many policy questions that could be informed by research. But are researchers addressing the questions for which policymakers need answers?
Even if researchers are addressing the “right” questions, are they doing so on a timely basis? There is a long history of studying new policy initiatives or demonstration projects, but doing so on a lengthy time frame that makes the findings less useful. Delays in getting articles accepted and published in peer-reviewed journals can exacerbate the problem, especially if the researchers lack or forgo a direct means of informing policymakers of results in advance of publication. In some cases, circumstances have changed to such a degree that the results can no longer inform policy decisions. Data availability can also be a major obstacle. Claims data that can provide insights may not be available until a year or more after the events in question have occurred, and surveys can take months to put in the field and obtain results.
One example draws on Medicaid. Kathleen Nolan from the National Association of Medicaid Directors pointed out in her remarks that data have been fairly broadly available for Medicaid programs. The gap has been the ability to sort through and use the data, while also dealing with the fact that the data elements collected may not be consistent across states. Timeliness has been a particular issue, as described by Michael Sparer in a recent research synthesis on Medicaid managed care.7 States are actively experimenting with different approaches to Medicaid managed care, but much of the published literature (even that published relatively recently) looks back at the record of the 1990s when the environment was quite different.
Many state initiatives operate as waivers from the federal government, which typically require evaluations to be conducted. But many of these evaluations have been done by researchers under direct contract to the states and resulted in only limited publications in peer-reviewed journals or in other widely available publications. Furthermore, the nature of conducting evaluations one state at a time works against the ability to compare the experience of different states. Questions can be raised about how well research findings from one state apply to states with different political and economic situations and different health care environments.
Nolan also pointed out in her remarks that state policymakers often see collaboration with outside researchers as politically risky. States operate under tremendous political and economic pressures and vary in both their capacity and their interest in making use of research when developing policies. There have been cases where state officials have felt burned by research findings that contribute to the pressures states face, especially when researchers or the media have misinterpreted data or findings. In an attempt to mitigate political risk by maintaining greater control over the analytic process, state officials use consultants and contractors to answer the questions they have. Federal policymakers also commission consultants to conduct research under contract. However, this approach raises important questions of intellectual property and the ability to broadly disseminate results without onerous governmental clearance processes.
In a panel at the workshop focused on the safety net delivery system environment for translation and dissemination, Ed Havranek from Denver Health put forward the idea that innovation can be hard to accomplish in safety net institutions. In particular, the uptake of innovative approaches occurs most easily when organizations are large, specialized, and characterized by decentralized leadership with flexible management relationships. Safety net institutions tend to work with challenging patient populations, operate on narrow margins, and their managers may have limited connections with researchers. In addition, many of these institutions must operate within government rules in areas such as hiring and procurement, which can limit their ability to introduce innovation, bring innovative products to scale, or end experiments that do not work. The health policy landscape is filled with examples of innovations that seemed to work in their test settings, but failed to succeed elsewhere. Yet it may be critically important for these institutions to benefit from innovative approaches.
Opportunities for Stronger Personal Relationships
Workshop participants believe there are important opportunities for better collaboration between research producers and research consumers. One means to do this is to create opportunities for academic researchers to spend a fellowship period working in government offices. Such programs, which exist in both federal and state government settings, bring research results to the government agencies but also make the researchers more aware of how they can maximize the impact of their research. AcademyHealth offers several of these programs such as the Delivery System Science Fellowship and the NCHS/AcademyHealth Health Policy Fellowship.8 Other programs have sought to translate research findings to government policymakers through seminars or policy briefs targeted specifically to this audience. One workshop participant pointed to programs where state legislators are selected as fellows to expose them to the research world. Examples include projects in Georgia and South Carolina.9 Cross-fertilization has the potential to bring considerable benefit to both the producers and consumers of research and to help produce research with more policy relevance. But this approach can involve a high level of resources to reach a relatively small number of policymakers.
Workshop participants believe there are important opportunities for better collaboration between research producers and research consumers. One means to do this is to create opportunities for academic researchers to spend a fellowship period working in government offices. Such programs, which exist in both federal and state government settings, bring research results to the government agencies but also make the researchers more aware of how they can maximize the impact of their research. AcademyHealth offers several of these programs such as the Delivery System Science Fellowship and the NCHS/AcademyHealth Health Policy Fellowship.8 Other programs have sought to translate research findings to government policymakers through seminars or policy briefs targeted specifically to this audience. One workshop participant pointed to programs where state legislators are selected as fellows to expose them to the research world. Examples include projects in Georgia and South Carolina.9 Cross-fertilization has the potential to bring considerable benefit to both the producers and consumers of research and to help produce research with more policy relevance. But this approach can involve a high level of resources to reach a relatively small number of policymakers.
A similar issue arises in the private sector. Even in health systems that have a research group, how good are the connections between the in-house researchers and the system managers and clinical leadership? Have they built relationships built on trust and ongoing communication? Do they speak the same language and have the same time perspectives? One workshop participant stated that sometimes CEOs in health systems say they do not care about research. To overcome this disconnect, the research community needs to find the place where there is a commonality of interest (perhaps in the office of the CFO not the CEO) if research can help identify value for the bottom line. Good relationships can help ensure that researchers identify study topics that accomplish this goal.
Opportunities for More Effective Linkages Through Research Funding
Research producers and users are not the only actors in the health services research and policy enterprise. Research funders bring a wide variety of goals and priorities to the research endeavor, some of which reflect the priorities of the original donors that create private foundations or the government or private agencies that sponsor research. But policy impact is a goal shared by many – if not most – funders of health services research and health policy research. In his presentation at the workshop, Kieran Walshe from the Manchester Business School drew on his recent analysis of health services, systems, and policy research in the United Kingdom to describe recent national efforts to integrate research, education, and service delivery at the organizational or delivery system level, underscoring that public and private funders can have a productive role alongside universities, policymakers, and providers in assuring knowledge-informed practice.10
Research producers and users are not the only actors in the health services research and policy enterprise. Research funders bring a wide variety of goals and priorities to the research endeavor, some of which reflect the priorities of the original donors that create private foundations or the government or private agencies that sponsor research. But policy impact is a goal shared by many – if not most – funders of health services research and health policy research. In his presentation at the workshop, Kieran Walshe from the Manchester Business School drew on his recent analysis of health services, systems, and policy research in the United Kingdom to describe recent national efforts to integrate research, education, and service delivery at the organizational or delivery system level, underscoring that public and private funders can have a productive role alongside universities, policymakers, and providers in assuring knowledge-informed practice.10
Although the observation that timeliness is a prerequisite for having a policy impact is not new, discussants at the workshop suggested that research funders might help facilitate that timeliness by being more attentive to how their funding cycles influence the ability of scholars to complete studies in time to make results available to public and private policymakers when they need them.
Evaluation research, in particular, can have a greater impact if done on a timely basis. There are numerous instances where policymakers who find a demonstration project attractive have been eager to move that project to full scale without waiting for an evaluation to be complete. An evaluation done over a longer time span can make a valuable contribution to the scholarly literature. But more immediate evaluation results can help guide decisions on whether to move the project to scale and whether to continue demonstration funding or incorporate a demonstration into a permanent program element.
In other arenas, crowd-sourced funding has sometimes provided a faster alternative mechanism to get ideas off the ground quickly. For example, a British group called Cancer Research UK has put together crowd-sourced funding for clinical research through its My Projects program,11 and uBiome and American Gut projects have raised money through Indiegogo for DNA sequencing of the microorganisms living inside the human body.12 Some funders have established rapid-response mechanisms to focus resources on short-term needs where the policy process may be positioned to move quickly.
A more general question raised in the discussion is whether demonstrations and case studies represent unrealized opportunities for researchers and research translators. Academic researchers often downplay case studies as not offering good research opportunities because generalization is limited. Furthermore, this type of research is not typically valued as highly in scholarly journals or in academic reward systems. As a result, high quality, systematic case studies are often lacking. Yet, case studies can be critical in understanding the success of any strategy, whether it be an accountable care organization, a long-term care demonstration project, or a global public health initiative.
Opportunities for More Effective Linkages Through the Dissemination and Implementation Process
In an era characterized by short attention spans and information overload, modes of disseminating information of all types are undergoing massive transformations. Social media mechanisms such as Twitter may become increasingly useful ways to alert research consumers that new studies are available. A growing number of researchers are using blogs, podcasts, and other modes of sharing research findings with a broader audience, though these are not yet widespread practices. A recent survey by workshop participant Zack Meisel and colleagues found only 14 percent of health services researchers reported tweeting and 21 percent reported blogging in the last year.13 Other researchers rely more (whether consciously or not) on trusted messengers and other intermediaries to help the dissemination process. As Lauren LeRoy, the workshop moderator, noted in a summary, the right messenger is critical to dissemination, and the right messenger is not always the researcher or research organization that produced the study.
In an era characterized by short attention spans and information overload, modes of disseminating information of all types are undergoing massive transformations. Social media mechanisms such as Twitter may become increasingly useful ways to alert research consumers that new studies are available. A growing number of researchers are using blogs, podcasts, and other modes of sharing research findings with a broader audience, though these are not yet widespread practices. A recent survey by workshop participant Zack Meisel and colleagues found only 14 percent of health services researchers reported tweeting and 21 percent reported blogging in the last year.13 Other researchers rely more (whether consciously or not) on trusted messengers and other intermediaries to help the dissemination process. As Lauren LeRoy, the workshop moderator, noted in a summary, the right messenger is critical to dissemination, and the right messenger is not always the researcher or research organization that produced the study.
Although not a new observation, workshop discussants emphasized the need for better education in the research community on how to disseminate research results and make them as relevant as possible to policymakers. They suggested that in some cases, senior researchers may simply need to look to younger colleagues and students who are more facile with some of the newer modes of dissemination. But in other cases, it may be a matter of training researchers to think more about audiences outside of academia and to consider how to target research results more effectively to research consumers.
Workshop participants noted it is unlikely that blogs and social media will eliminate the role of more traditional peer-reviewed journals as an outlet for publishing the results of research. In fact, some journals are increasingly embracing some of the newer communication techniques. But there is potentially a broader challenge for journals to rethink their current models, beyond just new use of graphics or social media. Elizabeth Bass from the Alan Alda Center for Communicating Science at Stony Brook University told the workshop that the public tends to focus on the bottom line first, then the “so what” question, and last the supporting details. By contrast, the researcher typically presents background, followed by supporting details, and ends with the results. It may not be necessary to flip the typical approach of a journal article, but more attention to the “so what” question may increase the value of these articles for policymakers. Furthermore, journals may want to consider how to respond to policymakers’ interest in demonstrations and evaluations, as well as rapid-response findings that rely more on qualitative research approaches. This may require greater openness to case studies, qualitative research, and negative results.
In his presentation to the group, Michael Wilson from the McMaster University Health Forum underscored that systematic reviews can provide another particularly efficient way for decision makers to understand whole bodies of evidence around particular issues. The availability of collections of systematic reviews such as through McMaster’s Health Systems Evidence database, the Cochrane Collaboration, the Campbell Collaboration, and the Robert Wood Johnson Foundation’s former Synthesis Project may present opportunities to help policymakers and delivery system leaders in the United States access and use these resources.
Finally, a presentation by Pierre Barker and Marie Schall from the Institute for Healthcare Improvement suggests the dissemination and implementation of health innovations in lower income countries may offer lessons for overcoming some of the challenges faced by U.S. safety net institutions in replicating and scaling successful pilots. They argue that scaling up is “not a straightforward task” and that “it can take many years for a new evidence-based idea to be broadly implemented.” In their eyes, a key to success in taking innovations to scale is incorporation of that intention from the start, while also taking into account the political dynamics of making changes. Thus, the set-up phase may include establishing an alignment of interests and identifying early adapters. The innovation can then move through developing a scalable unit as a prototype and testing the ability of the innovation to go to scale and to operate in different contexts. If innovators can anticipate the variety of settings for their new idea, they can increase the chance of going to scale successfully. Similarly, if they build support ranging from engaged leadership to relevant infrastructures, the chance for success grows.
Framing Research to Increase Its Usefulness for Policy and Practice
Although the primary workshop focus was translation and dissemination of health services research, participants also considered the content and focus of that research. A phrase first used at the workshop by Valerie Delva from Ketchum, Inc, but cited several times during discussions, was “content is king.” Participants debated exactly what this meant, but the bottom line seems to be that translation and dissemination will fail if the content of research findings does not fit the needs of the policy community.
As Marsha Gold from Mathematica Policy Research observed, health services research can remain relevant despite a changing policy environment. Policy gridlock can shift the policy focus – and thus the need for evidence – from legislation to policy implementation in the executive branch, whether in federal or state governments. Policies also cycle between a focus on federal action and one on state activities. This may reflect either the life cycles of political issues or political preferences as election results modify the environment. For example, the George W. Bush administration allowed the states more flexibility in their Medicaid programs than is the case under the Obama administration – although considerable flexibility remains even today. The Affordable Care Act began as a federal-level debate in the Congress. But once it was enacted, policy issues became the domain of the Departments of Health and Human Services, Labor, and Treasury. And further rounds of decision-making became the domain of states, at least for those states that chose to take up Medicaid expansions and to run insurance Marketplaces. Researchers seeking impact for their research need to be aware of the right policy venue depending on the stage of an issue’s life cycle.
According to Mark Peterson from UCLA, research experts tend to have a greater impact on low salience, routine policy issues. When policy issues become more politically salient, the focus of influence shifts to politicians. Research and other information often become ammunition in the political crossfire, rather than informing a reasoned debate. Peterson notes that as politics dominates the debate on a policy issue, there is a threat that “policy-analytic” information (research evidence that comes from scientific inquiry) will be challenged by “ordinary knowledge” (a combination of the common-sense perspectives and the biases held by average people). As Peterson sees it, another element of the threat is that research evidence is now routinely challenged by ideological mythmaking circulated by politicians and echoed by the ideologically-oriented media outlets that now dominate much of the public discourse.
To counteract these threats, researchers may need to be more attuned to ordinary knowledge and operate in broader interdisciplinary teams to help accomplish communications and dissemination goals. The right messenger may also be important – and may not always be the research organization that produced a study. Giving up some control to intermediaries or trusted messengers may be painful to researchers who worry about losing methodological nuances and caveats. But intermediaries may be more effective in getting the findings to those who make decisions.
Not surprisingly, workshop participants could offer no easy solutions to the growing political polarization of policy discussions. But the communication and translation strategies described below have some ability to help. Matthew Nisbet from Northeastern University described some of the ways that framing can be used by researchers to communicate the relevance of an issue and their findings and to make a case for allowing a discussion of policy solutions to overcome partisan differences. “Framing can be used to pare down information, giving greater weight to certain considerations and elements over others, thereby communicating personal relevance and shared interests or values.” Nisbet noted existing studies that indicate framing is critical because people (both the general public and policymakers) tend to look for cues that are consistent with what they believe to be important. They will often reject data and analysis that do not fit their world view (Box 2). He drew specific examples from the climate change debate to illustrate how framing has influenced the ability of scientists to communicate their findings in a highly polarized environment. He emphasized, however, that the frame for talking about research findings (like storytelling) is only as good as the underlying product. Nisbet also pointed to the potential role for expert organizations as honest brokers of information.
They will often reject data and analysis that do not fit their world view (Box 2). He drew specific examples from the climate change debate to illustrate how framing has influenced the ability of scientists to communicate their findings in a highly polarized environment. He emphasized, however, that the frame for talking about research findings (like storytelling) is only as good as the underlying product. Nisbet also pointed to the potential role for expert organizations as honest brokers of information.
Content may be king, but context is the king’s prime minister. Content increasingly needs to be considered through a lens of context, framed and associated with relation to the issues of the day by effective messengers. Workshop participants pointed to the need for the research community to be more aware of context. Context includes political, social, and economic elements. Researchers may prefer to believe that research findings are unaffected by the present-day context, but it seems that they do so at risk to their potential reach and influence.
Communicating Research More Effectively
One major theme of the meeting was whether there are better ways to communicate the results of health services research and health policy research to make those results more valuable to the policy community, whether that community is defined as policymakers, others involved in shaping policy, members of the media, or the broader public. Not surprisingly, discussion at the workshop surfaced a number of well-established principles and practices of effective communication (Box 3). However, presenters and participants also identified some ideas that, while not new, have received less attention in the health services research and policy community.
The Importance of RelationshipsOne overarching perspective that can facilitate effective communication of research with policymakers and other decision-makers, while also helping with other goals such as identifying the most appropriate content, is the continuing value of personal relationships. One discussant drew attention to the definition of good public relations as a process that builds mutually beneficial relationships between organizations and their publics. Another reminded the group that even in a world driven by electronic communication there is still no substitute for a trusted relationship and that a personal phone call or email between a researcher and a policymaker with such a relationship can be key to allowing research to inform policy.14 Good relationships help to ensure that researchers understand their audience and its needs and perceptions.
and that a personal phone call or email between a researcher and a policymaker with such a relationship can be key to allowing research to inform policy.14 Good relationships help to ensure that researchers understand their audience and its needs and perceptions.
The Importance of StorytellingOne of the clear messages articulated by Elizabeth Bass in her keynote was the use of stories to connect research to a broader audience. Some researchers may react that this is not the best approach for communicating research findings because it lacks the rigor and precision of more traditional presentations of research findings. For example, it can be too easy to interpret correlation as a causal relationship or to attribute too much importance to findings based on a small sample size. But storytelling elements can be valuable in making research findings accessible and engaging to non-research audiences, such as researchers may seek to do when summarizing research results for a press release, a blog, or testimony to a congressional committee or other government body. And the elements of storytelling can even help in constructing a journal article. One element of storytelling is to get into the head of the audience and build analogies that will help readers place the new information into their own context and setting. Storylines also help to engage the interest of the audience.
Techniques of storytelling may have to be modified to meet the context. The approach that works in a blog or in an interview for public radio will differ from the approach to congressional testimony, and both of these will differ from how a journal article is written.
Well-Established Principles of Effective Communication
In addition to some of the less commonly discussed concepts presented in this report, the workshop highlighted several common principles of effective communication that apply, regardless of whether the setting is a peer-reviewed journal article, a blog summarizing findings, or a congressional testimony. The importance of these ideas is found in both the extensive research literature on knowledge translation and communication as well as toolkits and other practical guides created for knowledge producers and translators.14
|
Communicating Research Results in an Era of Information Overload
For most academic researchers, publication in a peer-reviewed journal is the ideal outcome for a research project. Such publications are critical in the academic rewards system, and many researchers are content that the findings will be found over time by other researchers. Key findings from these publications may sometimes be highlighted for a broader audience through a press release or other summary document. But participants at the workshop noted that many researchers do not go beyond these simple steps to find ways of flagging findings for attention by government policy staffs, provider organizations, or others who might use the results to change health care.
Yet some researchers and some research journals have explored new means for getting their research findings noticed. Alicia Wilson, executive director of La Clinica del Pueblo, a community health center in Washington, observed that she recently learned about a relevant academic study through her Twitter feed. With a busy schedule, this represented one way that helpful information actually broke through to her. Workshop participants noted that whether Twitter is the right mechanism is not the point, because new social media are just one means that work for some professionals. Email news digests (i.e. Kaiser Health News) or content summarizers (i.e. Uptodate.com) can play a similar role for others and can provide more detail than a tweet. But more broadly, personal engagement of research producers and research consumers through intermediaries or trusted messengers can be critical.
The focus of workshop presentations on storytelling and framing, described above, can facilitate the process of drawing attention to a finding or an article, as can identifying policy implications. A news digest or content summarizer can more easily flag an article for attention by government or health system decision makers if the policy consequences are easy to see. Thus, a study of the impact of patient cost sharing on the use of services that uses the language of price elasticities and regression statistics will more likely be noticed if the authors link the findings to differences among the Bronze, Silver, Gold, and Platinum-level health plans being sold on the new insurance Marketplaces.
Is There a Role for Health Services Researchers in Communicating With the Lay Public?
Traditionally, academic researchers see the community of scholars as their audience and use scholarly journals and conferences as their primary mode of mass communication. But the public can be a key audience for researchers, as well. An unusual insight on this idea came from a presentation by Meena Selvakumar, formerly with the Pacific Science Center, and Erika Shugart with Erika Shugart Consulting, LLC. Although part of the presentation drew on their expertise in engaging science museum visitors in the museum, some of those ideas about how to engage audiences by incorporating dialogue and bringing groups together to talk about experiences on a difficult issue, such as race, have relevance to engaging controversial political issues in the health policy arena (in fact, some of the same approaches might be helpful in communicating with policymakers and their advisers). One such model, the “Portal to the Public” supported by the National Science Foundation, in which scientists engage with the public (see Box 4). “Science cafes” and “Science on Tap” programs in several communities around the country offer another model.15 Although met with some initial skepticism by the scientists, these engagements have proven quite successful. In the health policy arena, similar efforts may help build more general understanding of and support for health services research.
Traditionally, academic researchers see the community of scholars as their audience and use scholarly journals and conferences as their primary mode of mass communication. But the public can be a key audience for researchers, as well. An unusual insight on this idea came from a presentation by Meena Selvakumar, formerly with the Pacific Science Center, and Erika Shugart with Erika Shugart Consulting, LLC. Although part of the presentation drew on their expertise in engaging science museum visitors in the museum, some of those ideas about how to engage audiences by incorporating dialogue and bringing groups together to talk about experiences on a difficult issue, such as race, have relevance to engaging controversial political issues in the health policy arena (in fact, some of the same approaches might be helpful in communicating with policymakers and their advisers). One such model, the “Portal to the Public” supported by the National Science Foundation, in which scientists engage with the public (see Box 4). “Science cafes” and “Science on Tap” programs in several communities around the country offer another model.15 Although met with some initial skepticism by the scientists, these engagements have proven quite successful. In the health policy arena, similar efforts may help build more general understanding of and support for health services research.
Face-to-Face With Scientists in a Museum
Field-wide research shows that bringing community scientists and public audiences together in one-on-one or small group interactions with scientists can be transformative regarding the notion of “what a scientist is”- demystifying or humanizing the scientist who is often seen as a non-accessible stereotype.16,17 The Portal to the Public (PoP) approach is based on direct interaction of active researchers from organizations within the community with visitors to science museums. In this model, prior to engaging with museum visitors, researchers participate in science communication and interpretation workshops grounded in inquiry-based learning from experienced museum educators. They observe and experience existing hands-on, tabletop activities as models of effective engagement. They also learn to use questions as a strategy to facilitate inquiry-rich learning experiences to support learners in making their own discoveries.18 The Portal to the Public is now practiced in 30 science museums across the United States.
 One example of a typical PoP event is Scientist Spotlight at Pacific Science Center in Seattle. This event occurs on weekends and targets general museum-goers. Each Spotlight features between three and 15 local scientists from a range of disciplines with hands-on, materials-based activities based on their research. Scientist Spotlight is set up in a gallery with festive banners that separate the space into nooks where individual scientists gather with small groups of visitors. The open-ended, conversational nature of Scientist Spotlight allows scientists to tailor their content to their immediate audience, whether a five-year old child, an adult with limited understanding of the area of research, or a peer in their industry. A typical comment from a scientist underscores the fluid nature of these events “Many of the conversations were quite long (20 minutes); that was a nice bonus… It being a quiet day at the museum allowed the conversations to cover a lot of ground and follow whatever questions/tangents/curiosity the kids wanted to discuss.”19
Research and evaluation of PoP programs like Scientist Spotlight has shown that audiences value them because they provide hands-on experiences and insight into the work of community scientists, and give an appreciation for science, its relevance, and its mechanics. The programs also showcase scientists as role models. Audiences are also highly satisfied with the scientists’ ability to involve the group, communicate about science, and connect personally with visitors. The programs de-mystify the stereotype of scientists and show the breadth of ages, ethnicities, and gender represented within the scientific community.20 The personal and conversation-based interaction with research scientists allows museum visitors to explore current science that builds on their personal relevance and comfort level of discussion.
|
Workshop participants responded to this discussion with various ideas such as creating a time line with a slider to help the public visualize health-related issues around the aging population or a website to engage the public in how to help solve the health care crisis to encourage new ideas. Another example raised was the way that Consumer Reports has brought research into a more approachable format for consumers. Among other initiatives, Consumer Reports has teamed with researchers to use the findings of effectiveness research together with price information to highlight which drugs are “best buys” for consumers who seek to select the best combination of effectiveness and price. Others imagined different ways to engage the public around the Affordable Care Act or the Medicaid program and the ways that research helps to inform these public debates. A process similar to the Portal to the Public program might be used to help engage the public around the research that informs health care delivery system reforms, around the role of Medicaid, or around the theories underpinning insurance marketplaces. Alternatively, some workshop participants suggested that health services and policy research as a field might be well suited to engage indirectly with the public through opinion leaders and public-facing organizations.
Next Steps
In convening this workshop, AcademyHealth took a first step in helping researchers think about how the research enterprise might adapt in order to increase its impact. As summarized in this report, the process of digesting the papers and discussion from the workshop identified a variety of potential lessons and ideas that could inform how the field of health services and policy research makes the communication and implementation of research findings more effective. A natural next step would be to launch pilot projects testing innovative approaches to translation, dissemination, and implementation that incorporate the lessons from this project.
The workshop also demonstrated the value in bringing together professionals with very different experiences, skills, and perspectives but who share a common interest in effective communication of technical information. Another next step could be to continue and expand the conversation AcademyHealth started at the workshop. Creating and nurturing a “learning network,” a role that AcademyHealth has effectively played in the past, could help build on the enthusiasm, insights, and professional connections coming out of the Lessons Project. The network, which could gather in-person, online, or both, could determine its own agenda, but its members would represent an additional, on-going resource to help AcademyHealth and the field of health services research implement the lessons and innovations learned.
No hay comentarios:
Publicar un comentario