Integrating Large Pragmatic Trials into Payer Decision Making
Richard Gliklich, M.D., President of Quintiles Outcome
Sean Tunis, M.D., M.Sc., President and Chief Executive Officer, Center for Medical Technology Policy.
Large pragmatic trials are on the fast track to becoming a key vehicle for providing real-world outcomes data for comparing alternative treatment pathways. This responds to growing market pressure for companies to conduct studies that are more representative of usual care and results that are more relevant to and actionable by payers, providers, and patients.
Pragmatic trials seek to inform choices between feasible alternatives by estimating real world outcome probabilities, taking into account the interests of patients and value in the broadest sense. Explanatory trials, on the other hand, seek to estimate the maximum possible effect of an intervention, and to understand how and why the effect occurs.[1] Other terms are also used to describe pragmatic-type trials, including practical clinical trials, large simple trials, mega-trials, management trials, effectiveness trials, or naturalistic trials. All aim to identify the risks and benefits of interventions outside of the ideal conditions of the classic randomized clinical trial (RCT) but still using randomization to limit the impact of bias on the results.
Pragmatic trials are attractive because they have the potential to provide significant cost savings for sponsors (because usual care and follow-up data collection after randomization is almost entirely what the patient would receive if they were not in the trial). Sponsors are under pressure to look at study design and delivery approaches to minimize costs, and maximize value, while optimizing trial design to answer relevant research questions. This approach falls squarely within the scope of comparative effectiveness research (CER), which is defined as research that informs decisions, comparing alternatives with the potential to be best practice, and measuring outcomes that are important to patients in real-world settings.
When designed and conducted properly, data from these studies can assist in payer reimbursement decisions and patient and clinician choices, and have a significant influence on patient health outcomes.
Pragmatic trial publication rate rising
Overall, the results from around 7,000 RCTs were published in 1990, rising to 22,000 in 2010 (Figure 1).[2] The red dots in Figure 1 illustrate the percentage of published trials that had the words ‘pragmatic’ or ‘practical’ in the title or abstract. Even though this has risen at an accelerating rate in recent years, it still accounts for only just over 1% of total number of trials published.
![]()
Domains of Pragmatism
Pragmatic clinical trials can be viewed as the RCT analogue of CER or patient-centered outcomes research. Figure 2 illustrates one approach to defining the primary domains of pragmatism, which are the features of study design that can be more or less pragmatic or explanatory.
Figure 2: Domains of Pragmatism[3]
While certain trials may be highly pragmatic across all these domains, in reality, most trials fall somewhere along the continuum for each domain between the two extremes of pragmatic and explanatory. In some cases, for example, a trial may only be highly pragmatic in having very few exclusion criteria, broad inclusion criteria, and extensive reporting of quality of life and patient-reported outcomes.
Case history: Payer denial of reimbursement
An example of where reimbursement was denied based on patient eligibility and the choice of study comparator is Medicare’s denial of reimbursement for the cervical artificial disc, despite data from an Food and Drug Administration approved premarketing authorization (PMA) that were sufficient for approval of this product as a class 3 device. Factors cited in the non-coverage decision were:
Measuring more pragmatic outcomes
Psoriasis trials provide a good illustration of the need for more emphasis on “patient-focused” drug development – something that the FDA is now actively exploring as a result of recent legislative directives. Regulatory studies of drugs to treat severe psoriasis are required to report as a primary outcome the extent and severity of total body surface that is affected by plaques (using a scale referred to as the psoriasis area and severity index). In fact, patient interviews surveys suggest that the extent of face and joint involvement have the biggest impact on quality of life – not the overall extent of skin surface area affected. A more pragmatic design would therefore include a reliable measure of face/joint severity as a secondary endpoint, making it a high priority to develop and validate such a metric for eventual use as a primary endpoint.
Looking ahead, it will be increasingly important for Phase 3 and 4 trials to be designed in ways that are more reflective of the evidence expectations of for payers, clinicians and patients. Early input is needed, in forums such as payer or patient advisory boards, and ideally multi-stakeholder forums will generate guidance for the design of pragmatic trials that reflect input across all of these decision makers.
Case history: Patient reported outcomes in pragmatic oncology trials
To guide the design of explanatory trials, regulatory standards exist for patient reported outcomes and other domains of trial design, with guidance documents available from both the FDA and European Medicines Agency. This guidance tends to be stringent, intended to be informative for the design of studies to support the drug approval process and product labeling. The guidance establishes useful methodological standards that meet the needs of regulators, but is unlikely to be applicable to CER or fully informative for decision makers whose interests extend beyond what should appear on the label.
The Patient Centered Outcomes Research Institute (PCORI) patient centeredness standards advise researchers to “measure outcomes that people representing the population of interest notice and care about.” While this is useful guidance, it is not specific enough to assist researchers in designing good CER studies. The Center for Medical Technology Policy has recently established the Green Park Collaborative to develop condition-specific guidance for the design of pragmatic trials across a range of clinical conditions. These recommendations are summarized in the form of Effectiveness Guidance Documents (EGDs). These:
Figure 3: General Effectiveness Guidance Document Process
![]()
One such EGD addresses patient-reported outcomes in oncology drug trials, and provides recommendations to include a core set of patient-reported symptoms. Specifically, 14 symptoms were identified as essential to be measured and reported in trials of drugs developed for the treatment of advanced cancers.[4] These should be included in all Phase 3 and 4 trials of drugs to treat adult oncology indications. The high level of specificity in this EGD is intended to help researchers design more informative and consistent studies, and provide patients and clinicians with highly relevant information about these products that would not be generated from trials designed solely based on regulatory guidance.
Another recommendation from this EGD is to measure and report health-related quality of life (HRQL) in these trials, with the rationale that the global patient subjective experience with treatment is essential to understand the benefits and risks in real-world contexts. The guidance also recommended that the time required for collection of PRO data be limited so that the average patient will be able to complete the data collection process in a reasonable amount of time.
The features of EGDs that should uptake of these EGD recommendations could include:
The Health Care Systems (HCS) Collaboratory has recently been set up to strengthen national capacity to implement cost-effective large-scale research studies that engage healthcare delivery organizations. It is also intended to support design and rapid execution of high impact Demonstration Projects – with seven initially funded – that will be conducted in partnership with delivery systems. The initiative will make available data, tools and resources for research partnerships with HCSs. Electronic health records will be used for core data collection, with at least two integrated health systems collaborating on each Project. One of the initial topics is an examination of the impact of taking antihypertensive medications in the evening rather than the morning.
A major challenge across all seven projects is the tendency of Institutional Review Boards to impose the same level of oversight to these very low-risk, simple interventions as they do to a phase III drug trial, adding unnecessary delays and complexity. A logical approach could be to put this type of study into a separate category with streamlined review.
Conclusion
For future success, a very efficient infrastructure will be required to successfully implement these simple pragmatic trials in the context of healthcare delivery. As pressures mount to provide evidence of value, collaborative efforts between stakeholders are needed to identify appropriate endpoints and metrics, informing biopharma development decisions, and providing greater certainty for companies developing products that they will meet unmet needs and gain market access.
Sean Tunis, M.D., M.Sc., President and Chief Executive Officer, Center for Medical Technology Policy.
Large pragmatic trials are on the fast track to becoming a key vehicle for providing real-world outcomes data for comparing alternative treatment pathways. This responds to growing market pressure for companies to conduct studies that are more representative of usual care and results that are more relevant to and actionable by payers, providers, and patients.
Pragmatic trials seek to inform choices between feasible alternatives by estimating real world outcome probabilities, taking into account the interests of patients and value in the broadest sense. Explanatory trials, on the other hand, seek to estimate the maximum possible effect of an intervention, and to understand how and why the effect occurs.[1] Other terms are also used to describe pragmatic-type trials, including practical clinical trials, large simple trials, mega-trials, management trials, effectiveness trials, or naturalistic trials. All aim to identify the risks and benefits of interventions outside of the ideal conditions of the classic randomized clinical trial (RCT) but still using randomization to limit the impact of bias on the results.
Pragmatic trials are attractive because they have the potential to provide significant cost savings for sponsors (because usual care and follow-up data collection after randomization is almost entirely what the patient would receive if they were not in the trial). Sponsors are under pressure to look at study design and delivery approaches to minimize costs, and maximize value, while optimizing trial design to answer relevant research questions. This approach falls squarely within the scope of comparative effectiveness research (CER), which is defined as research that informs decisions, comparing alternatives with the potential to be best practice, and measuring outcomes that are important to patients in real-world settings.
When designed and conducted properly, data from these studies can assist in payer reimbursement decisions and patient and clinician choices, and have a significant influence on patient health outcomes.
Pragmatic trial publication rate rising
Overall, the results from around 7,000 RCTs were published in 1990, rising to 22,000 in 2010 (Figure 1).[2] The red dots in Figure 1 illustrate the percentage of published trials that had the words ‘pragmatic’ or ‘practical’ in the title or abstract. Even though this has risen at an accelerating rate in recent years, it still accounts for only just over 1% of total number of trials published.
Domains of Pragmatism
Pragmatic clinical trials can be viewed as the RCT analogue of CER or patient-centered outcomes research. Figure 2 illustrates one approach to defining the primary domains of pragmatism, which are the features of study design that can be more or less pragmatic or explanatory.
Figure 2: Domains of Pragmatism[3]
| Domain | Pragmatic | Explanatory |
| Patient eligibility criteria | Broad | Narrow |
| Intervention flexibility | Leave to practitioners | Control by protocol |
| Intervention comparator | Compare to usual practice or best available alternative | Inflexible comparison; possibly placebo |
| Practitioner expertise with intervention | Reflect usual variation in practice | Only expert practitioners |
| Follow-up intensity | Usual practice; no special requirements | Follow-up visits frequent, mandated; event triggers |
| Patient compliance | Unobtrusive measurements | Closely monitored and enforced |
| Primary outcome | Patient-important health benefit, quality of life | Direct consequence of intervention; surrogate |
| Data analysis | Includes all patients | Intent-to-treat; but also ‘compliers’ or subgroups with maximum treatment effect |
Case history: Payer denial of reimbursement
An example of where reimbursement was denied based on patient eligibility and the choice of study comparator is Medicare’s denial of reimbursement for the cervical artificial disc, despite data from an Food and Drug Administration approved premarketing authorization (PMA) that were sufficient for approval of this product as a class 3 device. Factors cited in the non-coverage decision were:
- The trial was limited to patients aged 18 to 60 (which from a Medicare policy point of view was not relevant, because Medicare eligibility occurs at age 65 and over)
- The non-inferiority comparison to spinal fusion was determined by Medicare to not be the most clinical relevant alternative – since spinal fusion had never been shown to be superior to non-surgical therapy.
Measuring more pragmatic outcomes
Psoriasis trials provide a good illustration of the need for more emphasis on “patient-focused” drug development – something that the FDA is now actively exploring as a result of recent legislative directives. Regulatory studies of drugs to treat severe psoriasis are required to report as a primary outcome the extent and severity of total body surface that is affected by plaques (using a scale referred to as the psoriasis area and severity index). In fact, patient interviews surveys suggest that the extent of face and joint involvement have the biggest impact on quality of life – not the overall extent of skin surface area affected. A more pragmatic design would therefore include a reliable measure of face/joint severity as a secondary endpoint, making it a high priority to develop and validate such a metric for eventual use as a primary endpoint.
Looking ahead, it will be increasingly important for Phase 3 and 4 trials to be designed in ways that are more reflective of the evidence expectations of for payers, clinicians and patients. Early input is needed, in forums such as payer or patient advisory boards, and ideally multi-stakeholder forums will generate guidance for the design of pragmatic trials that reflect input across all of these decision makers.
Case history: Patient reported outcomes in pragmatic oncology trials
To guide the design of explanatory trials, regulatory standards exist for patient reported outcomes and other domains of trial design, with guidance documents available from both the FDA and European Medicines Agency. This guidance tends to be stringent, intended to be informative for the design of studies to support the drug approval process and product labeling. The guidance establishes useful methodological standards that meet the needs of regulators, but is unlikely to be applicable to CER or fully informative for decision makers whose interests extend beyond what should appear on the label.
The Patient Centered Outcomes Research Institute (PCORI) patient centeredness standards advise researchers to “measure outcomes that people representing the population of interest notice and care about.” While this is useful guidance, it is not specific enough to assist researchers in designing good CER studies. The Center for Medical Technology Policy has recently established the Green Park Collaborative to develop condition-specific guidance for the design of pragmatic trials across a range of clinical conditions. These recommendations are summarized in the form of Effectiveness Guidance Documents (EGDs). These:
- Provide specific recommendations on design, analysis and use of pragmatic trials
- Are explicitly intended to inform the design of studies intended for decisions by patients, clinicians, clinical practice guideline developers and payers
- Are analogous and complementary to regulatory guidance documents, but focused on specific disease areas to align with the evidence preferences of patients and payers
- Are more context-specific than PCORI or Agency for Healthcare Research and Quality (AHRQ) methods standards.
Figure 3: General Effectiveness Guidance Document Process
One such EGD addresses patient-reported outcomes in oncology drug trials, and provides recommendations to include a core set of patient-reported symptoms. Specifically, 14 symptoms were identified as essential to be measured and reported in trials of drugs developed for the treatment of advanced cancers.[4] These should be included in all Phase 3 and 4 trials of drugs to treat adult oncology indications. The high level of specificity in this EGD is intended to help researchers design more informative and consistent studies, and provide patients and clinicians with highly relevant information about these products that would not be generated from trials designed solely based on regulatory guidance.
Another recommendation from this EGD is to measure and report health-related quality of life (HRQL) in these trials, with the rationale that the global patient subjective experience with treatment is essential to understand the benefits and risks in real-world contexts. The guidance also recommended that the time required for collection of PRO data be limited so that the average patient will be able to complete the data collection process in a reasonable amount of time.
The features of EGDs that should uptake of these EGD recommendations could include:
- The use of an inclusive, transparent process in which all stakeholders have the opportunity to provide input, with the goal of reassuring product developers that the recommendations accurately reflect the evidentiary needs of key decision makers.
- Endorsement by payers or guideline developers should provide further reassurance to researchers that adopting the EGD recommendations will ensure that studies are appropriately designed
- Publications in peer reviewed journals further enhance the credibility of EGD recommendations
- Incorporation into EGD recommendations into high profile CER/PCOR activities, will also encourage use in clinical trials, such as:
- Recognition by the National Cancer Institute as core measures for registries and other data collection efforts
- Use in clinical trials conducted by NIH-funded Cooperative group
- Acknowledgement by PCORI, AHRQ, and other organizations that produce methodological standards.
The Health Care Systems (HCS) Collaboratory has recently been set up to strengthen national capacity to implement cost-effective large-scale research studies that engage healthcare delivery organizations. It is also intended to support design and rapid execution of high impact Demonstration Projects – with seven initially funded – that will be conducted in partnership with delivery systems. The initiative will make available data, tools and resources for research partnerships with HCSs. Electronic health records will be used for core data collection, with at least two integrated health systems collaborating on each Project. One of the initial topics is an examination of the impact of taking antihypertensive medications in the evening rather than the morning.
A major challenge across all seven projects is the tendency of Institutional Review Boards to impose the same level of oversight to these very low-risk, simple interventions as they do to a phase III drug trial, adding unnecessary delays and complexity. A logical approach could be to put this type of study into a separate category with streamlined review.
Conclusion
For future success, a very efficient infrastructure will be required to successfully implement these simple pragmatic trials in the context of healthcare delivery. As pressures mount to provide evidence of value, collaborative efforts between stakeholders are needed to identify appropriate endpoints and metrics, informing biopharma development decisions, and providing greater certainty for companies developing products that they will meet unmet needs and gain market access.
[1] Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chron Dis 1967; 20: 637–648.
[2] Chalkidou K, Tunis S, Whicher D, Fowler R, Zwarenstein M. The role for pragmatic randomized controlled trials (pRCTs) in comparative effectiveness research. Clin Trials August 2012 vol. 9 no. 4 436-446. Published online before print July 2, 2012, doi: 10.1177/1740774512450097
[3] Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K. A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers. Journal of Clinical Epidemiology, Volume 62, Issue 5 , Pages 464-475, May 2009.
[4] Basch E, Abernethy AP, Mullins CD, Reeve BB, Smith ML, Coons SJ, Sloan J, Wenzel K, Chauhan C, Eppard W, Frank ES, Lipscomb J, Raymond SA, Spencer M, Tunis S. Recommendations for Incorporating Patient-Reported Outcomes Into Clinical Comparative Effectiveness Research in Adult Oncology. Journal of Clinical Oncology, October 15, 2012 JCO.2012.42.5967
.png)
No hay comentarios:
Publicar un comentario