Toward Fairness in Data Sharing

The International Consortium of Investigators for Fairness in Trial Data Sharing

N Engl J Med 2016; 375:405-407August 4, 2016DOI: 10.1056/NEJMp1605654

Comments open through August 10, 2016

Article
References
Citing Articles (1)
Comments (16)
Metrics: The International Committee of Medical Journal Editors (ICMJE) has proposed a plan for sharing data from randomized, controlled trials (RCTs) that will require, as a condition of acceptance of trial results for publication, that authors make publicly available the deidentified individual patient data underlying the analyses reported in an article.1 Before any data-sharing policy is enacted, we believe there is a need for the ICMJE, trialists, and other stakeholders to discuss the potential benefits, risks, and opportunity costs, as well as whether the same goals can be achieved by simpler means. Although we believe there are potential benefits to sharing data (e.g., occasional new discoveries), we believe there are also risks (e.g., misleading or inaccurate analyses and analyses aimed at unfairly discrediting or undermining the original publication) and opportunity costs (e.g., the ICMJE proposal would have enormous direct costs and would probably divert resources, both financial and human, from the actual conduct of trials). In 2010 alone, results of more than 27,000 RCTs were published.2 We believe consideration needs to be given to whether it is worthwhile to undertake data sharing for all published trials or just for those whose results are under question or those that are likely to influence care.
At least for large trials, there may be a case for sharing data in an appropriate and timely manner, but we do not support the ICMJE proposal as it currently stands. We believe that alternative approaches can achieve the benefits of data sharing (in particular, confirmation of the original findings and testing of new hypotheses) without the unintended adverse consequences that may result from the ICMJE proposal.
To complete an RCT, investigators must develop a protocol, obtain funding, overcome regulatory and bureaucratic challenges, recruit and follow participants, undertake analyses, and publish the results. This process takes several years, and for large clinical trials it can sometimes take a decade or longer. Adequate incentives for researchers to invest the substantial time and effort required to conduct RCTs and to publish the results in a timely fashion are important. The current ICMJE proposal requires that the data underlying the published results be made available for sharing within 6 months after the publication date. We believe that this interval is too short.
A key motivation for investigators to conduct RCTs is the ability to publish not only the primary trial report, but also major secondary articles based on the trial data. The original investigators almost always intend to undertake additional analyses of the data and explore new hypotheses. Moreover, large, multicenter trials with large numbers of investigators often require several articles to fully describe the results. These investigators are partly motivated by opportunities to lead these secondary publications. We believe 6 months is insufficient for performing the extensive analyses needed to adequately comprehend the data and publish even a few articles. Once the investigators who have conducted the trial no longer have exclusive access to the data, they will effectively be competing with people who have not contributed to the substantial efforts and often years of work required to conduct the trial.
The current ICMJE proposal therefore risks reducing the incentive for coinvestigators and site investigators to participate in trials. A reduction in the number of investigators willing to recruit patients into a trial or supervise its conduct in a country will compromise the likelihood of successful completion of large trials that address important questions.
The ICMJE proposal may also lead some investigators to delay publishing their primary trial results to allow time to prepare several secondary manuscripts. Delay or failure to publish the primary results of trials is already a substantial problem.3 We believe that the ICMJE’s plan is likely to exacerbate this problem.
If a policy on data sharing is to be implemented, then the timing of access to trial data should take into account the time required to complete the trial. We propose that study investigators be allowed exclusive use of the data for a minimum of 2 years after publication of the primary trial results and an additional 6 months for every year it took to complete the trial, with a maximum of 5 years before trial data are made available to those who were not involved in the trial. This approach would result in data release within 2.5 years for many small trials and within 5 years for many large ones. Such an approach would provide trial investigators a reasonable amount of time — consistent with their efforts — to explore the data they generated and would create an incentive to conduct RCTs and avoid delaying initial publication of their results.
One way to ensure confidence and trust in published trial data is for independent confirmatory analyses to be undertaken. However, making data freely available provides no guarantee that such analyses will be performed. Moreover, we believe that the best way to ensure that readers have confirmation of the validity of trial results is for journals to arrange for independent analyses. The ICMJE proposal does not include such a requirement and therefore will not provide readers the assurance they may want regarding data confirmation when it is most important — at the time they are reading the original publication, should this be deemed necessary (see the Supplementary Appendix, available at NEJM.org.).
Many trialists conducting investigator-initiated trials spend substantial amounts of money they have generated from other activities to cross-subsidize trials that are not of interest to commercial sponsors. Investigators should be able to recoup some of their costs from charges to people who seek access to their data. Furthermore, a mechanism will be needed to fund the data-preparation activities necessary for data sharing in such a way as to protect confidentiality and ensure data integrity (see the Supplementary Appendix).
The table in the Supplementary Appendix outlines our proposed process for accessing and publishing trial data. We believe that once data are released for public use after the appropriate interval, the deidentified trial data should be housed either in a reliable third-party data repository or at the trialists’ center. Whoever hosts the data will need to implement mechanisms to manage data requests in a timely and fair manner, avoid duplication of efforts, and ensure that such analyses are accurate and not conducted with the aim of inappropriately undermining the original findings. A review committee should evaluate all data requests and assess conflicts of interest; this committee should have representation both of investigators involved in the trial and academic trialists who did not participate in it. Furthermore, mechanisms are needed to ensure that the analyses conducted are accurate. To avoid misuse of the data, persons requesting data should agree that they will use them only for the approved purposes outlined in the statistical analysis plan; if they wish to undertake further analyses for publication, they should be required to submit another request to the review committee. To resolve any disputes related to data sharing, the ICMJE could appoint an ombudsman.
In summary, we recommend that the ICMJE come together with trialists and other stakeholders to discuss the potential benefits, risks, burdens, and opportunity costs of its proposal and explore alternatives that will achieve the same goals efficiently. Moreover, we recommend modifying the proposal as follows. First, the timeline for providing deidentified individual patient data should allow a minimum of 2 years after the first publication of the results and an additional 6 months for every year required to complete the study, up to a maximum of 5 years. Second, to enhance readers’ confidence in published data, an independent statistician should have the opportunity to conduct confirmatory analyses before publication of an article, thereby advancing the ICMJE’s stated goal of increasing “confidence and trust in the conclusions drawn from clinical trials.” Finally, persons who were not involved in an investigator-initiated trial but want access to the data should financially compensate the original investigators for their efforts and investments in the trial and the costs of making the data available.
Disclosure forms provided by the authors are available at NEJM.org.
The writing committee of the International Consortium of Investigators for Fairness in Trial Data Sharing included P.J. Devereaux, M.D., Ph.D., Gordon Guyatt, M.D., Hertzel Gerstein, M.D., Stuart Connolly, M.D., and Salim Yusuf, M.B., B.S., D.Phil. — all from McMaster University, Hamilton, ON, Canada. This article was reviewed and endorsed by 282 investigators in 33 countries, who are listed in the Supplementary Appendix.