AHRQ Public Access to Federally Funded Research

Publications and Data

This document is the Agency for Healthcare Research and Quality's (AHRQ's) plan for establishing a policy for public access to scientific publications and scientific data in digital format resulting from AHRQ funding.

1. Background & Purpose

On February 22, 2013, the White House Office of Science and Technology Policy (OSTP) released the memorandum entitled "Increasing Access to the Results of Federally Funded Scientific Research." This memorandum requires Federal agencies to make the results of federally funded scientific research available to and useful for the public, industry, and the scientific community. This document is the Agency for Healthcare Research and Quality's (AHRQ's) plan for establishing a policy for public access to scientific publications and scientific data in digital format resulting from AHRQ funding. AHRQ intends, to the fullest extent possible and contingent on the availability of funds, to make available to the public all scientific publications and data arising from unclassified research and programs funded wholly or in part by AHRQ.
The AHRQ mission is to produce evidence that makes health care safer, higher quality, more accessible, equitable and affordable, and to work within the U.S. Department of Health and Human Services (HHS) and with other partners to make sure the evidence is understood and used.
The AHRQ Public Access Policy promotes the following objectives:

Establish AHRQ's commitment to public access to scientific research results.
Ensure that the public can read, download, and analyze in digital format final published documents.
Facilitate easy public search, analysis of and access to peer-reviewed scholarly publications directly arising from research funded by AHRQ.
Ensure full public access to publications' metadata without charge.
Ensure the attributes to authors, journals, and original publishers are maintained.
Ensure that publications and metadata are in an archival solution.
Ensure that all extramural researchers receiving AHRQ grants, cooperative agreements, and contracts for scientific research and intramural researchers develop data management plans, as appropriate, describing how they will provide for long-term preservation of and access to scientific data in digital format.
Ensure the expedited translation of research results into knowledge, products, and procedures to improve health.
Facilitate the education of new researchers and enable the exploration of topics not envisioned by the initial investigators.
Permit the creation of new datasets when data from multiple sources are combined.

Top of Page

2. Scope

The AHRQ Public Access Policy will be fully effective by October 2015. The Public Access Policy will apply to all scientific publications and data in digital format. Implementation will be prospective and will not apply to any publication or digital data set arising from an AHRQ-sponsored grant, cooperative agreement, contract, or intramural research project funded prior to publication of the final AHRQ Public Access Policy.
The AHRQ Public Access Policy will apply to all research funded by AHRQ, except where such research is administered or performed by a partner Agency with a comparable Public Access Policy, in which case AHRQ will defer to the partner Agency's policies on the management of scholarly publications and digital data sets.
Digital scientific data that are covered by this policy include:

Field data.
Lab data.
Other data (e.g., quality control samples, sample ID data, and instrument calibration data).

Digital scientific, clinical, or institutional data that can be in scope at the discretion of the program or in an appropriate context:

Models and model-related content, including parameters and outputs, including models of public health emergencies.
Command files, such as SAS or STATA files created from secondary sources (typically referred to as secondary or outside data).
De-identified Electronic Health Record (EHR) patient treatment records.
Non-proprietary records and data collected as part of the National Hospital Preparedness Program.

Digital scientific data that are not in scope for this policy include:

Personally identifiable data, however, AHRQ will make an attempt to make all research data available to the public by creating de-identified public use data files.
Proprietary trade data.
Data related to protecting critical infrastructure.
Other data whose release is limited by law, regulation, security requirements, or policy.

Per OMB Circular A-110, the following are not research data and therefore not subject to this policy: preliminary analyses, drafts of scientific papers, plans for future research, peer reviews, communications with colleagues, or physical objects, such as laboratory specimens. However, investigators are encouraged to include relevant content from laboratory or field notebooks in their published data sets if that information provides documentation that would help future users successfully re-use the data. (That is, lab and field notebooks are not themselves scientific data, but may contain relevant metadata to be included in the published data set.)
Top of Page

3. Requirements

Publications

For scholarly publications, the AHRQ Public Access Policy will require that authors submit the final peer-reviewed accepted journal manuscripts to PubMed Central (PMC). In lieu of the final peer-reviewed manuscript, AHRQ will accept the final published article, provided the awardee can ensure AHRQ has the rights to make the published version public. AHRQ's Public Access Policy is subject to law; Agency mission; resource constraints; U.S. national, homeland, and economic security; and the objectives listed in the OSTP directive.

Digital data

To the extent feasible and consistent with applicable law and policy; Agency mission; resource constraints; U.S. national, homeland, and economic security; and the objectives listed below, digitally formatted scientific data resulting from unclassified research supported wholly or in part by Federal funding should be stored and publicly accessible to search, retrieve, and analyze. For sharing of data in digital format, all AHRQ-funded researchers will be required to include a data management plan for sharing final research data in digital format, or state why data sharing is not possible. Data management plans will be evaluated by AHRQ staff and peer review committees for merit while considering the values of long-term preservation, access, and the associated cost, and administrative burden. AHRQ expects the timely release and sharing of data to be no later than the acceptance for publication of the main findings from the final dataset. The specific time will be influenced by the nature of the data collected. Researchers will be directed to work with AHRQ staff and its contracted commercial repository to deposit data upon or prior to the embargo period. It may not be feasible to share all data; the costs and benefits of sharing data should be considered in data management planning. AHRQ intramural research is conducted using public use data. These data are also available to the public. AHRQ data that include personally identifiable data are available to the public in the AHRQ data center. AHRQ will attempt to make all research data available to the public by creating de-identified public use data files.
If an AHRQ-funded grant, cooperative agreement, contract or intramural research project transforms or links datasets, rather than producing a new set of data, the researchers will be required to submit a data management plan. If there are limitations associated with the data sharing agreements for the original data that preclude subsequent sharing, the research applicants must explain these limitations.
Data from small studies can be analyzed and submitted for publication relatively quickly. If data from large epidemiologic or longitudinal studies are collected over several discrete time periods or waves, it is reasonable to expect that the data would be released in waves as data become available or main findings from waves of the data are published. AHRQ-funded extramural and intramural researchers will be expected to include updates on public access to publications and data in digital format resulting from their research in quarterly reports, where applicable, annual progress reports, final reports, and via NLM's Bibliography tool, My NCBI. All new grant and cooperative agreement applications, and contract proposals will include a section detailing the applicants' progress to-date with sharing of publication and data in digital format arising from previous AHRQ funding.
In order to ensure long-term preservation and full access to the public, AHRQ will contract with a commercial repository to accept and manage data submitted by extramural, intramural, and contract researchers. The commercial repository and AHRQ staff will coordinate with the researcher upon notification of funding, prior to the start of the research project, and throughout its life cycle to ensure the current usability, long-run preservation and access to the data. Data will be made available in a digital format free of charge to the public via the selected commercial repository. If the data is made available via another mechanism, AHRQ will publicize the location of the data to the public on its Web site and provide a link to the data. Information on how to access all AHRQ's funded data will be provided on AHRQ's Web site with links to the data.

White House Office of Management and Budget Open Data Policy Memorandum (M-13-13)

AHRQ will meet the requirements of M-13-13 as specified in the accompanying memorandum, titled HHS Guiding Principles and Common Approach for Enhancing Public Access. AHRQ will develop and maintain an AHRQ Enterprise Data Inventory for the data sets related to their major Information Technology investments. AHRQ will develop and maintain an AHRQ Enterprise Data Inventory, cataloguing the various sets of data acquired or managed. The inventory will catalog both data sets that are made publicly available and data sets that are private to the Agency.
Top of Page

4. Definitions

Research Data: Research data is defined in OMB Circular A-110 as the recorded factual material commonly accepted in the scientific community as necessary to validate research findings, but not any of the following: preliminary analyses, drafts of scientific papers, plans for future research, peer reviews, or communications with colleagues. This 'recorded' material excludes physical objects (e.g., laboratory samples).
Research data also do not include:

Trade secrets, commercial information, materials necessary to be held confidential by a researcher until they are published, or similar information which is protected under law.
Personnel and medical information and similar information, the disclosure of which would constitute a clearly unwarranted invasion of personal privacy, such as information that could be used to identify a particular person in a research study.

Digital Scientific Data: For the purpose of this plan and consistent with OMB Circular A-110, digital scientific data is defined as "the digital recorded factual material commonly accepted in the scientific community as necessary to validate research findings including data sets used to support scholarly publications, but does not include laboratory notebooks, preliminary analyses, drafts of scientific papers, plans for future research, peer reviews, communications with colleagues, or physical objects, such as laboratory specimens."
For the purpose of this plan, the definition of digital scientific data does not include software. However, AHRQ recognizes that in some cases, software and other tools such as interview protocols, measures, coding guides, or manuals may be necessary to interpret data. In such cases, the data management plan will be expected to include a description of these tools.
A published data set consists of at least one formal metadata document, the digital scientific data described by that metadata, and supplemental information provided to assist the data user. The metadata for scientific data will include, at a minimum, the common core metadata schema in use by the Federal Government, found at https://project-open-data.cio.gov/.
A published data set can be cited in the scientific literature and, following the practice used by most publishers of scholarly journals, has a persistent and unique identifier associated with it. A published data set is expected to persist over time, just like the scholarly articles that are based on the digital scientific data.
Data Sharing Plan: A data sharing plan outlines whether and how data will be made available to others. The plan must also explicitly describe how the data that underlies scientific publications will be available for discovery, retrieval, and analysis. It may include the expected timeline for when the data will be available, the format of the final dataset, the documentation and any analytic tools that will be provided, and the mode of data sharing (e.g., under their own auspices by mailing a disk or posting data on their institutional or personal Web site, through a data archive or enclave). A plan might also specify whether or not a data sharing agreement will be required and, if so, a brief description of such an agreement (including the criteria for deciding who can receive the data and whether or not any conditions will be placed on their use). If data sharing is not possible, the plan would provide an explanation.
Data Management Plan: Data management plans are more comprehensive than data sharing plans in that they include additional elements such as descriptions of the data to be produced in the proposed study, any standards to be used for collected data and metadata, mechanisms for providing access to and sharing of the data (including provisions for protection of privacy, confidentiality, security, intellectual property, or other rights), provisions for reuse and redistribution, and plans for archiving and long-term preservation of the data, or explaining why long-term preservation and access cannot be justified.
Final peer-reviewed manuscript: A final peer-reviewed manuscript is defined as an author's final manuscript of a peer-reviewed paper accepted for journal publication, including all modifications from the peer-review process.
Final published article: A final published article is defined as a publisher's authoritative copy of the paper, including all modifications from the publishing peer-review process, copy editing, stylistic edits, and formatting changes.
Top of Page

5. Applicability

The AHRQ Public Access Policy will apply to the following groups:

All AHRQ intramural researchers.
Award recipients of AHRQ grants and cooperative agreements.
All AHRQ contractors conducting research.

Top of Page

6. Legal Authorities

Section 902(a) of the Public Health Service Act, 42 U.S.C. 299a(a).
Section 944(a) of the Public Health Service Act, 42 U.S.C. 299c -3(a).
Section 465 of the Public Health Service Act, 42 U.S.C. 286.
42 C.F.R. Part 67.
45 C.F.R. Part 74.
Section 103 of The America COMPETES Reauthorization Act of 2010 (Pub. L. 111-358).

Top of Page

7. Roles and Responsibilities

The Director of AHRQ:

Assigns overall responsibility for establishing, managing, implementing, and evaluating public access to scholarly publications and data in digital format funded by AHRQ.
Assigns overall responsibility for establishing, managing, implementing, and evaluating the AHRQ public access policy.

AHRQ Director of the Office of Extramural Research, Education and Priority Populations (OEREP):

Assumes AHRQ-wide authority, control, and management of the Public Access Policy.
Manages the implementation of this policy.
Coordinates with AHRQ's Grants and Contracts offices to ensure the requirements of this policy are included as terms and conditions of funding for grants, contracts, and cooperative agreements.
Develops practical funding strategies to ensure the successful implementation of this policy.
Coordinates the collaboration and cooperation of public access and data sharing with other Federal agencies.
Ensures compliance with this Public Access Policy.

Grant, Cooperative Agreement, and Contract Awardees and their institutions:

Ensure that authors are aware of and comply with the AHRQ Public Access Policy.
Ensure all terms and conditions of awards are met.
Submit data management plan with grant, cooperative agreement, and contract applications.
Include reasonable cost for public access to publications and data in digital format as a component of grant, cooperative agreements, and research contract applications.

Authors:

Work with the publisher before any publication rights are transferred to ensure that all conditions of the AHRQ Public Access Policy can be met.

Top of Page

8. Implementation

This Public Access Plan includes the following implementation steps.

Planning

AHRQ will collaborate and cooperate with HHS and Federal Government-wide agencies to use a transparent process for soliciting views from stakeholders, including federally funded researchers, universities, libraries, publishers, users of federally funded researcher results, and civil society groups, and take such views into account. AHRQ will contract with NLM to use PMC. PMC will allow AHRQ to engage in public-private partnerships, since PMC is a public-private partnership. AHRQ's Public Access Policy will be beneficial to businesses or will otherwise enhance innovation and competitiveness to satisfy a requirement in section 3 of the OSTP memo.
The AHRQ Public Access Policy development process will include the following:

Establish an iterative process of policy design, planning, implementation, evaluation/impact assessment, and policy redesign.
Work in full and open consultation with all stakeholders, including other Federal organizing bodies, to develop, maintain, and improve the Public Access Policy.
Explore new approaches and partnerships with publishers and other stakeholders to obtain final peer-reviewed manuscripts or published articles.
Provide full public access to publications no later than 12 months after the official date of publication. AHRQ will contract with the NLM to archive its publications in PubMed Central (PMC).
PMC is a public-private partnership to preserve and make public full-text journal articles in a nonproprietary and widely-distributed archival format. PMC makes the abstract and citation information available to NLM's abstract and metadata service, PubMed, where it can be accessed without charge. PubMed makes metadata available upon and combines them with any indexing terms or key works after the embargo period.
Metadata of texts will also be stored in nonproprietary or widely distributed archival formats.
Maintain attribution to authors, journals, and original publishers through the use of PMC and PubMed. PubMed abstracts include a publisher-supplied link to the paper on a publisher site and a link to the PMC version if available. PubMed metadata, like the full text of PMC, are permanently archived. Full text displays of papers on PMC always include publisher-supplied links to the publisher site, authors, journal name and the other information required for a complete academic citation.
Establish a mechanism for compliance with the policy.
Ensure the public can search, read, and download the final peer-reviewed manuscripts or published articles without charge no later than 12 months following publication.

AHRQ will post the final Public Access Policy on the AHRQ Web site. AHRQ will also publish the final Public Access Policy in the NIH Guide for Grants and Contracts. In order to ensure full integration of the Public Access Policy with the AHRQ research and award management systems, AHRQ will partner with the NIH to use PMC as the repository of publications resulting from AHRQ-funded grants, contracts, cooperative agreements, and intramural research.
AHRQ will include requirements to submit publications in accordance with the Public Access Policy in contract solicitations and grant Funding Opportunity Announcements. AHRQ will use PMC as an approach for optimizing search, archival, and dissemination features that encourage innovation in accessibility and interoperability, while ensuring long-term stewardship of the results of federally funded research. It will also encourage public-private collaboration to maximize the impact of the Federal research investment through the use of PMC. The significant usage of PMC underscores the value of public access to the peer-reviewed scientific literature.
The NLM databases, with their XML data accessible through a variety of Web services, can also support additional economic activity. NLM has found that more than 300 private sector firms obtain data from Medline for a variety of applications: to supplement literature database products and search tools; to build literature analysis tools for end users of research; to conduct research in a variety biomedical areas; and to gain intelligence on experts and trends. Many others that we do not have the means to track use NLM's other resources.

Submission of Manuscripts/Publications

AHRQ will use PMC to enable the submission of final peer-reviewed manuscripts or final published articles that includes the following functionalities:

Allow users to submit and manage manuscripts directly.
Allow submission by the author, the publisher, or the manager of the funding agreement.
Accept manuscripts in a range of common electronic formats.
Accept any additional files of figures, tables, or supplementary information included with the manuscript.
Provide flexible and multiple approaches to manuscript submission.
Authors will regularly report on the status of their deposit of materials using quarterly reports, where applicable; annual progress reports; and final progress reports.

Management

AHRQ will use PMC as a repository of full-text scholarly publications. The system will:

Store, organize, and manage the publications collected or submitted under this policy.
Be established using an open architecture and follow industry standards to facilitate open government, enable integration, and promote interoperability. AHRQ currently has interoperability through its use of the NIH's grant management system eRA Commons, used by several Federal agencies and HHS operating divisions. NIH has integrated eRA Commons with public access compliance information using My NCBI.
PMC makes access to AHRQ-supported papers, as well as over two million other papers, easier to access while taking appropriate steps to prevent unauthorized mass redistribution. NLM has established sophisticated monitoring systems to detect and prevent potential misuse that are very similar to the processes used by publishers' Web sites. NLM systems detect and prevent bulk downloading and will immediately cut off any sites, foreign or domestic, that appear to be abusing copyrighted property. Unless publishers have chosen PMC as the sole distributor of the electronic contents of their journals, all articles that are accessible on PMC are also accessible on publisher Web sites.
Enable AHRQ to monitor compliance with its Public Access Policy.

Access and Discoverability

The AHRQ will establish an interagency agreement with NLM to use PMC for its primary repository for AHRQ publications. The use of PMC will ensure easy search, analysis, and download of peer-reviewed scholarly publications arising from research funded by AHRQ without charge no later than 12 months following publication. The public's ability to locate and access digital data resulting from AHRQ's federally funded scientific research as well as for publications will be improved through the use of PMC. The full text of PMC papers can be searched on PMC. In addition, the metadata (abstract, citation, and any index terms) can be searched in PubMed.
AHRQ will ensure that the system is accessible to people with disabilities and compliant with Section 508a of the Rehabilitation Act (29 USC 794d). AHRQ will provide an accommodation for final publisher PDF versions that are not 508 compliant but cannot be remediated due to copyright constraints.
AHRQ's maximum embargo period is 12 months. If a shorter embargo period is proposed for scientific works developed as a result of research funded by AHRQ, AHRQ will rely on the Health and Human Services (HHS) petition process for considering requests to shorten the embargo period for publications in a specific field. This process is described in greater detail in the HHS Guiding Principles and Common Approach to Public Access.

Preservation

Preservation is one of the Public Access Policy's primary objectives. As discussed in NIH's Public Access Policy, NLM maintains multiple backup copies of the PMC database, both onsite and at a remote, secure location.
Further, content on PMC is actively curated in that all article views are generated dynamically from the XML record of an article. In this way, every use of an article validates that the archival record is still usable and viable.

Digital Data

Planning
AHRQ will fund a contract to support a data repository to allow public access to data in digital format. AHRQ will require all primary data collected by grant, contract, or intramural research to be submitted to a data repository, unless confidentiality restrictions prevent the data from being made public. For research using secondary data that is publicly available, such as survey or claims data, AHRQ is considering allowing researchers to submit command files, such as SAS or STATA files, to the AHRQ data repository. Researchers using public use data, such as the Medical Expenditure Panel Survey (MEPS)—a set of large-scale surveys that collect data on the cost and use of health care and health insurance coverage of families and individuals, their medical providers, and employers across the United States—or the Healthcare Cost and Utilization Project (HCUP), the largest collection of nationwide and State-specific longitudinal hospital care data in the United States, will be exempt from submitting these data to the data repository. The rationale for this exemption is that if the data are already publicly available, there is no value in making additional copies of the data available; what the public and other researchers need is the SAS or STATA code used to analyze the data. AHRQ-funded research that involves data use agreements or licensing agreements that prohibit data sharing will be exempt from submitting these data to the data repository. AHRQ's timeline for implementation of a commercial repository is October 2015. AHRQ will use existing Agency resources to fund the data repository.
Management
AHRQ will maximize access, by the general public and without charge, to digitally formatted scientific data created with Federal funds by designing, implementing and maintaining a sustainable network of data management capabilities to enable discovery, appropriate use, and long-term management of digitally formatted scientific data.
This includes but is not limited to:

Establishment of a standard set of attributes for metadata to enable discovery and identification.
Development of policies to ensure the appropriate cataloguing of data sets resulting from AHRQ-funded research to make them findable, accessible, and citable.
AHRQ will ensure that data generated by AHRQ-funded research can be cited and attribution can be provided in a consistent manner. AHRQ plans to work with its contracted commercial repository to develop a data discovery index that will use unique identifiers for data sets generated by AHRQ-funded research, analogous to the PMC identification number (PMCID) that is assigned to all submitted publications resulting from AHRQ-funded research. The data discovery index will be developed after the commercial repository is in place. The identifier would provide a means of linking the data with the associated PubMed records. Such indexing of data sets would also facilitate generation of full citations to allow for differential recognition of scientific contributions (authorship on the scientific paper) and data-related contributions (authorship on the AHRQ data index entry). Additional efforts will be made to work with Federal agencies, HHS operating divisions and other public and private stakeholders to develop consistent practices for citation of data sets across scientific communities and other data set attribution systems and to develop methods for federated searching across multiple indices. AHRQ will also consult with its commercial repository to determine the best method for identifying and providing appropriate attribution to scientific data sets that are made available under the plan.
In collaboration with other Departments and Agencies, AHRQ will explore the development of a research data commons, a shared space for research output including data, software and narrative associated with AHRQ-funded research. Such a commons would adhere to the FAIR principles of Find, Access, Interoperate and Reuse for data resulting from AHRQ-funded research. The goal will be to make digital research data accessible to the public in a manner that optimizes search, archival, and dissemination features that encourage innovation in accessibility and interoperability, while ensuring long-term stewardship of research data results. A particular focus of the effort will be on making the data underlying the conclusions of peer-reviewed scientific publications resulting from federally funded scientific research available for free at the time of publication.
Development of procedures to ensure data integrity over time.

AHRQ will provide guidance to awardees and training, education, and workforce development for its employees about the new requirements related to scientific data management, analysis, storage, preservation, and stewardship in advance of the implementation date. This guidance and training will describe any required changes to the conditions of awards and a publicly available timeline for the implementation of the policy. AHRQ will also use public Web sites and internal mechanisms to disseminate information and solicit feedback before and after the implementation date. The capability for AHRQ and the science communities it supports to help one another through ongoing dialogue is expected to be a key part of a successful policy implementation.
AHRQ will modify existing policies or create new policies as necessary to establish the expectation that data management plans will be developed by all researchers whether they are funded by a grant, cooperative agreement, contract, other transaction, or intramural funds. AHRQ will include requirements to comply with the AHRQ Public Access Policy in contract solicitations and grant Funding Opportunity Announcements.
AHRQ will allow the inclusion of appropriate costs for data management and access in applications and proposals for AHRQ funding. The appropriateness of requested funding levels for data management plans will be assessed on a case-by-case basis through technical evaluation, and by program and grants management staff. Comparable assessments will need to be explored for intramural researchers. AHRQ will confer with interagency partners to estimate the costs of current data management activities in order to determine how to support future data management.
In order to ensure public access to data in digital format resulting from AHRQ-funded grants, contracts, and cooperative agreements AHRQ will require all research applicants to include a data management plan as a component of their grant applications or contract proposals. Intramural researchers will also be required to submit a data management plan, where applicable.
AHRQ will require periodic reporting on compliance with the approved data management plans as part of the terms and conditions of award and will develop mechanisms to monitor such compliance. The data management plan will undergo objective review and approval by AHRQ's grant and technical peer review groups to assess its merits. The data management plan will become part of the terms and conditions of award for all appropriate grants and contracts. AHRQ will require all data management plans to be consistent with applicable laws and regulations governing the privacy and confidentiality of individual human data. AHRQ's Public Access Policy recognizes proprietary interests, business confidential information, and intellectual property rights and avoids significant negative impact on intellectual property rights, innovation, and U.S. competitiveness research grants and cooperative agreements and a requirement of appropriate contracts.
Data management plans will include:

A plan for protecting confidentiality and personal privacy.
A description of how scientific data in digital format will be shared, including a plan for long-term preservation and access to the data and the associated costs, or explanation of why data sharing is not possible.
AHRQ will promote the deposit of data in publicly accessible databases, where appropriate and available .

AHRQ encourages grantees and contractors to register all clinical trials with ClinicalTrials.gov (whether or not they are subject to Title VIII of Food and Drug Administration Amendments Act (FDAAA)) and is in the process of developing a policy for intramural researchers to extend the registration requirement to all clinical trials. AHRQ will give further consideration to taking steps to expand submission of results to encompass all AHRQ-funded clinical trials, not only those subject to FDAAA.
Preservation
In order to ensure long-term preservation and full access to the public, AHRQ will contract with a commercial repository to accept and manage data submitted by extramural, intramural, and contract researchers. The commercial repository and AHRQ staff will coordinate with the researcher upon notification of funding, prior to the start of the research project, and throughout its life cycle to ensure the current usability, long-run preservation and access to the data. Data will be made available in a digital format free of charge to the public via the selected commercial repository. If the data are made available via another mechanism, AHRQ will publicize the location of the data to the public on its Web site and provide a link to the data.
Long-term preservation and sustainability will be included in data management plans. Development of criteria for and conduct of periodic reviews to identify gaps in preservation coverage and respond to changing needs arising from new data types may become responsibilities of the Director of OEREP. In order to determine how best to develop and sustain repositories for digital scientific data, AHRQ intends to collaborate with HHS operating divisions and other agencies that support research in related areas.
Top of Page

9. Metrics, Compliance, and Evaluation

Oversight methods

AHRQ will include the public access requirement in all grant and contract solicitations. Standard requirements will be included in all research contracts and grant awards. All research proposals will include a Data Management Plan (DMP) as part of the overall research proposal. Approved DMPs will be included in terms and conditions of grant notices of award and contract awards. Accordingly, failure to comply with the terms and conditions of the funding agreement could lead to enforcement actions, including the withholding of funding, consistent with 45 CFR 74.62. Researchers will be expected to follow the approved DMP, to the best of their ability. Failure to do so will negatively influence future funding opportunities. AHRQ will monitor all non-competing continuation grant awards using the electronic Research Performance Progress Reports (RPPR) to verify compliance with the public access policy. Current oversight of grants allows for withholding or adjustments of funds at the end of each performance period. Grant final reports will be used to confirm compliance with the Public Access Policy. Contract progress reports will also be monitored for compliance.
Prior to funding new grant and contract awards AHRQ will confirm awardees are in compliance with the AHRQ Public Access Policy. AHRQ will not make grant or contract awards in cases of non-compliance. AHRQ may delay processing of non-competing continuation awards that are not in compliance with the Public Access Policy.
Intramural research proposals will also be required to include a DMP. AHRQ leadership will ensure intramural research compliance with the Public Access Policy.
AHRQ staff will periodically manually check applications, proposals, or reports for compliance with the Public Access Policy. They will confirm that a citation includes the appropriate reference number that indicates compliance, such as the PubMed Central Identifier (PMCID).
AHRQ will also use the NLM's Bibliography tool My NCBI. My NCBI tracks the public access compliance status of every paper associated with an award, generates a citation with the current identifier and clearly indicates its compliance status. It also allows awardees to collaborate with their colleagues to associate publications with AHRQ awards.
AHRQ will post a Notice in the NIH Guide stating the starting dates for PubMed and data sharing implementation. To simplify reporting and tracking, awardees will use My NCBI to report public access compliance on these awards. This process will be fully electronic for our Research Performance Progress Reports (RPPR).

Measures

Papers will be compliant when they are posted to PMC. This will form the numerator of an Agency-wide compliance measure. AHRQ's denominator for overall compliance will be determined by the number of papers AHRQ estimates fall under the policy. AHRQ will generate this number from papers linked to specific AHRQ awards by:

Authors in the AHRQ Manuscript Submission system.
Authors and principal investigators in the My NCBI bibliography management system.
Principal investigators and institutions in electronic progress reports.
Authors and publishers in the acknowledgement section of the text of papers, as indexed in Medline.

AHRQ's expectations for data sharing will be communicated to intramural and extramural researchers through guidance documents and outreach activities, including through the notice of grant award or the contract award. AHRQ staff will be responsible for monitoring and taking steps to ensure compliance with the terms and conditions of the award. Data management plans that are approved and made a term and condition in the notice of grant award or the contract award can be enforced. Accordingly, failure to comply with the terms and conditions of the funding agreement could lead to enforcement actions, including the withholding of funding, consistent with 45 CFR 74.62.
Program staff overseeing extramural grants and contracts will also review progress reports carefully for adherence with data management plans. Under the intramural program, the AHRQ's Center Directors will be responsible for ensuring intramural researchers are in compliance with data management plans. AHRQ will conduct an annual audit of its intramural researchers to monitor overall compliance with data management.
Top of Page

10. Timeline

Beginning in December 2014, AHRQ will require all intramural projects and new grant and contract research awards to submit manuscripts to PMC. This policy will be effective for research funded in February 2015. AHRQ will solicit contract proposals and fund a commercial repository by October 2015. Beginning October 2015, AHRQ will require all researchers to submit data in digital format to the commercial repository.
Top of Page

11. Resources

AHRQ will use annual appropriation funds for the implementation of PMC and for the commercial repository.
Top of Page

12. Public Consultation

AHRQ participated in the public consultation sessions convened May 14-17, 2013, by the National Science Foundation. This Public Access Plan is responsive to feedback provided during those sessions.
Top of Page

13. Interagency Coordination

AHRQ interagency coordination will be done by the following:

Full participation on the HHS interagency public access work group.
Full participation on the government-wide public access publications and data work groups facilitated by the Office of Science and Technology Policy.
Coordinating with NIH in the use of PMC and adherence to all established PMC requirements.

Top of Page

14. Public Notice

AHRQ will post the final Public Access Policy on the AHRQ Web site and in the NIH Guide for Grants and Contracts.
Top of Page

15. Update and Re-Evaluation of the Policy

AHRQ will conduct periodic updates on the final published Public Access Policy, which will be informed by feedback from all stakeholders. Feedback will be sought periodically through Requests for Information published in the NIH Guide for Grants and Contracts.
Top of Page

16. Additional Information

Additional public access policies and guidance include the following:

Memorandum for the Heads of Executive Departments and Agencies—Open Government Directive, Executive Office of the President, Office of Management and Budget, December 8, 2009:http://www.whitehouse.gov/omb/assets/memoranda_2010/m10-06.pdf (80.52 KB).
National Digital Information Infrastructure & Preservation Program: A Collaborative Initiative of the Library of Congress: http://www.digitalpreservation.gov/.
NIH Public Access: http://publicaccess.nih.gov/.
Open Government Directive, Memorandum for the Heads of Executive Departments and Agencies, M-10-06 (Dec 8, 2009): http://www.whitehouse.gov/sites/default/files/omb/assets/memoranda_2010/m10-06.pdf(80.52 KB).
Planets—Preservation and Long-Term Access through Networked Services. Open Planets Foundation:http://www.planets-project.eu/ .
Public Access to Scholarly Publications Policy Objectives—Interagency Working Group (March, 2012):http://www.whitehouse.gov/sites/default/files/microsites/ostp/public_access-final.pdf (411.75 KB).
Scholarly Publishing Roundtable, Report and Recommendations from the Scholarly Publishing Roundtable (2009): http://www.aau.edu/policy/scholarly_publishing_roundtable.aspx?id=6894 .

Top of Page

17. Material Superseded

Initial release.
AHRQ Staff Contact:
Francis D. Chesley, Jr., M.D.
Director, Office of Extramural Research, Education, and Priority Populations
Email: Francis.Chesley@ahrq.hhs.gov
Top of Page

Page last reviewed February 2015

Internet Citation: AHRQ Public Access to Federally Funded Research: Publications and Data . February 2015. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/funding/policies/publicaccess/index.html