A Standardized Reporting System for Assessment of Diverse Public Health Programs

Navigate This Article

Douglas Fernald, MA; Abigail Harris, MSPH; Elizabeth Ann Deaton, MA; Vicki Weister; Shannon Pray, MSPH; Carsten Baumann, MA; Arnold Levinson, PhD

Suggested citation for this article: Fernald D, Harris A, Deaton EA, Weister V, Pray S, Baumann C, et al. A Standardized Reporting System for Assessment of Diverse Public Health Programs. Prev Chronic Dis 2012;9:120004. DOI: http://dx.doi.org/10.5888/pcd9.120004

PEER REVIEWED

Abstract

State public health agencies face challenges when monitoring the efforts and effects of public health programs that use disparate strategies and address various diseases, locations, and populations. The external evaluators of a complex portfolio of grant funding sought a standardized reporting framework and tool that could be used for all grants in the portfolio, without having to redesign it for each disease or intervention approach. Evaluators iteratively reviewed grant-funded projects to identify common project delivery strategies, then developed and implemented a common reporting framework and spreadsheet-based data capture tool. Evaluators provided training, technical assistance, and ongoing data reviews. During 2 fiscal years, 103 public health programs throughout Colorado submitted quarterly reports; agencies funded to implement these programs ranged from small community-based organizations to university- and hospital-affiliated groups in urban and rural settings. Aggregated reports supported estimates of program reach by strategy and by disease area, and the system supported production of summary descriptions of program implementation. Standardized language and expectations for reporting helped to align grant applications and work plans with reporting tools. A common language and standardized reporting tool can be used for diverse projects in a comprehensive evaluation framework. Decentralized data collection using common spreadsheet software enabled the aggregation of common data elements across multiple programs and projects. Further refinements could enable wider dissemination of common reporting criteria and expectations.

Top of Page

Introduction

Large public health initiatives are hard to monitor and evaluate when the health objectives involve several diseases, dispersed geography, multiple populations, or comprehensive strategies being delivered by varied public and private entities with diverse evaluation capacity. Even modifying a single project for delivery to multiple sites complicates monitoring and evaluation and is often expensive (1). Previous researchers have suggested strategies for gathering standardized multisite information based on homogeneous programs or interventions focused on a single disease or delivery strategy (2-4). Theoretical frameworks and guides exist for evaluating large, single-objective public health programs and their individual projects (5-12). However, little is known about practical assessment strategies for large, dispersed, and heterogeneous public health program portfolios.
One such portfolio was created in 2004 when Colorado voters approved a tobacco-tax increase and earmarked a portion of revenue streams to address illnesses related to tobacco use through extending programs and translating evidence-based approaches. In the portfolio, 2 grant-making programs address highly diverse public health needs: 1) reduction of health disparities in illnesses related to tobacco use and 2) prevention, early detection, and treatment of cancer and cardiovascular and pulmonary diseases (CCPD). These 2 programs generated 103 unique projects, which were conducted during fiscal years (FYs) 2006 through 2009 with approximately $100 million of tobacco-tax revenues. Agencies funded to implement these programs ranged from small community-based organizations to university- and hospital-affiliated groups in urban and rural settings.
The number and diversity of these projects introduced monitoring and evaluation challenges. At a minimum, the legislation that created the grant-making programs requires annual reporting of each project’s implementation (number of people served, services provided) and overall results of aggregate program activities. Administrative oversight requires capacity to track grantee activities and compare them to initial work plans for accountability. Professional public health practice requires state-level information to guide periodic planning, program refinement, and project continuation decisions. Grantees need formative feedback for quality improvement and summative feedback to reinforce motivation and maintain enthusiasm. Because of the array of data required, an efficient and standardized evaluation system was needed.