Concurrent Validity of a Self-Reported Physical Activity “Vital Sign” Questionnaire With Adult Primary Care Patients
ORIGINAL RESEARCH — Volume 13 — February 4, 2016
Trever J. Ball, PhD, MS, MPH; Elizabeth A. Joy, MD, MPH; Lisa H. Gren, PhD, MSPH; Janet M. Shaw, PhD
Suggested citation for this article: Ball TJ, Joy EA, Gren LH, Shaw JM. Concurrent Validity of a Self-Reported Physical Activity “Vital Sign” Questionnaire With Adult Primary Care Patients. [Erratum appears in Prev Chronic Dis 2016;13. http://www.cdc.gov/pcd/issues/2016/15_0228e.htm.] Prev Chronic Dis 2016;13:150228. DOI: http://dx.doi.org/10.5888/pcd13.150228.
No tool currently used by primary health care providers to assess physical activity has been evaluated for its ability to determine whether or not patients achieve recommended levels of activity. The purpose of this study was to assess concurrent validity of physical activity self-reported to the brief (<30 sec) Physical Activity “Vital Sign” questionnaire (PAVS) compared with responses to the lengthier (3–5 min), validated Modifiable Activity Questionnaire (MAQ).
Agreement between activity reported to the PAVS and MAQ by primary care patients at 2 clinics in 2014 was assessed by using percentages and κ coefficients. Agreement consisted of meeting or not meeting the 2008 Aerobic Physical Activity Guidelines for Americans (PA Guidelines) of the US Department of Health and Human Services. We compared self-reported usual minutes per week of moderate-to-vigorous physical activity among patients at a primary care clinic in 2014 who reported to PAVS and to MAQ by using Pearson correlation and Bland–Altman plots of agreement.
Among 269 consenting patients who reported physical activity, PAVS results agreed with those of MAQ 89.6% of the time and demonstrated good agreement in identifying patients who did not meet PA Guidelines recommendations (κ = 0.55, ρ = 0.57; P < .001). Usual minutes per week of moderate-to-vigorous physical activity reported to PAVS had a high positive correlation with the same reported to MAQ (r = 0.71; P < .001).
PAVS may be a valid tool for identifying primary care patients who need counseling about physical activity. PAVS should be assessed further for agreement with repeated objective measures of physical activity in the patient population.
Physical inactivity may be the greatest public health problem of the 21st century (1). Assessing and counseling for physical activity in primary health care has led to increased physical activity and improved intermediate health outcomes, such as reductions in lipid levels, blood pressure, fasting blood glucose levels, weight, and incidence of diabetes (2–4). Assessing and counseling about physical activity in primary health care requires that providers and health care systems have a valid tool for assessing patients’ physical activity within clinic workflow (5).
A minimum requirement of a physical activity assessment tool should be to assess whether a patient meets levels of physical activity recommended by the US Department of Health and Human Services’ 2008 Aerobic Physical Activity Guidelines for Americans (PA Guidelines) (6–13). PA Guidelines recommends that to promote and maintain health, adults should engage in at least 150 minutes per week of moderate physical activity or 75 minutes per week of vigorous activity, or an equivalent combination of the two (14).
To our knowledge, no tool used in primary health care to assess patient physical activity has been evaluated for concurrent validity in identifying whether or not patients meet recommendations of PA Guidelines (6–9,11). Although not assessed for validity against these recommendations, other brief tools for assessing physical activity among patients show moderate discriminant validity with surveillance of public health physical activity. These tools also show moderate concurrent validity with earlier PA Guidelines recommendations and poor to fair criterion validity with activity measured objectively (6,8,9).
The primary objective of this study was to assess validity of the patient-reported Physical Activity “Vital Sign” questionnaire (PAVS) compared with responses to a concurrently administered Modifiable Activity Questionnaire (MAQ), which has been validated among adults (15,16). A secondary objective was to determine any differences in concurrent validity in patients’ levels of confidence in reporting activity to the PAVS.
Participants were volunteer, adult (aged ≥18 y) primary care patients at 2 clinics of Intermountain Healthcare in northern Utah. Patients were ineligible to participate if they did not speak English or if they had diagnosed dementia, because PAVS was administered only in English at the time of this study and because PAVS required cognitively recalling physical activity behavior. Intermountain Healthcare is a nonprofit integrated health care system of 22 hospitals and approximately 160 facilities throughout Utah and southern Idaho. This study was approved by the Intermountain Healthcare Institutional Review Board. We estimated that a sample of 268 participants completing both PAVS and MAQ would provide 95% confidence intervals around a Cohen’s κ for agreement of 46% (7,17).
Modifiable Activity Questionnaire (MAQ)
MAQ is a modifiable questionnaire, because it queries only activities identified as commonly performed by a study population. We followed steps recently developed by Sternfeld and Goldman-Rosas, a systematic approach, to choose MAQ for evaluating concurrent validity with the PAVS (18). The most notable reason for selecting MAQ to assess concurrent validity with PAVS was that MAQ had the strongest established validity among comparable instruments (19). Although previous studies found that MAQ associated strongly with objectively measured physical activity, MAQ is too lengthy and therefore impractical for use in a primary care clinic setting (7–9).
The MAQ used in this study included moderate-to-vigorous physical activities most commonly performed according to responses to the 2012 Utah Behavioral Risk Factor Surveillance System (20). Intensity of activities was identified as moderate to vigorous according to the Compendium of Physical Activities (21).
Physical Activity “Vital Sign” (PAVS)
PAVS is a brief (<30 seconds) physical activity questionnaire intended to be administered to every patient at every clinic visit just as vital signs (eg, blood pressure, temperature) and height and weight are measured. PAVS is intended to assess how much light, moderate, or vigorous physical activity a person performs in a typical week. This assessment facilitates determining whether or not a patient meets PA Guidelines recommendations and subsequent patient counseling about physical activity (14).
PAVS asks the following 2 questions, which a patient answers either on a small form when checking in for an appointment or in person to the medical assistant who measures vital signs: 1) “Please describe your level of physical activity, [first by] minutes per day, [followed by] number of days each week,” and 2) “At what intensity (how hard): light (like a casual walk), moderate (like a brisk walk), or vigorous (like a jog/run)?”
Responses are entered in the electronic health record by either medical assistants or the physician. Total minutes per week of activity the patient reports as light, moderate, or vigorous are automatically calculated by the electronic health record by multiplying average minutes per day of physical activity by average number of days per week of activity. Moderate-to-vigorous activity is automatically calculated by summing minutes per week of moderate and vigorous activity.
Patients were recruited by medical assistants after taking patients’ vital signs in private examination rooms. To help control bias of voluntary participation, a standardized invitation to participate was read to patients by medical assistants from an index card. Medical assistants did not mention physical activity while recruiting patients to facilitate attaining a representative sample of participants from the adult clinic population. Patients consented verbally to medical assistants to learn more about the study. Details of the study and MAQ were provided to patients by student research assistants in a private room.
Research assistants first asked patients how confident they felt in reporting their physical activity to PAVS. Confidence was reported on a Likert scale of 1 to 5 (1 = very unsure, 2 = quite unsure, 3 = about 50% sure, 4 = quite sure, and 5 = very sure). After patients reported confidence, research assistants administered MAQ. MAQ was completed an average of 30 minutes after patients reported their activity to PAVS. It was not feasible to randomize the order of administering the PAVS and MAQ assessments because of a lack of privacy in clinic waiting areas and our desire not to interfere with typical patient flow through the clinic.
To help assess how well the volunteer participants in this study represented the total eligible population, we compared demographic characteristics of both groups by using 2-sample tests of proportions.
Correlation and agreement were tested between weekly minutes of physical activity reported to PAVS and MAQ. Correlation and agreement between the 2 instruments were tested only for patients who indicated that activity reported to MAQ was usual for them. This was because PAVS assesses usual activity.
Concurrent validity of PAVS’s ability to identify whether or not patients met PA Guidelines recommendations was assessed by a κ coefficient of binary agreement between the 2 assessment tools’ proportions of patients meeting and not meeting recommended levels. Validity of weekly minutes of activity reported to PAVS was assessed by using Pearson correlation between weekly minutes reported to PAVS and MAQ. Agreement between weekly minutes of activity reported to PAVS and to MAQ was assessed by using Bland–Altman agreement plots (22). Data from outliers (n = 8) were excluded from agreement plots to facilitate visually interpreting agreement plots. These plots had 95% limits of agreement and were unadjusted and adjusted for trend. Correlation and agreement analyses were stratified by patient characteristics.
Participant-reported confidence levels for reporting activity to PAVS were dichotomized into low and high confidence groups according to being either at or below the 50th percentile or at or above the 51st percentile of scores. Differences in PAVS validity by confidence levels was assessed by comparing Pearson correlation coefficients between physical activity reported to PAVS and to MAQ. Statistical differences between correlation coefficients of the low and high confidence groups were tested using a z statistic and associated P value (23). All analyses were performed with Stata version 11.2 (StataCorp LP). The α level used was .05.
Participant characteristics and physical activity behaviors
A total of 305 patients consented to participate in this study (Figure 1). Demographic characteristics of participants were recorded for 298 participants ( Table 1). We found no significant differences in proportions of sex and age groups between volunteer and eligible participants. Most participants were women (61.4%) and white (88.9%); 269 of the 305 participants indicated that physical activity reported to MAQ was usual for them. Patients who reported the greatest weekly minutes of physical activity to both PAVS and MAQ were younger (<55 y) and more educated than patients who reported less physical activity ( Table 2).
Validity and agreement
PAVS agreed strongly with MAQ 89.6% of the time in identifying patients who were insufficiently active 89.6% of the time ( Table 3). PAVS demonstrated moderate agreement for correctly identifying patients as meeting or not meeting PA Guidelines recommendations when accounting for agreement occurring by chance (κ = 0.55, P < .001). PAVS correlated strongly with MAQ for assessing patient weekly minutes of activity (r = 0.71, P < .001).
Patients who most frequently self-identified on PAVS as being insufficiently active were patients who reported low confidence reporting physical activity to PAVS (70.9%), patients who were not high school graduates (77.8%), and female patients (63.2%) (Table 2). Agreement for identifying patients as meeting or not meeting PA Guidelines recommendations was strongest for patients with high confidence in reporting activity to PAVS (79.9% of the time; κ = 0.60, P < .001) and for older (>64 y) patients (80.0% of the time; κ = 0.60, P < .001). Patients who had high confidence in reporting activity to PAVS also had more valid measures of activity (ie, stronger correlation of activity minutes between PAVS and MAQ; r = 0.74, P < .001). Others that had more valid measures of activity were men (r = 0.81, P < .001) and younger (18–64 y) patients (r = 0.75, P < .001) (Table 3).
Bland-Altman agreement between PAVS and MAQ was fair (Figure 2). For total participants, 95% confidence limits were wide (−371.3 to 198.7 minutes/week). Participants reported an average of 86.3 fewer weekly minutes of activity to PAVS (128.5) compared with what they reported to MAQ (214.8; P < .001) (Figure 2).
Participant confidence reporting activity and common activities
Most patients felt “very sure” that their physical activity reported to PAVS was accurate (68%). Weekly minutes of activity assessed by PAVS were more strongly correlated with the same assessed by MAQ among the participants with high confidence self-reporting their activity (r = 0.74) compared with low confidence (r = 0.63, P = .01). The most common activities reported by all participants were walking (66.6%), lifting weights (24.6%), and calisthenics (20.0%) (eg, sit-ups, pushups).
Our study showed that PAVS may be able to identify patients who are insufficiently active according to PA Guidelines recommendations. PAVS accurately identified patients who need counseling for being physically inactive. This was demonstrated by PAVS and MAQ identifying insufficiently active patients 89.6% of the time. An advantage of PAVS over other tools, such as MAQ, is that PAVS takes less than 30 seconds to administer. To our knowledge, this is the first study to evaluate the ability of a patient physical activity assessment tool to compare patients’ physical activity with PA Guidelines recommendations.
Primary care providers have an opportunity to improve patient clinical outcomes through counseling for physical inactivity largely because physical inactivity may be the greatest health risk behavior when risk is assessed independent of other health risk behaviors (1). Counseling patients in a health care setting about physical inactivity is facilitated by determining whether patients meet PA Guidelines recommendations, because these guidelines recommendations were designed to prevent disease and promote health (13).
The first step in treating patients for physical inactivity is assessing their activity with a validated instrument. The tool should be feasible for administering in a clinical setting and must be easily understood by patients. Intermountain Healthcare identified the following points as necessary for successfully integrating an assessment of patient physical activity into clinical workflow:
- Educating leaders on leading health risks of physical inactivity.
- Having clinic champions of promoting physical activity (eg, clinic medical directors).
- Making the activity assessment brief while capturing data on activity levels that can be assessed against PA Guidelines recommendations (ie, making the assessment diagnostic as meeting or not meeting recommendations).
- Integrating the assessment into the electronic health record near other vital signs.
- Having patients report their activity on paper when checking in for an office visit. The medical assistant later records the assessment in the electronic health record at the same time other vital signs are recorded.
Other information about integrating a physical activity assessment into regular clinic flow can be found through the Exercise is Medicine initiative of the American College of Sports Medicine (www.exerciseismedicine.org). Once an assessment of physical activity becomes a regular part of clinic workflow, strategies for “treating,” or counseling, for physical inactivity need to be developed. Such strategies may be the provider giving brief counseling or developing a referral network to community physical activity resources provided by Exercise is Medicine.
Valid clinical assessments of physical activity could enable health care providers to treat only patients who do not meet PA Guidelines recommendations. Valid clinical assessment could also minimize measurement error for epidemiologic investigations of physical activity and health outcomes (6,24).
Bland-Altman plots of agreement between physical activity reported to PAVS and to MAQ suggest that dose–response estimates between activity reported to PAVS and clinical health outcomes would be attenuated. This attenuation would occur because PAVS underestimated usual minutes per week of moderate-to-vigorous physical activity by an average of 86.3 minutes per week compared with MAQ. However, although MAQ is strongly correlated with an objective measure of physical activity by accelerometry, MAQ may overestimate activity because it provides more opportunity to report activity than PAVS. That is, MAQ lists more domains, or types, of activity (25,26). The brief PAVS agreed concurrently with the MAQ better than a lengthier physical activity questionnaire agreed concurrently with another instrument (27). In other words, although PAVS is a much shorter physical activity questionnaire than most, it appears to assess moderate-to-vigorous physical activity just as well as lengthier questionnaires.
Physical activity reported by patients in this study agreed with and was correlated most strongly with patients who indicated feeling most confident (ie, upper 50th percentile of confidence rating) reporting their activity. Others have also observed this relationship (28). The accuracy of epidemiologic studies that use physical activity reported by patients and identifiable health outcomes in electronic health records may be improved by understanding factors such as confidence in self-reporting physical activity. Patient confidence in reporting physical activity might improve as patients report more frequently to PAVS, for instance, because PAVS is administered in clinics more regularly. This could be investigated with other studies and may help further assess measurement properties of PAVS.
A strength of this study was our assessing measurement properties of PAVS only in clinics that used this instrument as part of routine patient workflow. PAVS was recommended to assess physical activity in adult patients by administering it to every patient at every visit for approximately 2 years before this study. MAQ, the questionnaire we chose to assess concurrent validity of PAVS, demonstrated the best objective measures of validity among known physical activity questionnaires that measure the same constructs as PAVS (19).
Findings of this study are limited by MAQ’s reliance on self-report. Earlier studies showed that MAQ overestimated physical activity, and it appears to overestimate more than PAVS does (7). A better instrument for assessing concurrent validity of any physical activity self-report assessment might be an instrument with criterion validity assessed by agreement rather than by correlation. As with validation studies of MAQ with criterion measures of physical activity, this study found a strong positive correlation between PAVS and MAQ, but not, for example, by Bland–Altman agreement. The findings of concurrent validity of PAVS in this study are representative only of the patient population that participated.
Finally, in this study, PAVS assessed patient physical activity at only one of 3 levels of intensity: light, moderate, or vigorous. PAVS did not assess activity done at each of these intensities. For this assessment of patient physical activity to be used with, for example, patient electronic health records to examine physical activity–disease relationships, PAVS should assess patient physical activity at all 3 intensities.
Assessing and counseling patients for physical inactivity could significantly improve intermediate health outcomes. This study found strong evidence for the ability of PAVS to correctly identify patients who are insufficiently active according to PA Guidelines recommendations. On average, PAVS underestimated patients’ usual minutes per week of moderate-to-vigorous physical activity compared with MAQ’s assessment. The PAVS should be assessed further for its ability to reliably track trends in patient physical activity. Future study should also assess the validity of using electronic records of patient physical activity for epidemiologic investigations. These assessments would be done most reliably and validly with measures of physical activity that are repeated, objective, and assessed in the patient population.
This study was funded in part by the University of Utah Undergraduate Research Opportunities Program and the Health Studies Fund in The University of Utah’s Department of Family and Preventive Medicine. The authors thank the Avenues and Memorial Intermountain Healthcare clinic staff, physicians, and patients for their assistance and participation in this study. Thanks are also due to undergraduate student research assistants that helped administer questionnaires to patients.
Corresponding Author: Trever J. Ball, PhD, MS, MPH, Intermountain Healthcare, 36 South State St, Floor 8, Salt Lake City, UT 84111. Telephone: 801-442-3580. Email: firstname.lastname@example.org.
Author Affiliations: Elizabeth A. Joy, Intermountain Healthcare, Salt Lake City, Utah; Lisa H. Gren, Department of Family and Preventive Medicine, University of Utah, Salt Lake City, Utah; Janet M. Shaw, Department of Exercise and Sport Science, University of Utah, Salt Lake City, Utah.
- Blair SN. Physical inactivity: the biggest public health problem of the 21st century. Br J Sports Med 2009;43(1):1–2. PubMed
- Lin JS, O’Connor E, Whitlock EP, Beil TL. Behavioral counseling to promote physical activity and a healthful diet to prevent cardiovascular disease in adults: a systematic review for the US Preventive Services Task Force. Ann Intern Med 2010;153(11):736–50. CrossRef PubMed
- Haskell WL, Lee IM, Pate RR, Powell KE, Blair SN, Franklin BA, et al. ; American Heart Association. Physical activity and public health: updated recommendation for adults from the American College of Sports Medicine and the American Heart Association. Circulation 2007;116(9):1081–93. CrossRef PubMed
- Nawaz H, Katz DL. American College of Preventive Medicine Practice Policy statement. Weight management counseling of overweight adults. Am J Prev Med 2001;21(1):73–8. CrossRefPubMed
- Exercise is medicine; 2014. http://exerciseismedicine.org/. Accessed May 15, 2014.
- Coleman KJ, Ngor E, Reynolds K, Quinn VP, Koebnick C, Young DR, et al. Initial validation of an exercise “vital sign” in electronic medical records. Med Sci Sports Exerc 2012;44(11):2071–6.CrossRef PubMed
- Ball T, Joy E, Goh T, Hannon J, Gren L, Shaw J. Validity of two brief primary care physical activity questionnaires with accelerometry in clinic staff. Prim Health Care Res Dev 2015;16(1):100–8.PubMed
- Smith BJ, Marshall AL, Huang N. Screening for physical activity in family practice: evaluation of two brief assessment tools. Am J Prev Med 2005;29(4):256–64. CrossRef PubMed
- Meriwether RA, McMahon PM, Islam N, Steinmann WC. Physical activity assessment: validation of a clinical assessment tool. Am J Prev Med 2006;31(6):484–91. CrossRef PubMed
- Vuori IM, Lavie CJ, Blair SN. Physical activity promotion in the health care system. Mayo Clin Proc 2013;88(12):1446–61. CrossRef PubMed
- Heron N, Tully MA, McKinley MC, Cupples ME. Physical activity assessment in practice: a mixed methods study of GPPAQ use in primary care. BMC Fam Pract 2014;15(1):11. CrossRefPubMed
- Patrick K, Pratt M, Sallis RE. The health care sector’s role in the US national physical activity plan. J Phys Act Health 2009;6(Suppl 2):S211–9. PubMed
- Strath SJ, Kaminsky LA, Ainsworth BE, Ekelund U, Freedson PS, Gary RA, et al. ; American Heart Association Physical Activity Committee of the Council on Lifestyle and Cardiometabolic Health and Cardiovascular, Exercise, Cardiac Rehabilitation and Prevention Committee of the Council on Clinical Cardiology, and Council. Guide to the assessment of physical activity: clinical and research applications: a scientific statement from the American Heart Association. Circulation 2013;128(20):2259–79. CrossRef PubMed
- US Department of Health and Human Services. 2008 Physical activity guidelines for Americans. Atlanta (GA): Office of Disease Prevention and Health Promotion, Centers for Disease Control and Prevention; 2008.
- Pettee Gabriel K, McClain JJ, Lee CD, Swan PD, Alvar BA, Mitros MR, et al. Evaluation of physical activity measures used in middle-aged women. Med Sci Sports Exerc 2009;41(7):1403–12.CrossRef PubMed
- Pettee Gabriel K, McClain JJ, Schmid KK, Storti KL, Ainsworth BE. Reliability and convergent validity of the past-week Modifiable Activity Questionnaire. Public Health Nutr 2011;14(3):435–42. CrossRef PubMed
- Cantor AB. Sample-size calculations for Cohen’s kappa. Psychol Methods 1996;1(2):150–3. CrossRef
- Sternfeld B, Goldman-Rosas L. A systematic approach to selecting an appropriate measure of self-reported physical activity or sedentary behavior. J Phys Act Health 2012;9(Suppl 1):S19–28.PubMed
- Helmerhorst HJ, Brage S, Warren J, Besson H, Ekelund U. A systematic review of reliability and objective criterion-related validity of physical activity questionnaires. Int J Behav Nutr Phys Act 2012;9:103. CrossRef PubMed
- Utah Department of Health. Utah Behavioral Risk Factor Surveillance System. 2012. https://ibis.health.utah.gov/topic/PhysicalActivity.html. Accessed June 3, 2013.
- Ainsworth BE, Haskell WL, Herrmann SD, Meckes N, Bassett DR Jr, Tudor-Locke C, et al. 2011 Compendium of physical activities: a second update of codes and MET values. Med Sci Sports Exerc 2011;43(8):1575–81. CrossRef PubMed
- Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999;8(2):135–60. CrossRef PubMed
- Fisher RA. On the “probable error” of a coefficient of correlation deduced from a small sample. Metron 1921;1(1):3–32.
- Matthews CE, Moore SC, George SM, Sampson J, Bowles HR. Improving self-reports of active and sedentary behaviors in large epidemiologic studies. Exerc Sport Sci Rev 2012;40(3):118–26.PubMed
- Altschuler A, Picchi T, Nelson M, Rogers JD, Hart J, Sternfeld B. Physical activity questionnaire comprehension: lessons from cognitive interviews. Med Sci Sports Exerc 2009;41(2):336–43.CrossRef PubMed
- Prince SA, Adamo KB, Hamel ME, Hardt J, Connor Gorber S, Tremblay M. A comparison of direct versus self-report measures for assessing physical activity in adults: a systematic review. Int J Behav Nutr Phys Act 2008;5(1):56–80. CrossRef PubMed
- Hagströmer M, Oja P, Sjöström M. The International Physical Activity Questionnaire (IPAQ): a study of concurrent and construct validity. Public Health Nutr 2006;9(6):755–62. CrossRefPubMed
- Cust AE, Armstrong BK, Smith BJ, Chau J, van der Ploeg HP, Bauman A. Self-reported confidence in recall as a predictor of validity and repeatability of physical activity questionnaire data. Epidemiology 2009;20(3):433–41. CrossRef PubMed