Test-retest reliability of the Newest Vital Sign health literacy instrument: In-person and remote administration.

Russell AM1, Patel DA2, Curtis LM3, Kim KA4, Wolf MS5, Rowland ME6, McCarthy DM7.

Author information

1: Division of General Internal Medicine, Feinberg School of Medicine at Northwestern University, Chicago, IL, USA. Electronic address: andrea.russell@northwestern.edu.
2: Division of General Internal Medicine, Feinberg School of Medicine at Northwestern University, Chicago, IL, USA. Electronic address: deesha.patel@northwestern.edu.
3: Division of General Internal Medicine, Feinberg School of Medicine at Northwestern University, Chicago, IL, USA. Electronic address: l-curtis@northwestern.edu.
4: Department of Preventive Medicine, Feinberg School of Medicine at Northwestern University, Chicago, IL, USA. Electronic address: kykim@northwestern.edu.
5: Division of General Internal Medicine, Feinberg School of Medicine at Northwestern University, Chicago, IL, USA. Electronic address: mswolf@northwestern.edu.
6: Department of Emergency Medicine, Feinberg School of Medicine at Northwestern University, Chicago, IL, USA. Electronic address: megan.rowland@northwestern.edu.
7: Department of Emergency Medicine, Feinberg School of Medicine at Northwestern University, Chicago, IL, USA. Electronic address: d-mccarthy2@northwestern.edu.

Abstract

OBJECTIVE:

To determine the reliability of the Newest Vital Sign (NVS) administered via telephone by examining test-retest properties of the measure.

METHODS:

Data were obtained from a randomized controlled trial promoting opioid safe use. Participants were 18 or older and English-speaking. NVS assessment occurred in-person at baseline and in-person or via telephone at follow-up. Intraclass correlation coefficients (ICCs) were used to assess the test-retest reliability using raw NVS scores by mode of administration of the second NVS assessment. Kappa statistics were used to examine test-retest agreement based on categorized NVS score. Internal consistency was measured with Cronbach's alpha.

RESULTS:

Data from 216 patients (70 completing follow-up in-person and 146 via telephone) were included. Reliability was high (ICCs: in-person = 0.81, phone = 0.70). Agreement was lower for three category NVS score (Kappas: in-person = 0.58, 95% CI [0.39-0.77]; phone = 0.52, 95% CI [0.39-0.65]) compared to two category NVS (Kappas: in-person = 0.65, 95% CI [0.46-0.85]; phone = 0.64, 95% CI [0.51-0.78]). Correlations decreased as time between administrations increased. Internal consistency was moderately high (baseline NVS in-person (α = 0.76), follow-up NVS in-person (α = 0.76), and phone follow-up (α = 0.78).