Category: Dissemination / Implementation


Psychometric and Pragmatic Evaluation of Implementation Science Measures

Friday, November 17
12:45 PM - 1:45 PM
Location: Indigo Ballroom B, Level 2, Indigo Level

Keywords: Implementation | Measurement | Psychometrics
Presentation Type: Symposium

Psychometric validation is key to establishing a sound empirical infrastructure for measurement in any field (Cook & Beckman, 2006; Stichter & Conroy 2004); however, there are a paucity of studies within the field of dissemination and implementation providing this fundamental information. And, recent reviews indicate that the majority of measures are of poor quality. Establishing the pragmatic (i.e., practical) utility of measures could allow stakeholders to efficiently make implementation-relevant decisions (Glasgow & Riley, 2013). Having psychometrically strong and pragmatic measures could position stakeholders to form focused implementation interventions targeting mechanisms, the processes through which interventions exert their effect. Identifying mechanisms of change is critical in curating the strategies necessary for integrating evidence-based practices into real world settings. The current study sought to address these interrelated measurement issues by systematically identifying, categorizing, and empirically rating the psychometric and pragmatic strength of measures assessing constructs outlined by the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009) and the Implementation Outcomes Framework (IOF; Proctor et al., 2010). To identify relevant measures, we conducted a systematic review of PubMed and PsycINFO. All articles citing measure use were compiled and relevant psychometric and pragmatic data was extracted. Using our Evidence-Based Assessment (EBA) criteria coupled with newly developed Pragmatic Rating criteria, two independent raters examined the data to rate each measure’s psychometric and pragmatic qualities. Preliminary results revealed 104 IOF-relevant measures (Lewis et al., 2015) with limited and variable information. Only one measure (Goodman et al., 1993) provided information for all six EBA criteria, with a total score of 19.5 out of a possible 24. Across all outcomes, total scores ranged from two to 19.5, with a median total score of eight and a modal total score of seven. Additional results will report on EBA ratings for CFIR-relevant measures and ratings for pragmatic strength. The resulting repository and associated quality ratings will position researchers and stakeholders to assess putative mechanisms and outcomes. 

Cara C. Lewis

Associate Scientific Investigator
Kaiser Permanente Washington Health Research Institute


Send Email for Cara Lewis


Psychometric and Pragmatic Evaluation of Implementation Science Measures

Attendees who have favorited this

Please enter your access key

The asset you are trying to access is locked. Please enter your access key to unlock.

Send Email for Psychometric and Pragmatic Evaluation of Implementation Science Measures