Category: Dissemination / Implementation
Keywords: Implementation | Measurement | Psychometrics
Presentation Type: Symposium
Psychometric validation is key to establishing a sound empirical infrastructure for measurement in any field (Cook & Beckman, 2006; Stichter & Conroy 2004); however, there are a paucity of studies within the field of dissemination and implementation providing this fundamental information. And, recent reviews indicate that the majority of measures are of poor quality. Establishing the pragmatic (i.e., practical) utility of measures could allow stakeholders to efficiently make implementation-relevant decisions (Glasgow & Riley, 2013). Having psychometrically strong and pragmatic measures could position stakeholders to form focused implementation interventions targeting mechanisms, the processes through which interventions exert their effect. Identifying mechanisms of change is critical in curating the strategies necessary for integrating evidence-based practices into real world settings. The current study sought to address these interrelated measurement issues by systematically identifying, categorizing, and empirically rating the psychometric and pragmatic strength of measures assessing constructs outlined by the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009) and the Implementation Outcomes Framework (IOF; Proctor et al., 2010). To identify relevant measures, we conducted a systematic review of PubMed and PsycINFO. All articles citing measure use were compiled and relevant psychometric and pragmatic data was extracted. Using our Evidence-Based Assessment (EBA) criteria coupled with newly developed Pragmatic Rating criteria, two independent raters examined the data to rate each measure’s psychometric and pragmatic qualities. Preliminary results revealed 104 IOF-relevant measures (Lewis et al., 2015) with limited and variable information. Only one measure (Goodman et al., 1993) provided information for all six EBA criteria, with a total score of 19.5 out of a possible 24. Across all outcomes, total scores ranged from two to 19.5, with a median total score of eight and a modal total score of seven. Additional results will report on EBA ratings for CFIR-relevant measures and ratings for pragmatic strength. The resulting repository and associated quality ratings will position researchers and stakeholders to assess putative mechanisms and outcomes.
Associate Scientific Investigator
Kaiser Permanente Washington Health Research Institute
Friday, November 17
12:45 PM – 1:45 PM
Friday, November 17
3:00 PM – 4:00 PM
Saturday, November 18
11:30 AM – 1:00 PM
The asset you are trying to access is locked. Please enter your access key to unlock.