Category: Assessment

PS6- #C76 - Measuring CBT Competency Among School Professionals: Validation of the Clinical Practices Assessment Measure Toward TRAILS in Schools

Friday, Nov 17
2:45 PM – 3:45 PM
Location: Indigo Ballroom CDGH, Level 2, Indigo Level

Keywords: Psychometrics | Assessment | CBT

School-based access to CBT has potential to help the 20-30% of school-aged children that struggle with mood and anxiety disorders and concomitant academic and interpersonal difficulties. CBT is effective in treating these disorders, but access is limited. The TRAILS program has piloted an implementation strategy that incorporates didactic CBT training for school-based providers (SPs) followed by expert-led coaching in CBT delivery to students. Successful implementation and evaluation of TRAILS requires development and utilization of a measure assessing basic SP competency in CBT concepts and skills. TRAILS pilot work developed a measure for this purpose, the Clinical Practices Assessment Measure (CPAM). The CPAM consists of 40 self-report items assessing Clinical Expertise (CE), Skills Use Frequency (UF), Perceptions of CBT (P), and 10 criterion-referenced items that solicit responses to two hypothetical case vignettes. Here we present results from the first psychometric evaluation of the CPAM. Data were collected from a sample of 53 SPs across 24 public schools prior to initial didactic training, thus represent SP competency at baseline only. Nine items were dropped due to insufficient variability, resulting in 31 self-report items that comprise the overall scale (CE, UF, and P). Internal consistency of these items was high (Cronbach α=.996). Exploratory factor analysis indicated there was a general Clinical (CE+UF) factor (N=22; eigenvalue=27.7) and a second factor isolating the P items and 1 UF item (N=10; eigenvalue=1.8). Factors were highly correlated (r=0.81). To examine criterion validity of the Clinical factor (M=62.9, SD=19.1, range=23-103) and P scale scores (M=37.5, SD=5.3, range=28-50), linear regression examined their effects on the criterion-referenced vignette items (M=4.7, SD=2.2, range=0-9). Controls for age, sex, race, professional degree, number of years in current professional role, theoretical orientation, and prior CBT training were also included. After adjustment, the Perception scale was associated with better vignette scores, t(1)=2.90, p < .01, 95CI=(.014, .066), but effects were relatively small with an increase of 10 points on P scale reflecting an improvement of 1.8 points on vignette score (range: 0-9; adjusted correlation=.46). The CE+UF scale was not predictive (adj. correlation=.007).  These preliminary results will be further replicated and refined through data collected from a larger sample of SPs, but demonstrate that the CPAM holds promise for offering a reliable assessment of SP CBT competency for CBT delivery, providing a first step toward effective implementation and evaluation of TRAILS. 

Katherine Prenovost

Statistician/Psychometrician
Department of Psychiatry, University of Michigan Medical School, Ann Arbor, MI
Ann Arbor, Michigan

Shawna Smith

Research Investigator
Department of Psychiatry, University of Michigan Medical School, Ann Arbor, MI

Jennifer Vichich

Project Coordinator
Department of Psychiatry, University of Michigan Medical School, Ann Arbor, MI

Emily Berregaard

Project Coordinator
Department of Psychiatry, University of Michigan Medical School, Ann Arbor, MI

Elizabeth Koschmann

Research Investigator
Department of Psychiatry, University of Michigan Medical School, Ann Arbor, MI